PDAGENT-BENCH: Evaluating the Agent Capabilities of Large Models in Chip Physical Design

This article introduces PDAGENT-BENCH, the first comprehensive evaluation benchmark for LLM/VLM agents in the VLSI physical design field, covering 353 tasks and assessing model capabilities across five dimensions from conceptual understanding to full-process implementation, filling the gap in standardized evaluation for this domain.

Original Author/Maintainer: arXiv authors Source Platform: arXiv Original Title: PDAGENT-BENCH: Characterizing, Grounding, and Architecting LLM Agents for VLSI Physical Design Original Link: http://arxiv.org/abs/2606.17253v1 Publication Time: 2026-06-15T19:54:57Z

Chip design is the core infrastructure of modern technology. VLSI physical design involves complex tasks such as placement, routing, and timing optimization, which are traditionally completed by experienced engineers and EDA tools. In recent years, LLM/VLMs have performed well in chip front-end design (e.g., RTL code generation), but their application in physical design lags behind. The core reason is the lack of a standardized evaluation benchmark to measure the performance of agents in tool interaction and iterative optimization processes.

PDAGENT-BENCH is the first comprehensive evaluation benchmark for LLM/VLM agents in the VLSI physical design field. Its core concept combines "task-level evaluation" and "workflow-level execution", requiring agents to complete end-to-end tasks in a real EDA environment. It includes 353 tasks (conceptual questions + industrial cases) covering five capability dimensions:

1. Basic Knowledge: Testing basic concepts and principles of physical design
2. Report Understanding: Parsing timing, power, and other reports generated by EDA tools
3. Root Cause Analysis: Diagnosing design violations or performance bottlenecks and proposing recommendations
4. Script Generation: Generating Tcl/Python scripts for tools like Innovus
5. Full-Process Implementation: Completing the entire design flow from netlist to layout

Evaluations of 11 advanced LLM/VLM models show: They perform well in conceptual tasks, but there are large performance gaps in tool interaction and execution. For example, the accuracy of Innovus script generation is only 42.2%; models perform poorly in long-range multi-stage reasoning tasks and struggle to maintain coherent reasoning across stages.

Agent workflows enhanced with human skills significantly improve end-to-end physical design performance. At the current stage, human-agent collaboration is more optimal: LLMs excel at quickly generating candidate solutions and automating repetitive tasks; human engineers provide domain intuition, handle exceptions, and make strategic decisions.

PDAGENT-BENCH is a standardized and reproducible evaluation framework that defines unified agent physical design workflow specifications and supports closed-loop evaluation in real EDA environments. Its values include:

• Fair comparison of different methods
• Precisely identifying model capability shortcomings
• Continuously monitoring domain progress
• Facilitating integration with industrial EDA toolchains

PDAGENT-BENCH marks the deepening of LLM agent evaluation into professional domains. Future AI evaluations need to focus on "professional capabilities" (completing practical work using domain-specific toolchains). This benchmark reveals that tool interaction, long-range planning, and iterative optimization are core challenges that the next generation of agents need to overcome.