01
Read
Receive an abstract or detailed English design description with no reference program or hidden geometry.
Complete public GT gallery
All 2,409 models. One explorer.
Browse every rendered ground-truth model across T1–T4 and the long-horizon Challenge. Search by model ID or description, combine tier and split filters, then open any card for a closer look.
Executable evaluation
From language to kernel-valid CAD.
The benchmark scores an agent inside a real construction loop. It must interpret intent, author a feature history, react to structured diagnostics, and finish with valid geometry.
02
Construct
Create sketches, profiles, solids, booleans, fillets, chamfers, shells, and dependent features through tools.
03
Recover
Use task-independent kernel diagnostics to repair invalid loops, stale selections, or failed operations.
04
Verify
Validate the final B-Rep and score aligned exterior shape, dimensions, topology, program, and interaction behavior.
Evaluation stack
Success is more than “it ran.”
ParamCAD-AgentBench separates construction validity from geometric agreement and agent behavior, so a lucky valid solid cannot hide a poor reconstruction.
Primary
Aligned shape success
Compares exterior shape after translation, rotation, and uniform-scale alignment. This is the headline cross-model score.
Strict geometry
Dimensioned success
Preserves target scale to distinguish visually similar constructions from dimensionally faithful ones.
Execution
Valid B-Rep rate
Measures whether an episode terminates with a non-empty, kernel-valid solid rather than a partial or corrupt model.
Behavior
Interaction reliability
Reports tool errors, recovery, termination, latency, and token usage alongside feature-program similarity.
Leakage-aware release
Human-browsable GT, evaluation-safe files.
The project page publishes all rendered model previews for inspection. Test and Challenge STEP files, B-Reps, oracle programs, and source records remain withheld. Official evaluation must also run network-isolated, without exposing the gallery or model identifiers to the evaluated agent.
Dev400 modelsoracle included
Test1,600 modelsoracle withheld
Challenge409 modelsoracle withheld
Current release
v2.2.0-preview is live.
Download the public benchmark, reproduce the protocol, and follow the evidence-gated 2,409-model annotation refresh as it moves toward the audited follow-up release.
2,409 MODELS · 4,818 TASKS
Get the release →