Ranking / AutomationBench-AA

AutomationBench-AA

Data updated 23 Sept 2026

Bucket

Supporting evidence

Unit

percent

Direction

Higher is better

Version

aa-score

Display harness

Artificial Analysis AutomationBench-AA

Board

https://artificialanalysis.ai/evaluations/automationbench-aa

AutomationBench-AA score: share of task objectives completed with no guardrail violation, on Artificial Analysis's private 657-task holdout. This is not Zapier's tasks-completed rate. The effort dataset already includes this evaluation in the agentic family under the existing equal-family budget. These rows are the board, not a second weight.

Compare published benchmark results with category weights →

Models

Results are ordered by score in the display harness, followed by models with no result. Other harnesses are listed separately below. Missing results remain unknown.

226–250 of 455 entries