UnifyBench

RankingAutomationBench-AA

AutomationBench-AA

Data updated 23 Sept 2026

Bucket
Supporting evidence
Unit
percent
Direction
Higher is better
Version
aa-score
Display harness
Artificial Analysis AutomationBench-AA
Board
https://artificialanalysis.ai/evaluations/automationbench-aa

AutomationBench-AA score: share of task objectives completed with no guardrail violation, on Artificial Analysis's private 657-task holdout. This is not Zapier's tasks-completed rate. The effort dataset already includes this evaluation in the agentic family under the existing equal-family budget. These rows are the board, not a second weight.

Compare published benchmark results with category weights →

Models

Results are ordered by score in the display harness, followed by models with no result. Other harnesses are listed separately below. Missing results remain unknown.

1–25 of 455 entries

ModelScoreEvidenceSource-recorded date
Claude Opus 5.5Anthropic69.5386812863607%official board2026-09-23
GPT-6 AstraOpenAI68.49174806688876%official board2026-09-23
Grok 4.6xAI66.67849638121398%official board2026-09-23
Grok 4.7xAI65.55835148969457%official board2026-09-23
GPT-6 SolOpenAI61.63369295959864%official board2026-09-23
GPT-5.6 SolOpenAI60.08114996276329%official board2026-09-23
Claude Fable 5.1Anthropic59.37591715646424%official board2026-09-23
Kimi K3Moonshot58.272977800152034%official board2026-09-23
DeepSeek V4-Pro 0813DeepSeek56.711655463440025%official board2026-09-23
Claude Opus 5Anthropic56.57355576493251%official board2026-09-23
DeepSeek V4-Pro 0424DeepSeek56.289685574342705%official board2026-09-23
Qwen 3.8-MaxQwen56.1905381087678%official board2026-09-23
Qwen3.8-Flash-NextQwen55.9083218951688%official board2026-09-23
Claude Fable 5Anthropic54.071301140653816%official board2026-09-23
DeepSeek-V4-Flash-0731DeepSeek53.96771855133674%official board2026-09-23
GPT-6 LunaOpenAI53.16814034483175%official board2026-09-23
Claude Opus 4.8Anthropic45.59343691527719%official board2026-09-23
Agnes 2.5 Pro AlphaAgnes AI38.43295195770323%official board2026-09-23
Claude Sonnet 5Anthropic36.51267571703639%official board2026-09-23
Gemini 3.1 ProGoogle35.427718398098364%official board2026-09-23
Gemini 3.5 Flash-LiteGoogle25.011270511840955%official board2026-09-23
MiniMax-M3MiniMax21.25125760074941%official board2026-09-23
MiMo-V2.5Xiaomi18.42951233004315%official board2026-09-23
Hy3Tencent18.409128371172397%official board2026-09-23
MiMo-V2.5-ProXiaomi13.703616508004313%official board2026-09-23