Evidence coverage
3498 published observations across 337 measures. Capability scoring groups compatible evidence into 44 benchmark families.
A published result contributes only when a matching reference-panel result is available. Missing evidence stays unknown. Coverage describes breadth, separately from performance.
Coverage across our labs
| Lab | Models with results | Published measures | Launch sources |
|---|---|---|---|
| OpenAI | 34 / 40 | 241 | 20 |
| Anthropic | 14 / 14 | 251 | 22 |
| xAI | 4 / 8 | 35 | 2 |
| 22 / 35 | 133 | 9 | |
| Meta | 14 / 16 | 46 | 2 |
| Mistral | 17 / 28 | 39 | 2 |
| Moonshot | 3 / 7 | 133 | 7 |
| Qwen | 32 / 82 | 104 | 4 |
| DeepSeek | 7 / 16 | 33 | 2 |
| NVIDIA | 2 / 10 | 47 | 1 |
| Z.ai | 17 / 52 | 105 | 7 |
| MiniMax | 6 / 14 | 69 | 2 |
| Cohere | 10 / 26 | 18 | 1 |
| Amazon | 5 / 6 | 20 | 1 |
| Microsoft | 9 / 24 | 19 | 1 |
| Xiaomi | 5 / 14 | 10 | 0 |
| StepFun | 3 / 5 | 5 | 0 |
| LG AI Research | 5 / 13 | 3 | 0 |
| Upstage | 5 / 9 | 4 | 0 |
| ByteDance | 1 / 16 | 1 | 0 |
| Tencent | 2 / 13 | 7 | 0 |
| Baidu | 1 / 20 | 1 | 0 |
| OpenBMB | 1 / 1 | 2 | 0 |
| Agnes AI | 1 / 2 | 2 | 0 |
Models and contributing families
Raw published counts include all source types. Contributing and independent family counts follow the selected evidence policy.
16 models match “Meta”. Clear search
Family coverage
| Benchmark family | Capability | Models contributing | With independent support |
|---|---|---|---|
| OSWorld | Agentic | 21 | 6 |
| Webarena Verified | Agentic | 4 | 0 |
| Browsecomp | Agentic | 22 | 0 |
| Toolathlon | Agentic | 15 | 0 |
| MCP-Atlas | Agentic | 19 | 0 |
| AutomationBench | Agentic | 21 | 0 |
| DeepSearchQA | Agentic | 10 | 0 |
| GDPval | Agentic | 64 | 45 |
| Bfcl V4 | Agentic | 12 | 0 |
| Apex Agents | Agentic | 10 | 0 |
| OfficeQA | Agentic | 11 | 0 |
| GDP.pdf | Agentic | 14 | 0 |
| Terminal-Bench Science | Agentic | 7 | 0 |
| Humanity’s Last Exam | Hard reasoning | 32 | 9 |
| GPQA | Hard reasoning | 45 | 18 |
| ARC-AGI | Hard reasoning | 60 | 60 |
| Aime | Hard reasoning | 22 | 0 |
| FrontierMath | Hard reasoning | 10 | 0 |
| Imo Answerbench | Hard reasoning | 5 | 0 |
| SWE-bench Verified | Coding | 38 | 19 |
| SWE-bench Pro | Coding | 20 | 4 |
| SWE-bench Multilingual | Coding | 9 | 0 |
| DeepSWE | Coding | 33 | 28 |
| LiveCodeBench | Coding | 25 | 9 |
| Terminal-Bench | Coding | 22 | 22 |
| Scicode | Coding | 0 | 0 |
| Human preference (Arena) | Human pref | 119 | 119 |
| MMLU-Pro | Knowledge | 68 | 52 |
| Gmmlu | Knowledge | 3 | 0 |
| Milu | Knowledge | 3 | 0 |
| Simpleqa Verified | Knowledge | 2 | 0 |
| Mmmu | Multimodal | 2 | 0 |
| Mmmu Pro | Multimodal | 24 | 0 |
| CharXiv | Multimodal | 16 | 0 |
| Mathvista | Multimodal | 2 | 0 |
| Chartography | Multimodal | 3 | 0 |
| Lvbench | Multimodal | 6 | 0 |
| ScreenSpot | Multimodal | 12 | 0 |
| Ocrbench V2 Average Accuracy | Multimodal | 12 | 0 |
| MRCR | Long context | 12 | 0 |
| Longbench | Long context | 8 | 0 |
| Ruler 1M | Long context | 2 | 0 |
| Graphwalks | Long context | 7 | 0 |
| Aa Lcr | Long context | 5 | 0 |
Model evidence matrix
| Model | OSWorld | Webarena Verified | Browsecomp | Toolathlon | MCP-Atlas | AutomationBench | DeepSearchQA | GDPval | Bfcl V4 | Apex Agents | OfficeQA | GDP.pdf | Terminal-Bench Science | Humanity’s Last Exam | GPQA | ARC-AGI | Aime | FrontierMath | Imo Answerbench | SWE-bench Verified | SWE-bench Pro | SWE-bench Multilingual | DeepSWE | LiveCodeBench | Terminal-Bench | Scicode | Human preference (Arena) | MMLU-Pro | Gmmlu | Milu | Simpleqa Verified | Mmmu | Mmmu Pro | CharXiv | Mathvista | Chartography | Lvbench | ScreenSpot | Ocrbench V2 Average Accuracy | MRCR | Longbench | Ruler 1M | Graphwalks | Aa Lcr |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Muse Spark 1.1 | 57.8 | 50.0 | 66.7 | 100.0 | 66.7 | 33.3 | 100.0 | 83.3 | 30.6 | 96.1 | 66.7 | 50.0 | ||||||||||||||||||||||||||||||||
| Muse Spark 1.3 | 75.0 | 75.0 | 50.0 | 75.0 | 100.0 | 100.0 | 14.3 | 100.0 | ||||||||||||||||||||||||||||||||||||
| Llama 4 Maverick | 0.0 | 25.0 | 3.4 | 0.0 | 58.5 | 58.8 | ||||||||||||||||||||||||||||||||||||||
| Muse Spark 1.2 | 0.0 | 0.0 | 0.0 | 40.7 | 0.0 | |||||||||||||||||||||||||||||||||||||||
| Llama-3.1-70B-Instruct | 43.9 | 31.4 | ||||||||||||||||||||||||||||||||||||||||||
| Llama-3.1-8B-Instruct | 24.4 | 11.8 | ||||||||||||||||||||||||||||||||||||||||||
| Llama-3.3-70B-Instruct | 53.7 | 37.3 | ||||||||||||||||||||||||||||||||||||||||||
| Llama-4-Scout-17B-16E-Instruct | 56.1 | 48.0 | ||||||||||||||||||||||||||||||||||||||||||
| Llama-3.1-405B-Instruct | 45.1 | |||||||||||||||||||||||||||||||||||||||||||
| Llama-3.2-1B-Instruct | 7.3 | |||||||||||||||||||||||||||||||||||||||||||
| Llama-3.2-3B-Instruct | 14.6 | |||||||||||||||||||||||||||||||||||||||||||
| Llama-3.2-11B-Vision-Instruct | ||||||||||||||||||||||||||||||||||||||||||||
| Llama-3.2-90B-Vision-Instruct | ||||||||||||||||||||||||||||||||||||||||||||
| Muse Glimmer 30B | ||||||||||||||||||||||||||||||||||||||||||||
| Muse Wonder | ||||||||||||||||||||||||||||||||||||||||||||
| Muse Wonder Full |
Family definitions and scoring rules · Compare capabilities · Compare published evidence