RankingMCPAtlas · source release snapshot; version not specified
MCPAtlas · source release snapshot; version not specified
- Bucket
- Agentic
- Unit
- percent
- Direction
- Higher is better
- Version
- source release snapshot; version not specified
- Display harness
- Amazon Nova 2 technical report
- Board
- https://cdn.amazon.science/c5/3d/84514a224666b5be6de4b43ef4aa/nova-2-0-technical-report2.pdf
Compare published benchmark results with category weights →
Models
| Model | Score | Harness | Evidence | Source-recorded date |
|---|---|---|---|---|
| GPT-5OpenAI | 44.5%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table2 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Claude Sonnet 4.5Anthropic | 43.8%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table2 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Amazon Nova 2 LiteAmazon | 24.6%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table2 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| GPT-5 MiniOpenAI | 22.2%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table2 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Gemini 2.5 ProGoogle | 8.8%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table2 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Claude Opus 4.7Anthropic | 77%Reported settings & sourceMiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted. First-party reported result; comparator results retain the source evaluation setup. MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06 | MiniMax M3 model card benchmark figure | lab self-report | 2026-09-06 |
| Claude Opus 4.8Anthropic | 82.2%Reported settings & sourceMeta Model API xhigh; Gemini high; Opus max; GPT xhigh. Best of self-reported or Meta reproduction unless specified. OSWorld GUI only; Terminal bash only5attempts89tasks6CPU8GB; DeepSWE no internet5attempts. First-party reported result; comparator results retain the source evaluation setup. Muse Spark 1.1 evaluation report Figure44 · Figure44 physical page101; methodology pages101–105 · reviewed 2026-09-06 | Muse Spark 1.1 evaluation report Figure44 | lab self-report | 2026-09-06 |
| Claude Sonnet 4.6Anthropic | 61.3%Reported settings & sourceMiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted. First-party reported result; comparator results retain the source evaluation setup. MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06 | MiniMax M3 model card benchmark figure | lab self-report | 2026-09-06 |
| Gemini 3.1 ProGoogle | 69.2%Reported settings & sourceMiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted. First-party reported result; comparator results retain the source evaluation setup. MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06 | MiniMax M3 model card benchmark figure | lab self-report | 2026-09-06 |
| Gemini 3.1 ProGoogle | 78.2%Reported settings & sourceMeta Model API xhigh; Gemini high; Opus max; GPT xhigh. Best of self-reported or Meta reproduction unless specified. OSWorld GUI only; Terminal bash only5attempts89tasks6CPU8GB; DeepSWE no internet5attempts. First-party reported result; comparator results retain the source evaluation setup. Muse Spark 1.1 evaluation report Figure44 · Figure44 physical page101; methodology pages101–105 · reviewed 2026-09-06 | Muse Spark 1.1 evaluation report Figure44 | lab self-report | 2026-09-06 |
| GLM-5.1Z.ai | 71.8%Reported settings & sourceMiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted. First-party reported result; comparator results retain the source evaluation setup. MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06 | MiniMax M3 model card benchmark figure | lab self-report | 2026-09-06 |
| GPT-5.5OpenAI | 75.3%Reported settings & sourceMiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted. First-party reported result; comparator results retain the source evaluation setup. MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06 | MiniMax M3 model card benchmark figure | lab self-report | 2026-09-06 |
| GPT-5.5OpenAI | 75.3%Reported settings & sourceMeta Model API xhigh; Gemini high; Opus max; GPT xhigh. Best of self-reported or Meta reproduction unless specified. OSWorld GUI only; Terminal bash only5attempts89tasks6CPU8GB; DeepSWE no internet5attempts. First-party reported result; comparator results retain the source evaluation setup. Muse Spark 1.1 evaluation report Figure44 · Figure44 physical page101; methodology pages101–105 · reviewed 2026-09-06 | Muse Spark 1.1 evaluation report Figure44 | lab self-report | 2026-09-06 |
| Kimi-K2.6Moonshot | 66.6%Reported settings & sourceMiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted. First-party reported result; comparator results retain the source evaluation setup. MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06 | MiniMax M3 model card benchmark figure | lab self-report | 2026-09-06 |
| MiniMax-M2.7MiniMax | 49.4%Reported settings & sourceMiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted. First-party reported result; comparator results retain the source evaluation setup. MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06 | MiniMax M3 model card benchmark figure | lab self-report | 2026-09-06 |
| MiniMax-M3MiniMax | 74.2%Reported settings & sourceMiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted. First-party reported result; comparator results retain the source evaluation setup. MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06 | MiniMax M3 model card benchmark figure | lab self-report | 2026-09-06 |
| Muse Spark 1.1Meta | 88.1%Reported settings & sourceMeta Model API xhigh; Gemini high; Opus max; GPT xhigh. Best of self-reported or Meta reproduction unless specified. OSWorld GUI only; Terminal bash only5attempts89tasks6CPU8GB; DeepSWE no internet5attempts. First-party reported result; comparator results retain the source evaluation setup. Muse Spark 1.1 evaluation report Figure44 · Figure44 physical page101; methodology pages101–105 · reviewed 2026-09-06 | Muse Spark 1.1 evaluation report Figure44 | lab self-report | 2026-09-06 |