RankingAIME 2025 · source release snapshot; version not specified
AIME 2025 · source release snapshot; version not specified
- Bucket
- Hard reasoning
- Unit
- percent
- Direction
- Higher is better
- Version
- source release snapshot; version not specified
- Display harness
- MAI-Thinking-1 technical report
- Board
- https://microsoft.ai/pdf/mai-thinking-1.pdf
Compare published benchmark results with category weights →
Models
| Model | Score | Harness | Evidence | Source-recorded date |
|---|---|---|---|---|
| Claude Opus 4.6Anthropic | 99.8%Reported settings & sourceMAI avg 4 runs, temperature 1, top-p .97; simple ReAct bash/string-replace; Terminal-Bench ignores predefined timeouts. Comparator configurations from cited reports. First-party reported result; comparator results retain the source evaluation setup. MAI-Thinking-1 technical report · Table 11, PDF page 53 · reviewed 2026-09-06 | MAI-Thinking-1 technical report | lab self-report | 2026-09-06 |
| MAI-Thinking-1Microsoft | 97%Reported settings & sourceMAI avg 4 runs, temperature 1, top-p .97; simple ReAct bash/string-replace; Terminal-Bench ignores predefined timeouts. Comparator configurations from cited reports. First-party reported result; comparator results retain the source evaluation setup. MAI-Thinking-1 technical report · Table 11, PDF page 53 · reviewed 2026-09-06 | MAI-Thinking-1 technical report | lab self-report | 2026-09-06 |
| Claude Sonnet 4.6Anthropic | 95.6%Reported settings & sourceMAI avg 4 runs, temperature 1, top-p .97; simple ReAct bash/string-replace; Terminal-Bench ignores predefined timeouts. Comparator configurations from cited reports. First-party reported result; comparator results retain the source evaluation setup. MAI-Thinking-1 technical report · Table 11, PDF page 53 · reviewed 2026-09-06 | MAI-Thinking-1 technical report | lab self-report | 2026-09-06 |
| Amazon Nova 2 LiteAmazon | 91%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Amazon Nova 2 Pro PreviewAmazon | 92.3%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Amazon Nova LiteAmazon | 7%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Amazon Nova PremierAmazon | 17.3%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Amazon Nova ProAmazon | 7%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Claude Haiku 4.5Anthropic | 80.7%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Claude Sonnet 4.5Anthropic | 87%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Gemini 2.5 FlashGoogle | 72%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Gemini 2.5 ProGoogle | 88%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| GPT-5OpenAI | 94.6%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| GPT-5 MiniOpenAI | 91.1%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| GPT-5.1OpenAI | 94%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |