RankingMMLU-Pro · source release snapshot; version not specified
MMLU-Pro · source release snapshot; version not specified
- Bucket
- Knowledge
- Unit
- percent
- Direction
- Higher is better
- Version
- source release snapshot; version not specified
- Display harness
- nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16
- Board
- https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16
Compare published benchmark results with category weights →
Models
| Model | Score | Harness | Evidence | Source-recorded date |
|---|---|---|---|---|
| Qwen3.5-397B-A17BQwen | 88.3%Reported settings & sourceNVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 88.3 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: MMLU-Pro · reviewed 2026-09-06 | nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 | lab self-report | 2026-09-06 |
| Kimi-K2.6Moonshot | 88.1%Reported settings & sourceNVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 88.1 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: MMLU-Pro · reviewed 2026-09-06 | nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 | lab self-report | 2026-09-06 |
| NVIDIA Nemotron 3 UltraNVIDIA | 86.8%Reported settings & sourceNVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 86.8 The linked NVIDIA reproduction recipe explicitly enables thinking for this model and benchmark. It does not establish Low/Medium/High/Max or a fixed reasoning-token budget; the request adapter removes max_tokens and max_completion_tokens. Comparator settings are not specified by this recipe. nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: MMLU-Pro · reviewed 2026-09-06 | nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 | lab self-report | 2026-09-06 |
| GLM-5.1Z.ai | 85.9%Reported settings & sourceNVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 85.9 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: MMLU-Pro · reviewed 2026-09-06 | nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 | lab self-report | 2026-09-06 |
| MiniMax-M2.7MiniMax | 81.9%Reported settings & sourceNVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 81.9 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: MMLU-Pro · reviewed 2026-09-06 | nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 | lab self-report | 2026-09-06 |
| Amazon Nova 2 LiteAmazon | 80.9%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Amazon Nova 2 Pro PreviewAmazon | 81.6%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Amazon Nova LiteAmazon | 59%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Amazon Nova PremierAmazon | 73.3%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Amazon Nova ProAmazon | 69.1%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Claude Haiku 4.5Anthropic | 80%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Claude Sonnet 4.5Anthropic | 87.5%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Gemini 2.5 FlashGoogle | 83.2%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| Gemini 2.5 ProGoogle | 86.2%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| GPT-5OpenAI | 87.1%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| GPT-5 MiniOpenAI | 83.7%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |
| GPT-5.1OpenAI | 87%Reported settings & sourceNova2 launch evaluation; benchmark-specific settings in report. tau2 Verified avg@3; telecom cited AA because user model sensitivity; comparators mix own runs and cited provider scores. First-party reported result; comparator results retain the source evaluation setup. Amazon Nova 2 technical report · Table1 · reviewed 2026-09-06 | Amazon Nova 2 technical report | lab self-report | 2026-09-06 |