RankingNVIDIA Nemotron 3 Ultra

NVIDIA Nemotron 3 Ultra

Data updated 22 Oct 2026

47 published benchmark measures · 13 benchmark families contribute across 6 task areas. 3 capability estimates available in the full profile. See all results ↓ · Compare published benchmarks →

Model evidence summary

This profile combines published settings. It is not a runnable configuration or a leaderboard rank. Compare measured configurations →

Performance profile

Capabilities

Adjusted comparison score · 0–100

Filled points are supported; hollow points are preliminary. Lines stop at unknown capabilities. Exact values and sources follow below.

Agentic2 families · 1 with independent evidence · Supported2.7

2 core families; 45 direct opponents across 16 labs.

Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.

Refit without one family: 2.3–3.6; 2/13 scenarios unsupported. Without one publisher: 1.8–71.0; 1/15 unsupported. Smoothing check: 1.1–8.2; 0/3 unsupported. These are sensitivity checks, not confidence intervals.

Audit: family and publisher holdouts improved on neutral predictions within evaluated coverage; source sensitivity still requires inspection. Validation audit.

Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.

GDPval: 43.5 observed win share
NVIDIA, artificialanalysis.ai · Source 1 Source 2
Browsecomp: 25.0 observed win share
NVIDIA · Source 1

Hard reasoning3 families · 0 with independent evidence · Supported13.3

3 core families; 5 direct opponents across 5 labs.

Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.

Refit without one family: 4.0–25.4; 0/6 scenarios unsupported. Without one publisher: 3.2–53.8; 1/16 unsupported. Smoothing check: 5.5–25.8; 0/3 unsupported. These are sensitivity checks, not confidence intervals.

Audit: family and publisher holdouts improved on neutral predictions within evaluated coverage. Validation audit.

Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.

GPQA: 25.0 observed win share
NVIDIA, artificialanalysis.ai · Source 1 Source 2
Humanity’s Last Exam: 12.5 observed win share
NVIDIA · Source 1 Source 2
Imo Answerbench: 75.0 observed win share
NVIDIA · Source 1 Source 2

Coding3 families · 0 with independent evidence · Supported1.7

3 core families; 4 direct opponents across 4 labs.

Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.

Refit without one family: 0.4–16.4; 0/6 scenarios unsupported. Without one publisher: 1.0–7.3; 1/19 unsupported. Smoothing check: 0.8–5.5; 0/3 unsupported. These are sensitivity checks, not confidence intervals.

Audit: family and publisher holdouts improved on neutral predictions within evaluated coverage. Validation audit.

Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.

LiveCodeBench: 75.0 observed win share
NVIDIA · Source 1
SWE-bench Multilingual: 0.0 observed win share
NVIDIA · Source 1
SWE-bench Verified: 0.0 observed win share
NVIDIA · Source 1

Human pref1 families · 1 with independent evidence · PreliminaryUnknown

1 core families; 41 direct opponents across 13 labs. No connected comparison to the complete reference panel

Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.

Refit without one family: No supported estimate; 1/1 scenarios unsupported. Without one publisher: No supported estimate; 1/1 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.

Audit: cross-family and cross-publisher predictions could not be evaluated from the available comparison network. Validation audit.

Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.

Human preference (Arena): 82.9 observed win share
lmarena.ai · Source 1

Knowledge1 families · 0 with independent evidence · PreliminaryUnknown

1 core families; 4 direct opponents across 4 labs. No connected comparison to the complete reference panel

Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.

Refit without one family: No supported estimate; 4/4 scenarios unsupported. Without one publisher: No supported estimate; 5/5 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.

Audit: almost no publisher-held-out comparisons remained estimable. Cross-source validity is unresolved. Validation audit.

Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.

MMLU-Pro: 37.5 observed win share
NVIDIA · Source 1 Source 2

MultimodalNo comparable evidenceUnknown

0 core families; 0 direct opponents across 0 labs. No admitted core comparisons

Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.

Refit without one family: No supported estimate; 8/8 scenarios unsupported. Without one publisher: No supported estimate; 8/8 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.

Audit: limited publisher-held-out coverage. Cross-source validity is unresolved. Validation audit.

Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.

Long context3 families · 0 with independent evidence · PreliminaryUnknown

3 core families; 4 direct opponents across 4 labs. No connected comparison to the complete reference panel

Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.

Refit without one family: No supported estimate; 5/5 scenarios unsupported. Without one publisher: No supported estimate; 5/5 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.

Audit: family and publisher holdouts underperformed neutral predictions. Treat this as an exploratory estimate. Validation audit.

Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.

Aa Lcr: 0.0 observed win share
NVIDIA · Source 1
Longbench: 0.0 observed win share
NVIDIA · Source 1
Ruler 1M: 100.0 observed win share
NVIDIA · Source 1

Scores estimate outcomes against a shared reference panel; they are not accuracy percentages. Sparse or disconnected evidence cannot qualify an overall profile. Open a capability to inspect its evidence.

Reported effort · Thinking + unspecified

Settings reported in this model's published benchmark results, including results outside the aggregate. Effort names are provider-specific. These are not API defaults or equal compute budgets.

Thinking: 12 observations
Not specified: 39 observations

Mixed settings means multiple settings occur in the evidence. Best across efforts means the source selected its best reported result across settings; it does not mean Max. Unspecified settings stay unknown. This model-summary chart combines reported settings. The leaderboard keeps identified configurations separate and excludes unknown effort.

Inspect each result and its source ↓ · Download effort evidence

Score contributions and missing evidence

13 contributing families across 6 capabilities. Fixed reference panels do not change when the catalog expands.

Capability is fitted jointly across families. Capability estimates below describe different task areas; their weighted sum is not the Capability score.

Results without reviewed compatibility or a reference match remain in the raw evidence below. Coverage counts only contributing results.

Model information & shareable badge

Lab: NVIDIA
Catalog status: open-weights-available
Availability: Open weights available for self-hosting
Family: Nemotron 3
Released: —
Context: —
License: —
Model card: https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16
Default Capability family coverage: /badge/nemotron-3-ultra.svg

Benchmark scores & sources

Original results, evaluation harnesses, and evidence behind this model.

Benchmark	Bucket	Score	Harness	Evidence	Source-recorded date
LMArena Text Arena	Human pref	1426	LMArena Text Effort: Not specified contributes to capability	official board	2026-10-15
Longbench v2 (≤ 1M) · source release snapshot; version not specified	Long context	61.9% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 61.9 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: Longbench v2 (≤ 1M) · reviewed 2026-09-06	Published configuration Effort: Not specified contributes to capability	lab self-report	Reviewed 2026-09-06
MMLU-Pro	Knowledge	86.8%	MMLU-Pro reported Effort: Not specified	lab self-report Legacy self-report lacks a reviewed comparison configuration	2026-06-04
MMLU-Pro · source release snapshot; version not specified	Knowledge	86.8% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 86.8 The linked NVIDIA reproduction recipe explicitly enables thinking for this model and benchmark. It does not establish Low/Medium/High/Max or a fixed reasoning-token budget; the request adapter removes max_tokens and max_completion_tokens. Comparator settings are not specified by this recipe. nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: MMLU-Pro · reviewed 2026-09-06	Published configuration Effort: Thinking contributes to capability	lab self-report	Reviewed 2026-09-06
MMLU-ProX (avg en/de/fr/es/it/ja/zh/hi/pt/ko) · source release snapshot; version not specified	Knowledge	83% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 83.0 The linked NVIDIA reproduction recipe explicitly enables thinking for this model and benchmark. It does not establish Low/Medium/High/Max or a fixed reasoning-token budget; the request adapter removes max_tokens and max_completion_tokens. Comparator settings are not specified by this recipe. nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: MMLU-ProX (avg en/de/fr/es/it/ja/zh/hi/pt/ko) · reviewed 2026-09-06	Published configuration Effort: Thinking contributes to capability	lab self-report	Reviewed 2026-09-06
Multi-Challenge · source release snapshot; version not specified	Supporting evidence	63.8% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 63.8 The linked NVIDIA reproduction recipe explicitly enables thinking for this model and benchmark. It does not establish Low/Medium/High/Max or a fixed reasoning-token budget; the request adapter removes max_tokens and max_completion_tokens. Comparator settings are not specified by this recipe. nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: Multi-Challenge · reviewed 2026-09-06	Published configuration Effort: Thinking	lab self-report Supporting evidence outside the reviewed capability core	Reviewed 2026-09-06
OmniScience Accuracy · source release snapshot; version not specified	Supporting evidence	24.1% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 24.1 The linked NVIDIA reproduction recipe explicitly enables thinking for this model and benchmark. It does not establish Low/Medium/High/Max or a fixed reasoning-token budget; the request adapter removes max_tokens and max_completion_tokens. Comparator settings are not specified by this recipe. nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: OmniScience Accuracy · reviewed 2026-09-06	Published configuration Effort: Thinking	lab self-report Narrow specialist biological, medical or security evaluation; preserved as raw evidence outside general capability scoring.	Reviewed 2026-09-06
PinchBench · source release snapshot; version not specified	Supporting evidence	90% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 90.0 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: PinchBench · reviewed 2026-09-06	Published configuration Effort: Not specified	lab self-report Supporting evidence outside the reviewed capability core	Reviewed 2026-09-06
ProfBench (Search) · source release snapshot; version not specified	Supporting evidence	56% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 56.0 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: ProfBench (Search) · reviewed 2026-09-06	Published configuration Effort: Not specified	lab self-report Supporting evidence outside the reviewed capability core	Reviewed 2026-09-06
RULER (1M) · source release snapshot; version not specified	Long context	94.7% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 94.7 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: RULER (1M) · reviewed 2026-09-06	Published configuration Effort: Not specified contributes to capability	lab self-report	Reviewed 2026-09-06
SciCode (subtask) · source release snapshot; version not specified	Coding	44.6% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 44.6 The linked NVIDIA reproduction recipe explicitly enables thinking for this model and benchmark. It does not establish Low/Medium/High/Max or a fixed reasoning-token budget; the request adapter removes max_tokens and max_completion_tokens. Comparator settings are not specified by this recipe. nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: SciCode (subtask) · reviewed 2026-09-06	Published configuration Effort: Thinking	lab self-report Original SciCode grading defects are under review; retained as supporting evidence, excluded from aggregate and capability scores. Corrected SciCode-Verified requires a separate protocol review. Sources: https://arxiv.org/abs/2608.04975 and https://artificialanalysis.ai/evaluations/scicode	Reviewed 2026-09-06
SWE-Bench Multilingual · source release snapshot; version not specified	Coding	67.7% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 67.7 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: SWE-Bench Multilingual · reviewed 2026-09-06	Published configuration Effort: Not specified contributes to capability	lab self-report	Reviewed 2026-09-06
SWE-bench Verified	Coding	71.9%	SWE-bench Verified official Effort: Not specified	lab self-report Legacy self-report lacks a reviewed comparison configuration	2026-06-04
SWE-Bench Verified · source release snapshot; version not specified	Coding	70.7% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 70.7 The linked NVIDIA reproduction recipe explicitly enables thinking for this model and benchmark. It does not establish Low/Medium/High/Max or a fixed reasoning-token budget; the request adapter removes max_tokens and max_completion_tokens. Comparator settings are not specified by this recipe. nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: SWE-Bench Verified · reviewed 2026-09-06	Published configuration Effort: Thinking contributes to capability	lab self-report	Reviewed 2026-09-06
TauBench V3 Airline · source release snapshot; version not specified	Supporting evidence	81.5% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 81.5 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: Airline · reviewed 2026-09-06	Published configuration Effort: Not specified	lab self-report Supporting evidence outside the reviewed capability core	Reviewed 2026-09-06
TauBench V3 Average · source release snapshot; version not specified	Supporting evidence	70.9% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 70.9 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: Average · reviewed 2026-09-06	Published configuration Effort: Not specified	lab self-report Supporting evidence outside the reviewed capability core	Reviewed 2026-09-06
TauBench V3 Banking · source release snapshot; version not specified	Supporting evidence	22.6% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 22.6 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: Banking · reviewed 2026-09-06	Published configuration Effort: Not specified	lab self-report Supporting evidence outside the reviewed capability core	Reviewed 2026-09-06
TauBench V3 Retail · source release snapshot; version not specified	Supporting evidence	86.4% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 86.4 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: Retail · reviewed 2026-09-06	Published configuration Effort: Not specified	lab self-report Supporting evidence outside the reviewed capability core	Reviewed 2026-09-06
TauBench V3 Telecom · source release snapshot; version not specified	Supporting evidence	92.9% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 92.9 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: Telecom · reviewed 2026-09-06	Published configuration Effort: Not specified	lab self-report Supporting evidence outside the reviewed capability core	Reviewed 2026-09-06
Terminal Bench 2.1 · 2.1	Coding	56.4% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 56.4 The linked public reproduction recipe names Terminal Bench 2.0, while this card labels 2.1. Recipe settings are not transferred across that unresolved version mismatch. nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: Terminal Bench 2.1 · reviewed 2026-09-06	Published configuration Effort: Not specified	lab self-report Terminal-Bench coding slot admits version 4.0 only. Earlier versions remain stored and are not mixed into that slot.	Reviewed 2026-09-06
Terminal-Bench 2.1	Coding	56.4%	Terminal-Bench 2.1 reported Effort: Not specified	lab self-report Legacy self-report lacks a reviewed comparison configuration	2026-06-04
Toolathlon-Verified (official board)	Supporting evidence	34.3%	Toolathlon-Verified Default agent Effort: Not specified	official board Supporting evidence outside the reviewed capability core	2026-07-15
Vals.ai Financial Agent 1.1 with web search · 1.1	Supporting evidence	53.7% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 53.7 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: with web search · reviewed 2026-09-06	Published configuration Effort: Not specified	lab self-report Supporting evidence outside the reviewed capability core	Reviewed 2026-09-06
Vals.ai Financial Agent 1.1 without web search · 1.1	Supporting evidence	60.1% Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 60.1 nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: without web search · reviewed 2026-09-06	Published configuration Effort: Not specified	lab self-report Supporting evidence outside the reviewed capability core	Reviewed 2026-09-06
WMT24++ (en→xx) · source release snapshot; version not specified	Supporting evidence	83.7 score (source scale) Reported settings & source NVIDIA evaluation harness/settings per benchmark in model card; comparator scores are NVIDIA-reported under that protocol. First-party reported result. Original cell: 83.7 The linked NVIDIA reproduction recipe explicitly enables thinking for this model and benchmark. It does not establish Low/Medium/High/Max or a fixed reasoning-token budget; the request adapter removes max_tokens and max_completion_tokens. Comparator settings are not specified by this recipe. nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 · Performance table: WMT24++ (en→xx) · reviewed 2026-09-06	Published configuration Effort: Thinking	lab self-report Supporting evidence outside the reviewed capability core	Reviewed 2026-09-06