Filled points are supported; hollow points are preliminary. Lines stop at unknown capabilities. Exact values and sources follow below.
Agentic1 families · 0 with independent evidence · Preliminary0.5
1 core families; 11 direct opponents across 4 labs. Needs broader benchmark and opponent support
Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.
Refit without one family: No supported estimate; 13/13 scenarios unsupported. Without one publisher: No supported estimate; 15/15 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.
Audit: family and publisher holdouts improved on neutral predictions within evaluated coverage; source sensitivity still requires inspection. Validation audit.
Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.
- Bfcl V4: 77.3 observed win share
Amazon · Source 1
Hard reasoning2 families · 0 with independent evidence · Supported16.1
2 core families; 11 direct opponents across 4 labs.
Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.
Refit without one family: 12.1–24.3; 2/6 scenarios unsupported. Without one publisher: 10.7–17.3; 1/16 unsupported. Smoothing check: 6.8–29.2; 0/3 unsupported. These are sensitivity checks, not confidence intervals.
Audit: family and publisher holdouts improved on neutral predictions within evaluated coverage. Validation audit.
Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.
- Aime: 81.8 observed win share
Amazon · Source 1 - GPQA: 45.5 observed win share
Amazon · Source 1
Coding2 families · 0 with independent evidence · Supported1.1
2 core families; 9 direct opponents across 4 labs.
Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.
Refit without one family: 0.5–6.0; 2/6 scenarios unsupported. Without one publisher: 0.7–5.4; 1/19 unsupported. Smoothing check: 0.4–4.7; 0/3 unsupported. These are sensitivity checks, not confidence intervals.
Audit: family and publisher holdouts improved on neutral predictions within evaluated coverage. Validation audit.
Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.
- LiveCodeBench: 55.6 observed win share
Amazon · Source 1 - SWE-bench Verified: 59.7 observed win share
Amazon · Source 1 Source 2
Human prefNo comparable evidenceUnknown
0 core families; 0 direct opponents across 0 labs. No admitted core comparisons
Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.
Refit without one family: No supported estimate; 1/1 scenarios unsupported. Without one publisher: No supported estimate; 1/1 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.
Audit: cross-family and cross-publisher predictions could not be evaluated from the available comparison network. Validation audit.
Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.
Knowledge1 families · 0 with independent evidence · PreliminaryUnknown
1 core families; 11 direct opponents across 4 labs. No connected comparison to the complete reference panel
Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.
Refit without one family: No supported estimate; 4/4 scenarios unsupported. Without one publisher: No supported estimate; 5/5 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.
Audit: almost no publisher-held-out comparisons remained estimable. Cross-source validity is unresolved. Validation audit.
Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.
- MMLU-Pro: 45.5 observed win share
Amazon · Source 1
Multimodal3 families · 0 with independent evidence · PreliminaryUnknown
3 core families; 11 direct opponents across 4 labs. No connected comparison to the complete reference panel
Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.
Refit without one family: No supported estimate; 8/8 scenarios unsupported. Without one publisher: No supported estimate; 8/8 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.
Audit: limited publisher-held-out coverage. Cross-source validity is unresolved. Validation audit.
Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.
- Ocrbench V2 Average Accuracy: 100.0 observed win share
Amazon · Source 1 - Mmmu Pro: 45.5 observed win share
Amazon · Source 1 - ScreenSpot: 100.0 observed win share
Amazon · Source 1
Long contextNo comparable evidenceUnknown
0 core families; 0 direct opponents across 0 labs. No admitted core comparisons
Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.
Refit without one family: No supported estimate; 5/5 scenarios unsupported. Without one publisher: No supported estimate; 5/5 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.
Audit: family and publisher holdouts underperformed neutral predictions. Treat this as an exploratory estimate. Validation audit.
Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.
Scores estimate outcomes against a shared reference panel; they are not accuracy percentages. Sparse or disconnected evidence cannot qualify an overall profile. Open a capability to inspect its evidence.
/badge/amazon-nova-2-pro-preview.svg