Filled points are supported; hollow points are preliminary. Lines stop at unknown capabilities. Exact values and sources follow below.
Agentic5 families · 1 with independent evidence · Supported8.6
5 core families; 14 direct opponents across 9 labs.
Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.
Refit without one family: 3.9–12.1; 0/13 scenarios unsupported. Without one publisher: 7.7–36.9; 0/15 unsupported. Smoothing check: 5.5–16.5; 0/3 unsupported. These are sensitivity checks, not confidence intervals.
Audit: family and publisher holdouts improved on neutral predictions within evaluated coverage; source sensitivity still requires inspection. Validation audit.
Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.
- OSWorld: 20.0 observed win share
MiniMax, os-world.github.io · Source 1 Source 2 - Browsecomp: 25.0 observed win share
MiniMax, Mistral · Source 1 Source 2 - Apex Agents: 20.0 observed win share
MiniMax · Source 1 - GDPval: 71.4 observed win share
MiniMax · Source 1 - MCP-Atlas: 14.3 observed win share
MiniMax · Source 1
Hard reasoning3 families · 1 with independent evidence · Supported11.2
3 core families; 65 direct opponents across 12 labs.
Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.
Refit without one family: 8.2–25.1; 0/6 scenarios unsupported. Without one publisher: 8.0–19.4; 0/16 unsupported. Smoothing check: 4.0–23.7; 0/3 unsupported. These are sensitivity checks, not confidence intervals.
Audit: family and publisher holdouts improved on neutral predictions within evaluated coverage. Validation audit.
Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.
- ARC-AGI: 54.2 observed win share
arcprize.org · Source 1 - Aime: 37.5 observed win share
Microsoft, Mistral · Source 1 Source 2 - GPQA: 40.0 observed win share
Microsoft · Source 1
Coding2 families · 1 with independent evidence · Supported3.5
2 core families; 37 direct opponents across 12 labs.
Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.
Refit without one family: 1.7–13.8; 2/6 scenarios unsupported. Without one publisher: 2.3–13.4; 1/19 unsupported. Smoothing check: 1.7–9.2; 0/3 unsupported. These are sensitivity checks, not confidence intervals.
Audit: family and publisher holdouts improved on neutral predictions within evaluated coverage. Validation audit.
Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.
- DeepSWE: 3.7 observed win share
deepswe.datacurve.ai · Source 1 - SWE-bench Verified: 36.1 observed win share
Microsoft, MiniMax, Mistral · Source 1 Source 2 Source 3
Human pref1 families · 1 with independent evidence · PreliminaryUnknown
1 core families; 76 direct opponents across 17 labs. No connected comparison to the complete reference panel
Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.
Refit without one family: No supported estimate; 1/1 scenarios unsupported. Without one publisher: No supported estimate; 1/1 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.
Audit: cross-family and cross-publisher predictions could not be evaluated from the available comparison network. Validation audit.
Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.
- Human preference (Arena): 85.5 observed win share
lmarena.ai · Source 1
Knowledge2 families · 0 with independent evidence · PreliminaryUnknown
2 core families; 1 direct opponents across 1 labs. No connected comparison to the complete reference panel
Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.
Refit without one family: No supported estimate; 4/4 scenarios unsupported. Without one publisher: No supported estimate; 5/5 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.
Audit: almost no publisher-held-out comparisons remained estimable. Cross-source validity is unresolved. Validation audit.
Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.
- MMLU-Pro: 100.0 observed win share
Microsoft · Source 1 - Simpleqa Verified: 0.0 observed win share
Microsoft · Source 1
Multimodal1 families · 0 with independent evidence · Preliminary1.2
1 core families; 5 direct opponents across 5 labs. Needs broader benchmark and opponent support
Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.
Refit without one family: No supported estimate; 8/8 scenarios unsupported. Without one publisher: No supported estimate; 8/8 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.
Audit: limited publisher-held-out coverage. Cross-source validity is unresolved. Validation audit.
Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.
- Mmmu Pro: 0.0 observed win share
MiniMax · Source 1
Long context2 families · 0 with independent evidence · PreliminaryUnknown
2 core families; 1 direct opponents across 1 labs. No connected comparison to the complete reference panel
Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.
Refit without one family: No supported estimate; 5/5 scenarios unsupported. Without one publisher: No supported estimate; 5/5 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.
Audit: family and publisher holdouts underperformed neutral predictions. Treat this as an exploratory estimate. Validation audit.
Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.
- Graphwalks: 100.0 observed win share
Microsoft · Source 1 - Longbench: 100.0 observed win share
Microsoft · Source 1
Scores estimate outcomes against a shared reference panel; they are not accuracy percentages. Sparse or disconnected evidence cannot qualify an overall profile. Open a capability to inspect its evidence.