Methodology

A rank needs 3 benchmark families, 1 reviewed independent evaluator, 3 opponent models and 2 independent families with 3 opponents each. Lab-only evidence cannot qualify. There is no requirement to cover every capability area.

Multiple independent sources means more than one reviewed independent evaluator contributes. It does not guarantee agreement. Matched provider reruns may contribute within their own test setup. Results using fallback models, unmatched settings or safety stress tests remain visible without entering the ranking. Source percentages describe direct comparison weight, not confidence or fractions of the final score. Family cells are descriptive matched win shares, not separate aggregate ratings.

Source balancing does not prove independence: overlapping benchmarks may remain correlated. Leave-one-source-out ranks compare the same surviving configurations before and after removal, with lost comparison support counted separately. These diagnostic cohorts include separate effort settings; they are not the default leaderboard. Eligibility is not rechecked after removal.

Movement is a sensitivity diagnostic, not a spread of source-specific ranks or a confidence measure. Sensitivity values appear only when the dated audit matches the current inputs and scoring version.

All capability axes use their own fixed panel: Sol High, Opus 4.7 Max and Gemini 3.5 Flash High, measured across the populated areas. Capability profiles retain 20% smoothing. The overall calibration review did not validate a change to each separate capability.

Effort labels do not establish equal compute. Settings can cover different tests: the highest aggregate is not proof that one effort beats another on shared tasks. Model pages flag aggregate orders that conflict with shared results across at least three benchmark families. These flags describe comparisons, not independent statistical trials.

Sources cover different models, tasks and setups. Only reviewed, comparable evidence enters the score; not every public result qualifies. A small benchmark lead and a large one both count as a win. Run variance and sample sizes are not modeled, so score gaps and sensitivity ranges do not establish statistical significance.

The September 21 review reduced overall smoothing from 20% to 10%. Excluding Grok 4.7 from method selection, mean held-out log loss improved from 0.459 to 0.444 across sources and 0.431 to 0.411 across families, with unchanged coverage. Two additional unanimous shared-result reversals remained, both under 0.6 score points. This is reused-data method development, not prospective validation or proof of a definitive order. Calibration audit

The dated audit predicts 96.8% of held-out family comparisons and 79.4% of held-out source comparisons on average, including empty folds. It retains 13 consistent shared-benchmark reversals. This is reused-data diagnosis, not an untouched final test.

The source-budget policy, with the previous smoothing, was frozen before acquiring LiveBench and SWE-rebench results. On comparable predictions, accuracy rose from 68.4% to 69.3%, with lower probability error. This small gain does not establish statistical superiority. The results were newly acquired, not necessarily newly published. Earlier listwise and family-varying candidates remain unpromoted under their separate temporal gates.

A joint source-and-family budget remains unpromoted: it reduced effort-order reversals but worsened held-out probability error and source sensitivity. New independent Vals AI evidence also favored the then-deployed formula in a frozen prediction check; acquisition dates alone do not make this future-data validation.

Frozen external validation · Dated configuration audit, alternatives and coverage gaps · Candidate diagnostics and release status

Observation admission decisions · Scores and contributing evidence · Source measurements

The budget is applied before selecting the best effort per model; lower cost breaks exact, unrounded score ties. With no budget, higher Capability scores lead among priced configurations. A cost frontier shows configurations with no cheaper, equal-or-higher point estimate. Similar scores do not prove equivalent task performance or statistically proven winners.

Measured usage: the complete AA Intelligence Index v4.3 workload, priced from its reported tokens, including reasoning, cache treatment and reviewed per-evaluation fallback token fractions at the fallback model's prices. Missing families and unresolved fallback routing receive no task cost. This is one evaluator's workload, not a universal cost of quality.

Price scenario: fixed uncached input and total billable output tokens at AA's published rates. It cannot establish effort efficiency. Endpoint, context tier, tool charges and fallback costs are not verified in this estimate. Open weights are not treated as free inference.

Coverage and freshness: exact reviewed configurations only. Missing and over-budget costs do not receive a Value rank. Missing costs stay unknown; pricing older than 30 days is suspended from comparison. Source dates and skipped records are downloadable.

Use UnifyBench to find candidates, then inspect the original results from Artificial Analysis, LMArena and task-specific benchmarks. OpenRouter rankings show API usage, a separate signal from measured performance.

Agent results also depend on tools and the surrounding software. They describe a model in that setup, not model quality in isolation. No single leaderboard establishes the best model for every task and budget.