RankingClaude Haiku 5.5

Claude Haiku 5.5

Compare models

Data updated 22 Nov 2026

Claude Haiku 5.5 by Anthropic is a Claude Haiku model, released 2026-10-07. This profile collects the published benchmark results and reported effort settings for this model. Configurations are compared on the effort page. Ranks are on the leaderboard.

37 published benchmark measures · 136 measured effort results · 4 benchmark families contribute across 2 task areas. 2 capability estimates available in the full profile. See all results ↓ · Compare published benchmarks →

Model evidence summary

This profile combines published settings. It is not a runnable configuration or a leaderboard rank. Compare measured configurations →

Performance profile

Capabilities

Adjusted comparison score · 0–100
Claude Haiku 5.5 capability radarComparison scores from 0 to 100. Unknown capabilities have no point; lines stop at gaps. Hollow points are preliminary. Exact values and support labels follow the chart.AgenticHard reasoningCodingHuman prefKnowledgeMultimodalLong context50100Claude Haiku 5.5 · Hard reasoning: 28.7 · SupportedClaude Haiku 5.5 · Coding: 66.4 · Supported

Filled points are supported; hollow points are preliminary. Lines stop at unknown capabilities. Exact values and sources follow below.

AgenticNo comparable evidenceUnknown

0 core families; 0 direct opponents across 0 labs. No admitted core comparisons

Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.

Refit without one family: No supported estimate; 13/13 scenarios unsupported. Without one publisher: No supported estimate; 15/15 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.

Audit: family and publisher holdouts improved on neutral predictions within evaluated coverage; source sensitivity still requires inspection. Validation audit.

Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.

    Hard reasoning2 families · 2 with independent evidence · Supported28.7

    2 core families; 62 direct opponents across 11 labs.

    Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.

    Refit without one family: 25.4–30.0; 2/6 scenarios unsupported. Without one publisher: 24.2–29.9; 2/16 unsupported. Smoothing check: 20.1–38.5; 0/3 unsupported. These are sensitivity checks, not confidence intervals.

    Audit: family and publisher holdouts improved on neutral predictions within evaluated coverage. Validation audit.

    Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.

    • ARC-AGI: 58.5 observed win share
      arcprize.org · Source 1
    • Humanity’s Last Exam: 62.5 observed win share
      scale.com · Source 1
    Coding2 families · 2 with independent evidence · Supported66.4

    2 core families; 30 direct opponents across 10 labs.

    Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.

    Refit without one family: 53.3–67.5; 2/6 scenarios unsupported. Without one publisher: 61.1–71.8; 2/19 unsupported. Smoothing check: 61.0–69.2; 0/3 unsupported. These are sensitivity checks, not confidence intervals.

    Audit: family and publisher holdouts improved on neutral predictions within evaluated coverage. Validation audit.

    Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.

    • Terminal-Bench: 65.5 observed win share
      tbench.ai · Source 1
    • SWE-bench Pro: 12.5 observed win share
      labs.scale.com · Source 1
    Human prefNo comparable evidenceUnknown

    0 core families; 0 direct opponents across 0 labs. No admitted core comparisons

    Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.

    Refit without one family: No supported estimate; 1/1 scenarios unsupported. Without one publisher: No supported estimate; 1/1 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.

    Audit: cross-family and cross-publisher predictions could not be evaluated from the available comparison network. Validation audit.

    Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.

      KnowledgeNo comparable evidenceUnknown

      0 core families; 0 direct opponents across 0 labs. No admitted core comparisons

      Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.

      Refit without one family: No supported estimate; 4/4 scenarios unsupported. Without one publisher: No supported estimate; 5/5 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.

      Audit: almost no publisher-held-out comparisons remained estimable. Cross-source validity is unresolved. Validation audit.

      Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.

        MultimodalNo comparable evidenceUnknown

        0 core families; 0 direct opponents across 0 labs. No admitted core comparisons

        Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.

        Refit without one family: No supported estimate; 8/8 scenarios unsupported. Without one publisher: No supported estimate; 8/8 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.

        Audit: limited publisher-held-out coverage. Cross-source validity is unresolved. Validation audit.

        Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.

          Long contextNo comparable evidenceUnknown

          0 core families; 0 direct opponents across 0 labs. No admitted core comparisons

          Estimated against the same complete panel: gpt-5.6-sol claude-opus-4-8 gemini-3.1-pro . Indirect comparisons assume performance can be summarized by one strength within this capability.

          Refit without one family: No supported estimate; 5/5 scenarios unsupported. Without one publisher: No supported estimate; 5/5 unsupported. Smoothing check: No supported estimate; 3/3 unsupported. These are sensitivity checks, not confidence intervals.

          Audit: family and publisher holdouts underperformed neutral predictions. Treat this as an exploratory estimate. Validation audit.

          Matched family win shares below describe source evidence; they are not averaged to produce the adjusted score.

            Scores estimate outcomes against a shared reference panel; they are not accuracy percentages. Sparse or disconnected evidence cannot qualify an overall profile. Open a capability to inspect its evidence.

            Reported effort · Mixed settings

            Settings reported in this model's published benchmark results, including results outside the aggregate. Effort names are provider-specific. These are not API defaults or equal compute budgets.

            Mixed settings means multiple settings occur in the evidence. Best across efforts means the source selected its best reported result across settings; it does not mean Max. Unspecified settings stay unknown. This model-summary chart combines reported settings. The leaderboard keeps identified configurations separate and excludes unknown effort.

            Inspect each result and its source ↓ · Download effort evidence
            Score contributions and missing evidence

            4 contributing families across 2 capabilities. Fixed reference panels do not change when the catalog expands.

            Capability is fitted jointly across families. Capability estimates below describe different task areas; their weighted sum is not the Capability score.

            Results without reviewed compatibility or a reference match remain in the raw evidence below. Coverage counts only contributing results.

            Model information & shareable badge
            Lab
            Anthropic
            Catalog status
            active
            Availability
            Claude API, Amazon Bedrock, Google Cloud, Microsoft Foundry and Claude Platform on AWS; account and region restrictions may apply.
            Family
            Claude Haiku
            Released
            2026-10-07
            Context
            1,000,000 tokens
            API list price
            $0.10 input / $0.50 output per million tokens
            Prompts up to 100,000 tokens. Above 100,000 tokens, $0.50 input and $2.50 output per million tokens. Cache reads $0.01/$0.05; 5-minute cache writes $0.125/$0.625 per million tokens. Adaptive thinking defaults to Medium; supported Low, Medium, High, XHigh and Max are not inferred for observations.
            License
            proprietary
            Model card
            https://platform.claude.com/docs/en/models/haiku-5-5/overview
            Default Capability family coverage
            Documented-evidence family coverage/badge/claude-haiku-5-5.svg

            Claude Haiku 5.5 measured results by effort setting

            Each row is one published result for one effort setting. Evidence collected 2026-11-22. Compare the effort settings →

            BenchmarkEffortScoreSource
            AA-AnalystAgentMax53.8%Artificial Analysis
            AA-Briefcase EloHigh1441Artificial Analysis
            AA-Briefcase EloLow1113Artificial Analysis
            AA-Briefcase EloMax1572Artificial Analysis
            AA-Briefcase EloMedium1366Artificial Analysis
            AA-Briefcase EloXHigh1523Artificial Analysis
            AA-LCR v1.1High77.3%Artificial Analysis
            AA-LCR v1.1Low72.0%Artificial Analysis
            AA-LCR v1.1Max82.7%Artificial Analysis
            AA-LCR v1.1Medium77.3%Artificial Analysis
            AA-LCR v1.1XHigh78.3%Artificial Analysis
            AA-Omniscience AccuracyHigh31.8%Artificial Analysis
            AA-Omniscience AccuracyLow30.4%Artificial Analysis
            AA-Omniscience AccuracyMax34.7%Artificial Analysis
            AA-Omniscience AccuracyMedium31.7%Artificial Analysis
            AA-Omniscience AccuracyXHigh33.8%Artificial Analysis
            AA-Omniscience IndexHigh5.3Artificial Analysis
            AA-Omniscience IndexLow2.2Artificial Analysis
            AA-Omniscience IndexMax12.3Artificial Analysis
            AA-Omniscience IndexMedium4.2Artificial Analysis
            AA-Omniscience IndexXHigh7.8Artificial Analysis
            ARC-AGI-2High30.7%ARC Prize
            ARC-AGI-2Low2.8%ARC Prize
            ARC-AGI-2Medium9.4%ARC Prize
            ARC-AGI-2XHigh61.3%ARC Prize
            ARC-AGI-3 · provider-adapterMax2.8%ARC Prize
            ARC-AGI-3 · standardMax0.5%ARC Prize
            AutomationBench-AAHigh33.7%Artificial Analysis
            AutomationBench-AALow22.9%Artificial Analysis
            AutomationBench-AAMax35.4%Artificial Analysis
            AutomationBench-AAMedium28.6%Artificial Analysis
            AutomationBench-AAXHigh36.0%Artificial Analysis
            CritPtHigh18.6%Artificial Analysis
            CritPtLow9.1%Artificial Analysis
            CritPtMax18.9%Artificial Analysis
            CritPtMedium12.9%Artificial Analysis
            CritPtXHigh22.6%Artificial Analysis
            FrontierCode 1.1 · extended-claude-codeHigh55.9%Cognition
            FrontierCode 1.1 · extended-claude-codeLow48.2%Cognition
            FrontierCode 1.1 · extended-claude-codeMax58.4%Cognition
            FrontierCode 1.1 · extended-claude-codeMedium55.0%Cognition
            FrontierCode 1.1 · extended-claude-codeXHigh58.1%Cognition
            FrontierCode 1.1 · main-claude-codeHigh41.9%Cognition
            FrontierCode 1.1 · main-claude-codeLow34.8%Cognition
            FrontierCode 1.1 · main-claude-codeMax46.4%Cognition
            FrontierCode 1.1 · main-claude-codeMedium41.6%Cognition
            FrontierCode 1.1 · main-claude-codeXHigh45.8%Cognition
            FrontierSWE v2Unspecified43.8%Proximal
            GDP.pdf: All-pass RateHigh17.2%Artificial Analysis
            GDP.pdf: All-pass RateLow11.2%Artificial Analysis
            GDP.pdf: All-pass RateMax20.8%Artificial Analysis
            GDP.pdf: All-pass RateMedium15.2%Artificial Analysis
            GDP.pdf: All-pass RateXHigh18.2%Artificial Analysis
            GDPval-AA v2.1 LeaderboardHigh1424Artificial Analysis
            GDPval-AA v2.1 LeaderboardLow1130Artificial Analysis
            GDPval-AA v2.1 LeaderboardMax1629Artificial Analysis
            GDPval-AA v2.1 LeaderboardMedium1282Artificial Analysis
            GDPval-AA v2.1 LeaderboardXHigh1523Artificial Analysis
            Harvey LAB-AA v1.1: Hallucination-Gated All-Pass RateHigh1.1%Artificial Analysis
            Harvey LAB-AA v1.1: Hallucination-Gated All-Pass RateLow1.7%Artificial Analysis
            Harvey LAB-AA v1.1: Hallucination-Gated All-Pass RateMax0.6%Artificial Analysis
            Harvey LAB-AA v1.1: Hallucination-Gated All-Pass RateMedium0.6%Artificial Analysis
            Harvey LAB-AA v1.1: Hallucination-Gated All-Pass RateXHigh2.8%Artificial Analysis
            Humanity's Last ExamHigh37.3%Artificial Analysis
            Humanity's Last ExamLow27.0%Artificial Analysis
            Humanity's Last ExamMax44.4%Artificial Analysis
            Humanity's Last ExamMedium33.8%Artificial Analysis
            Humanity's Last ExamXHigh42.7%Artificial Analysis
            LiveBench · AMPS Hard · 2026-06-25Max97.0%LiveBench
            LiveBench · AMPS Hard · 2026-06-25XHigh98.0%LiveBench
            LiveBench · code completion · 2026-06-25Max78.3%LiveBench
            LiveBench · code completion · 2026-06-25XHigh73.9%LiveBench
            LiveBench · code generation · 2026-06-25Max77.5%LiveBench
            LiveBench · code generation · 2026-06-25XHigh78.9%LiveBench
            LiveBench · connections · 2026-06-25Max96.0%LiveBench
            LiveBench · connections · 2026-06-25XHigh93.8%LiveBench
            LiveBench · consecutive events · 2026-06-25Max10.6%LiveBench
            LiveBench · consecutive events · 2026-06-25XHigh71.0%LiveBench
            LiveBench · integrals with game · 2026-06-25Max67.0%LiveBench
            LiveBench · integrals with game · 2026-06-25XHigh89.0%LiveBench
            LiveBench · javascript · 2026-06-25Max68.2%LiveBench
            LiveBench · javascript · 2026-06-25XHigh59.1%LiveBench
            LiveBench · logic with navigation · 2026-06-25Max66.0%LiveBench
            LiveBench · logic with navigation · 2026-06-25XHigh56.0%LiveBench
            LiveBench · math comp · 2026-06-25Max98.0%LiveBench
            LiveBench · math comp · 2026-06-25XHigh97.1%LiveBench
            LiveBench · olympiad · 2026-06-25Max83.7%LiveBench
            LiveBench · olympiad · 2026-06-25XHigh87.0%LiveBench
            LiveBench · paraphrase · 2026-06-25Max50.2%LiveBench
            LiveBench · paraphrase · 2026-06-25XHigh62.5%LiveBench
            LiveBench · plot unscrambling · 2026-06-25Max27.9%LiveBench
            LiveBench · plot unscrambling · 2026-06-25XHigh42.7%LiveBench
            LiveBench · python · 2026-06-25Max60.0%LiveBench
            LiveBench · python · 2026-06-25XHigh55.0%LiveBench
            LiveBench · simplify · 2026-06-25Max50.3%LiveBench
            LiveBench · simplify · 2026-06-25XHigh62.8%LiveBench
            LiveBench · spatial · 2026-06-25Max92.0%LiveBench
            LiveBench · spatial · 2026-06-25XHigh94.0%LiveBench
            LiveBench · story generation · 2026-06-25Max64.5%LiveBench
            LiveBench · story generation · 2026-06-25XHigh73.6%LiveBench
            LiveBench · summarize · 2026-06-25Max53.6%LiveBench
            LiveBench · summarize · 2026-06-25XHigh67.0%LiveBench
            LiveBench · tablejoin · 2026-06-25Max46.7%LiveBench
            LiveBench · tablejoin · 2026-06-25XHigh49.2%LiveBench
            LiveBench · tablereformat · 2026-06-25Max98.0%LiveBench
            LiveBench · tablereformat · 2026-06-25XHigh98.0%LiveBench
            LiveBench · theory of mind · 2026-06-25Max86.5%LiveBench
            LiveBench · theory of mind · 2026-06-25XHigh76.9%LiveBench
            LiveBench · typescript · 2026-06-25Max43.3%LiveBench
            LiveBench · typescript · 2026-06-25XHigh40.0%LiveBench
            LiveBench · typos · 2026-06-25Max64.0%LiveBench
            LiveBench · typos · 2026-06-25XHigh54.0%LiveBench
            LiveBench · zebra puzzle · 2026-06-25Max94.0%LiveBench
            LiveBench · zebra puzzle · 2026-06-25XHigh98.0%LiveBench
            MLCR-AAHigh15.0%Artificial Analysis
            MLCR-AALow12.2%Artificial Analysis
            MLCR-AAMax35.0%Artificial Analysis
            MLCR-AAMedium16.1%Artificial Analysis
            MLCR-AAXHigh15.0%Artificial Analysis
            SciCodeHigh48.7%Artificial Analysis
            SciCodeLow49.2%Artificial Analysis
            SciCodeMax55.0%Artificial Analysis
            SciCodeMedium49.0%Artificial Analysis
            SciCodeXHigh51.7%Artificial Analysis
            SWE-Bench Pro V2 Hard · mini-swe-agentHigh52.9%Scale
            Terminal-Bench 4.0 · claudecodeMax40.0%Terminal-Bench
            Terminal-Bench v4.0High21.7%Artificial Analysis
            Terminal-Bench v4.0Low12.6%Artificial Analysis
            Terminal-Bench v4.0Max32.8%Artificial Analysis
            Terminal-Bench v4.0Medium15.2%Artificial Analysis
            Terminal-Bench v4.0XHigh29.3%Artificial Analysis
            Terminal-Bench-Science 0.1High10.0%Artificial Analysis
            Terminal-Bench-Science 0.1Low1.9%Artificial Analysis
            Terminal-Bench-Science 0.1Max20.0%Artificial Analysis
            Terminal-Bench-Science 0.1Medium7.1%Artificial Analysis
            Terminal-Bench-Science 0.1XHigh18.1%Artificial Analysis
            Benchmark scores & sources

            Original results, evaluation harnesses, and evidence behind this model.

            BenchmarkBucketScoreHarnessEvidenceSource-recorded date
            HealthBench Professional · Length-adjustedSupporting evidence61.3%
            Reported settings & source

            Safety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. high effort. Length-adjusted score; five runs per effort. Low–xhigh onOct4; max onOct1–2.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07

            Published configuration

            Effort: High

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            HealthBench Professional · Length-adjustedSupporting evidence61%
            Reported settings & source

            Safety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. xhigh effort. Length-adjusted score; five runs per effort. Low–xhigh onOct4; max onOct1–2.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07

            Published configuration

            Effort: Xhigh

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            HealthBench Professional · Length-adjustedSupporting evidence64.8%
            Reported settings & source

            Safety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. max effort. Length-adjusted score; five runs per effort. Low–xhigh onOct4; max onOct1–2.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            HealthBench Professional · Length-adjusted / Oct4 repeatSupporting evidence64.2%
            Reported settings & source

            Safety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. Max effort; thirteen repeat runs onOctober4; length-adjusted score, distinct dated rerun.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.11.3; p.133 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            HealthBench Professional · RawSupporting evidence61.6%
            Reported settings & source

            Safety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. low effort; raw score before length adjustment; five-run mean.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.11.3; p.133 · reviewed 2026-10-07

            Published configuration

            Effort: Low

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            HealthBench Professional · RawSupporting evidence71%
            Reported settings & source

            Safety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. max effort; raw score before length adjustment; five-run mean.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.11.3; p.133 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            Humanity's Last ExamHard reasoning30.12%HLE no tools

            Effort: Not specified

            contributes to capability
            official board2026-10-08
            Humanity’s Last Exam · No toolsHard reasoning45.9%
            Reported settings & source

            No tools; max effort from summary table; thinking auto in methodology; 980K task budget, no context compaction; Claude Opus4.6 grader. Reasoning only; no tools.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · Table 8.1.A; section 8.8.1; pp.111,118–120 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            Humanity’s Last Exam · With toolsHard reasoning57.4%
            Reported settings & source

            With tools; max effort from summary table; thinking auto in methodology; 980K task budget, no context compaction; Claude Opus4.6 grader. Web search/fetch, programmatic tool calling, code execution; blocklist and transcript contamination screening.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · Table 8.1.A; section 8.8.1; pp.111,118–120 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            LMArena Agent ArenaSupporting evidence-0.61 LMArena Agent Arena · high

            Effort: Not specified

            official board

            Supporting evidence outside the reviewed capability core

            2026-11-18
            Medicinal chemistrySupporting evidence58%
            Reported settings & source

            API, biology safeguards disabled; adaptive thinking max effort; five attempts/problem unless explicitly noted. Scores describe underlying model capability, not deployed safeguards. 504 questions about ADME-related measured properties of depicted drug-like compounds.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.13.3; pp.136–139 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            MILUKnowledge87.6%
            Reported settings & source

            Adaptive thinking max effort; safety classifiers enabled; five-trial mean; no tools or customized system prompts;11-language mean. Blocked/unanswered under0.2% excluded from average.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.12.2; pp.135–136 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            Morphology-to-molecule matchingSupporting evidence25.4%
            Reported settings & source

            API, biology safeguards disabled; adaptive thinking max effort; five attempts/problem unless explicitly noted. Scores describe underlying model capability, not deployed safeguards. Match four human-liver assay image sets to shuffled molecule-dose perturbations.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.13.2; pp.136–139 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            OfficeQAAgentic73.5%
            Reported settings & source

            Agentic evaluation with relevant Treasury Bulletin documents preselected as extracted text; max effort; five-run mean.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.10.1; p.130 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            OfficeQA ProAgentic60.3%
            Reported settings & source

            Agentic evaluation with relevant Treasury Bulletin documents preselected as extracted text; max effort; five-run mean. Harder133-question subset.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.10.1; p.130 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            OSWorld · 2.1 offline subset / Partial creditAgentic72.4%
            Reported settings & source

            Official offline82-task subset of108 tasks; max effort; 1080p;500 action steps; five attempts/task; Claude Opus4.8 grader where needed; metric:Partial credit.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.9.3; pp.127–130 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            OSWorld · 2.1 offline subset / Strict pass rateAgentic37.1%
            Reported settings & source

            Official offline82-task subset of108 tasks; max effort; 1080p;500 action steps; five attempts/task; Claude Opus4.8 grader where needed; metric:Strict pass rate.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.9.3; pp.127–130 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            PhysicianBenchSupporting evidence17.8%
            Reported settings & source

            Safety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. low effort. Five attempts per task across100 EHR tasks; share of attempts passed.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07

            Published configuration

            Effort: Low

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            PhysicianBenchSupporting evidence25.2%
            Reported settings & source

            Safety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. medium effort. Five attempts per task across100 EHR tasks; share of attempts passed.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07

            Published configuration

            Effort: Medium

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            PhysicianBenchSupporting evidence31.6%
            Reported settings & source

            Safety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. high effort. Five attempts per task across100 EHR tasks; share of attempts passed.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07

            Published configuration

            Effort: High

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            PhysicianBenchSupporting evidence35.8%
            Reported settings & source

            Safety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. xhigh effort. Five attempts per task across100 EHR tasks; share of attempts passed.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07

            Published configuration

            Effort: Xhigh

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            PhysicianBenchSupporting evidence43%
            Reported settings & source

            Safety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. max effort. Five attempts per task across100 EHR tasks; share of attempts passed.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            ProgramBenchSupporting evidence82%
            Reported settings & source

            Modified 166-task subset after excluding 34 flaky-reference tasks; hidden-test pass rate; mini-swe-agent harness without upstream six-hour limit; no internet or decompilation. Effort is not explicitly stated in this section.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Not the unmodified 200-task benchmark; scores only tests passed by reference binaries.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.7.1; p.117 · reviewed 2026-10-07

            Published configuration

            Effort: Not specified

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            Protein design — Library RankingSupporting evidence45.3%
            Reported settings & source

            API, biology safeguards disabled; adaptive thinking max effort; five attempts/problem unless explicitly noted. Scores describe underlying model capability, not deployed safeguards. Three attempts/problem; prioritize protein designs for experimental testing.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.13.4; pp.136–139 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07
            Protein design — Sequence GenerationSupporting evidence33.1%
            Reported settings & source

            API, biology safeguards disabled; adaptive thinking max effort; five attempts/problem unless explicitly noted. Scores describe underlying model capability, not deployed safeguards. One attempt/problem; novel protein sequences conditioned on design constraints.

            Provider-published lab_self_report; retained as launch evidence, not an independent board admission.

            Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Claude Haiku 5.5 System Card · section 8.13.4; pp.136–139 · reviewed 2026-10-07

            Published configuration

            Effort: Max

            lab self-report

            Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

            Reviewed 2026-10-07