RankingClaude Haiku 5.5
Claude Haiku 5.5
Claude Haiku 5.5 by Anthropic is a Claude Haiku model, released 2026-10-07. This profile collects the published benchmark results and reported effort settings for this model. Configurations are compared on the effort page. Ranks are on the leaderboard.
37 published benchmark measures · 136 measured effort results · 4 benchmark families contribute across 2 task areas. 2 capability estimates available in the full profile. See all results ↓ · Compare published benchmarks →
Model evidence summary
This profile combines published settings. It is not a runnable configuration or a leaderboard rank. Compare measured configurations →
Performance profile
Capabilities
Filled points are supported; hollow points are preliminary. Lines stop at unknown capabilities. Exact values and sources follow below.
Scores estimate outcomes against a shared reference panel; they are not accuracy percentages. Sparse or disconnected evidence cannot qualify an overall profile. Open a capability to inspect its evidence.
Reported effort · Mixed settings
Settings reported in this model's published benchmark results, including results outside the aggregate. Effort names are provider-specific. These are not API defaults or equal compute budgets.
- High: 3 observations
- Low: 4 observations
- Max: 36 observations
- Medium: 5 observations
- Xhigh: 4 observations
- Not specified: 9 observations
Mixed settings means multiple settings occur in the evidence. Best across efforts means the source selected its best reported result across settings; it does not mean Max. Unspecified settings stay unknown. This model-summary chart combines reported settings. The leaderboard keeps identified configurations separate and excludes unknown effort.
Inspect each result and its source ↓ · Download effort evidenceScore contributions and missing evidence
4 contributing families across 2 capabilities. Fixed reference panels do not change when the catalog expands.
Capability is fitted jointly across families. Capability estimates below describe different task areas; their weighted sum is not the Capability score.
Results without reviewed compatibility or a reference match remain in the raw evidence below. Coverage counts only contributing results.
Model information & shareable badge
- Lab
- Anthropic
- Catalog status
- active
- Availability
- Claude API, Amazon Bedrock, Google Cloud, Microsoft Foundry and Claude Platform on AWS; account and region restrictions may apply.
- Family
- Claude Haiku
- Released
- 2026-10-07
- Context
- 1,000,000 tokens
- API list price
- $0.10 input / $0.50 output per million tokens
- License
- proprietary
- Model card
- https://platform.claude.com/docs/en/models/haiku-5-5/overview
- Default Capability family coverage
/badge/claude-haiku-5-5.svg
Claude Haiku 5.5 measured results by effort setting
Each row is one published result for one effort setting. Evidence collected 2026-11-22. Compare the effort settings →
| Benchmark | Effort | Score | Source |
|---|---|---|---|
| AA-AnalystAgent | Max | 53.8% | Artificial Analysis |
| AA-Briefcase Elo | High | 1441 | Artificial Analysis |
| AA-Briefcase Elo | Low | 1113 | Artificial Analysis |
| AA-Briefcase Elo | Max | 1572 | Artificial Analysis |
| AA-Briefcase Elo | Medium | 1366 | Artificial Analysis |
| AA-Briefcase Elo | XHigh | 1523 | Artificial Analysis |
| AA-LCR v1.1 | High | 77.3% | Artificial Analysis |
| AA-LCR v1.1 | Low | 72.0% | Artificial Analysis |
| AA-LCR v1.1 | Max | 82.7% | Artificial Analysis |
| AA-LCR v1.1 | Medium | 77.3% | Artificial Analysis |
| AA-LCR v1.1 | XHigh | 78.3% | Artificial Analysis |
| AA-Omniscience Accuracy | High | 31.8% | Artificial Analysis |
| AA-Omniscience Accuracy | Low | 30.4% | Artificial Analysis |
| AA-Omniscience Accuracy | Max | 34.7% | Artificial Analysis |
| AA-Omniscience Accuracy | Medium | 31.7% | Artificial Analysis |
| AA-Omniscience Accuracy | XHigh | 33.8% | Artificial Analysis |
| AA-Omniscience Index | High | 5.3 | Artificial Analysis |
| AA-Omniscience Index | Low | 2.2 | Artificial Analysis |
| AA-Omniscience Index | Max | 12.3 | Artificial Analysis |
| AA-Omniscience Index | Medium | 4.2 | Artificial Analysis |
| AA-Omniscience Index | XHigh | 7.8 | Artificial Analysis |
| ARC-AGI-2 | High | 30.7% | ARC Prize |
| ARC-AGI-2 | Low | 2.8% | ARC Prize |
| ARC-AGI-2 | Medium | 9.4% | ARC Prize |
| ARC-AGI-2 | XHigh | 61.3% | ARC Prize |
| ARC-AGI-3 · provider-adapter | Max | 2.8% | ARC Prize |
| ARC-AGI-3 · standard | Max | 0.5% | ARC Prize |
| AutomationBench-AA | High | 33.7% | Artificial Analysis |
| AutomationBench-AA | Low | 22.9% | Artificial Analysis |
| AutomationBench-AA | Max | 35.4% | Artificial Analysis |
| AutomationBench-AA | Medium | 28.6% | Artificial Analysis |
| AutomationBench-AA | XHigh | 36.0% | Artificial Analysis |
| CritPt | High | 18.6% | Artificial Analysis |
| CritPt | Low | 9.1% | Artificial Analysis |
| CritPt | Max | 18.9% | Artificial Analysis |
| CritPt | Medium | 12.9% | Artificial Analysis |
| CritPt | XHigh | 22.6% | Artificial Analysis |
| FrontierCode 1.1 · extended-claude-code | High | 55.9% | Cognition |
| FrontierCode 1.1 · extended-claude-code | Low | 48.2% | Cognition |
| FrontierCode 1.1 · extended-claude-code | Max | 58.4% | Cognition |
| FrontierCode 1.1 · extended-claude-code | Medium | 55.0% | Cognition |
| FrontierCode 1.1 · extended-claude-code | XHigh | 58.1% | Cognition |
| FrontierCode 1.1 · main-claude-code | High | 41.9% | Cognition |
| FrontierCode 1.1 · main-claude-code | Low | 34.8% | Cognition |
| FrontierCode 1.1 · main-claude-code | Max | 46.4% | Cognition |
| FrontierCode 1.1 · main-claude-code | Medium | 41.6% | Cognition |
| FrontierCode 1.1 · main-claude-code | XHigh | 45.8% | Cognition |
| FrontierSWE v2 | Unspecified | 43.8% | Proximal |
| GDP.pdf: All-pass Rate | High | 17.2% | Artificial Analysis |
| GDP.pdf: All-pass Rate | Low | 11.2% | Artificial Analysis |
| GDP.pdf: All-pass Rate | Max | 20.8% | Artificial Analysis |
| GDP.pdf: All-pass Rate | Medium | 15.2% | Artificial Analysis |
| GDP.pdf: All-pass Rate | XHigh | 18.2% | Artificial Analysis |
| GDPval-AA v2.1 Leaderboard | High | 1424 | Artificial Analysis |
| GDPval-AA v2.1 Leaderboard | Low | 1130 | Artificial Analysis |
| GDPval-AA v2.1 Leaderboard | Max | 1629 | Artificial Analysis |
| GDPval-AA v2.1 Leaderboard | Medium | 1282 | Artificial Analysis |
| GDPval-AA v2.1 Leaderboard | XHigh | 1523 | Artificial Analysis |
| Harvey LAB-AA v1.1: Hallucination-Gated All-Pass Rate | High | 1.1% | Artificial Analysis |
| Harvey LAB-AA v1.1: Hallucination-Gated All-Pass Rate | Low | 1.7% | Artificial Analysis |
| Harvey LAB-AA v1.1: Hallucination-Gated All-Pass Rate | Max | 0.6% | Artificial Analysis |
| Harvey LAB-AA v1.1: Hallucination-Gated All-Pass Rate | Medium | 0.6% | Artificial Analysis |
| Harvey LAB-AA v1.1: Hallucination-Gated All-Pass Rate | XHigh | 2.8% | Artificial Analysis |
| Humanity's Last Exam | High | 37.3% | Artificial Analysis |
| Humanity's Last Exam | Low | 27.0% | Artificial Analysis |
| Humanity's Last Exam | Max | 44.4% | Artificial Analysis |
| Humanity's Last Exam | Medium | 33.8% | Artificial Analysis |
| Humanity's Last Exam | XHigh | 42.7% | Artificial Analysis |
| LiveBench · AMPS Hard · 2026-06-25 | Max | 97.0% | LiveBench |
| LiveBench · AMPS Hard · 2026-06-25 | XHigh | 98.0% | LiveBench |
| LiveBench · code completion · 2026-06-25 | Max | 78.3% | LiveBench |
| LiveBench · code completion · 2026-06-25 | XHigh | 73.9% | LiveBench |
| LiveBench · code generation · 2026-06-25 | Max | 77.5% | LiveBench |
| LiveBench · code generation · 2026-06-25 | XHigh | 78.9% | LiveBench |
| LiveBench · connections · 2026-06-25 | Max | 96.0% | LiveBench |
| LiveBench · connections · 2026-06-25 | XHigh | 93.8% | LiveBench |
| LiveBench · consecutive events · 2026-06-25 | Max | 10.6% | LiveBench |
| LiveBench · consecutive events · 2026-06-25 | XHigh | 71.0% | LiveBench |
| LiveBench · integrals with game · 2026-06-25 | Max | 67.0% | LiveBench |
| LiveBench · integrals with game · 2026-06-25 | XHigh | 89.0% | LiveBench |
| LiveBench · javascript · 2026-06-25 | Max | 68.2% | LiveBench |
| LiveBench · javascript · 2026-06-25 | XHigh | 59.1% | LiveBench |
| LiveBench · logic with navigation · 2026-06-25 | Max | 66.0% | LiveBench |
| LiveBench · logic with navigation · 2026-06-25 | XHigh | 56.0% | LiveBench |
| LiveBench · math comp · 2026-06-25 | Max | 98.0% | LiveBench |
| LiveBench · math comp · 2026-06-25 | XHigh | 97.1% | LiveBench |
| LiveBench · olympiad · 2026-06-25 | Max | 83.7% | LiveBench |
| LiveBench · olympiad · 2026-06-25 | XHigh | 87.0% | LiveBench |
| LiveBench · paraphrase · 2026-06-25 | Max | 50.2% | LiveBench |
| LiveBench · paraphrase · 2026-06-25 | XHigh | 62.5% | LiveBench |
| LiveBench · plot unscrambling · 2026-06-25 | Max | 27.9% | LiveBench |
| LiveBench · plot unscrambling · 2026-06-25 | XHigh | 42.7% | LiveBench |
| LiveBench · python · 2026-06-25 | Max | 60.0% | LiveBench |
| LiveBench · python · 2026-06-25 | XHigh | 55.0% | LiveBench |
| LiveBench · simplify · 2026-06-25 | Max | 50.3% | LiveBench |
| LiveBench · simplify · 2026-06-25 | XHigh | 62.8% | LiveBench |
| LiveBench · spatial · 2026-06-25 | Max | 92.0% | LiveBench |
| LiveBench · spatial · 2026-06-25 | XHigh | 94.0% | LiveBench |
| LiveBench · story generation · 2026-06-25 | Max | 64.5% | LiveBench |
| LiveBench · story generation · 2026-06-25 | XHigh | 73.6% | LiveBench |
| LiveBench · summarize · 2026-06-25 | Max | 53.6% | LiveBench |
| LiveBench · summarize · 2026-06-25 | XHigh | 67.0% | LiveBench |
| LiveBench · tablejoin · 2026-06-25 | Max | 46.7% | LiveBench |
| LiveBench · tablejoin · 2026-06-25 | XHigh | 49.2% | LiveBench |
| LiveBench · tablereformat · 2026-06-25 | Max | 98.0% | LiveBench |
| LiveBench · tablereformat · 2026-06-25 | XHigh | 98.0% | LiveBench |
| LiveBench · theory of mind · 2026-06-25 | Max | 86.5% | LiveBench |
| LiveBench · theory of mind · 2026-06-25 | XHigh | 76.9% | LiveBench |
| LiveBench · typescript · 2026-06-25 | Max | 43.3% | LiveBench |
| LiveBench · typescript · 2026-06-25 | XHigh | 40.0% | LiveBench |
| LiveBench · typos · 2026-06-25 | Max | 64.0% | LiveBench |
| LiveBench · typos · 2026-06-25 | XHigh | 54.0% | LiveBench |
| LiveBench · zebra puzzle · 2026-06-25 | Max | 94.0% | LiveBench |
| LiveBench · zebra puzzle · 2026-06-25 | XHigh | 98.0% | LiveBench |
| MLCR-AA | High | 15.0% | Artificial Analysis |
| MLCR-AA | Low | 12.2% | Artificial Analysis |
| MLCR-AA | Max | 35.0% | Artificial Analysis |
| MLCR-AA | Medium | 16.1% | Artificial Analysis |
| MLCR-AA | XHigh | 15.0% | Artificial Analysis |
| SciCode | High | 48.7% | Artificial Analysis |
| SciCode | Low | 49.2% | Artificial Analysis |
| SciCode | Max | 55.0% | Artificial Analysis |
| SciCode | Medium | 49.0% | Artificial Analysis |
| SciCode | XHigh | 51.7% | Artificial Analysis |
| SWE-Bench Pro V2 Hard · mini-swe-agent | High | 52.9% | Scale |
| Terminal-Bench 4.0 · claudecode | Max | 40.0% | Terminal-Bench |
| Terminal-Bench v4.0 | High | 21.7% | Artificial Analysis |
| Terminal-Bench v4.0 | Low | 12.6% | Artificial Analysis |
| Terminal-Bench v4.0 | Max | 32.8% | Artificial Analysis |
| Terminal-Bench v4.0 | Medium | 15.2% | Artificial Analysis |
| Terminal-Bench v4.0 | XHigh | 29.3% | Artificial Analysis |
| Terminal-Bench-Science 0.1 | High | 10.0% | Artificial Analysis |
| Terminal-Bench-Science 0.1 | Low | 1.9% | Artificial Analysis |
| Terminal-Bench-Science 0.1 | Max | 20.0% | Artificial Analysis |
| Terminal-Bench-Science 0.1 | Medium | 7.1% | Artificial Analysis |
| Terminal-Bench-Science 0.1 | XHigh | 18.1% | Artificial Analysis |
Benchmark scores & sources
Original results, evaluation harnesses, and evidence behind this model.
| Benchmark | Bucket | Score | Harness | Evidence | Source-recorded date |
|---|---|---|---|---|---|
| HealthBench Professional · Length-adjusted | Supporting evidence | 61.3%Reported settings & sourceSafety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. high effort. Length-adjusted score; five runs per effort. Low–xhigh onOct4; max onOct1–2. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| HealthBench Professional · Length-adjusted | Supporting evidence | 61%Reported settings & sourceSafety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. xhigh effort. Length-adjusted score; five runs per effort. Low–xhigh onOct4; max onOct1–2. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| HealthBench Professional · Length-adjusted | Supporting evidence | 64.8%Reported settings & sourceSafety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. max effort. Length-adjusted score; five runs per effort. Low–xhigh onOct4; max onOct1–2. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| HealthBench Professional · Length-adjusted / Oct4 repeat | Supporting evidence | 64.2%Reported settings & sourceSafety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. Max effort; thirteen repeat runs onOctober4; length-adjusted score, distinct dated rerun. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.11.3; p.133 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| HealthBench Professional · Raw | Supporting evidence | 61.6%Reported settings & sourceSafety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. low effort; raw score before length adjustment; five-run mean. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.11.3; p.133 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| HealthBench Professional · Raw | Supporting evidence | 71%Reported settings & sourceSafety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. max effort; raw score before length adjustment; five-run mean. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.11.3; p.133 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| Humanity's Last Exam | Hard reasoning | 30.12% | HLE no toolscontributes to capability | official board | 2026-10-08 |
| Humanity’s Last Exam · No tools | Hard reasoning | 45.9%Reported settings & sourceNo tools; max effort from summary table; thinking auto in methodology; 980K task budget, no context compaction; Claude Opus4.6 grader. Reasoning only; no tools. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · Table 8.1.A; section 8.8.1; pp.111,118–120 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| Humanity’s Last Exam · With tools | Hard reasoning | 57.4%Reported settings & sourceWith tools; max effort from summary table; thinking auto in methodology; 980K task budget, no context compaction; Claude Opus4.6 grader. Web search/fetch, programmatic tool calling, code execution; blocklist and transcript contamination screening. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · Table 8.1.A; section 8.8.1; pp.111,118–120 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| LMArena Agent Arena | Supporting evidence | -0.61 | LMArena Agent Arena · high | official board | 2026-11-18 |
| Medicinal chemistry | Supporting evidence | 58%Reported settings & sourceAPI, biology safeguards disabled; adaptive thinking max effort; five attempts/problem unless explicitly noted. Scores describe underlying model capability, not deployed safeguards. 504 questions about ADME-related measured properties of depicted drug-like compounds. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.13.3; pp.136–139 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| MILU | Knowledge | 87.6%Reported settings & sourceAdaptive thinking max effort; safety classifiers enabled; five-trial mean; no tools or customized system prompts;11-language mean. Blocked/unanswered under0.2% excluded from average. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.12.2; pp.135–136 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| Morphology-to-molecule matching | Supporting evidence | 25.4%Reported settings & sourceAPI, biology safeguards disabled; adaptive thinking max effort; five attempts/problem unless explicitly noted. Scores describe underlying model capability, not deployed safeguards. Match four human-liver assay image sets to shuffled molecule-dose perturbations. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.13.2; pp.136–139 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| OfficeQA | Agentic | 73.5%Reported settings & sourceAgentic evaluation with relevant Treasury Bulletin documents preselected as extracted text; max effort; five-run mean. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.10.1; p.130 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| OfficeQA Pro | Agentic | 60.3%Reported settings & sourceAgentic evaluation with relevant Treasury Bulletin documents preselected as extracted text; max effort; five-run mean. Harder133-question subset. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.10.1; p.130 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| OSWorld · 2.1 offline subset / Partial credit | Agentic | 72.4%Reported settings & sourceOfficial offline82-task subset of108 tasks; max effort; 1080p;500 action steps; five attempts/task; Claude Opus4.8 grader where needed; metric:Partial credit. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.9.3; pp.127–130 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| OSWorld · 2.1 offline subset / Strict pass rate | Agentic | 37.1%Reported settings & sourceOfficial offline82-task subset of108 tasks; max effort; 1080p;500 action steps; five attempts/task; Claude Opus4.8 grader where needed; metric:Strict pass rate. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.9.3; pp.127–130 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| PhysicianBench | Supporting evidence | 17.8%Reported settings & sourceSafety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. low effort. Five attempts per task across100 EHR tasks; share of attempts passed. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| PhysicianBench | Supporting evidence | 25.2%Reported settings & sourceSafety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. medium effort. Five attempts per task across100 EHR tasks; share of attempts passed. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| PhysicianBench | Supporting evidence | 31.6%Reported settings & sourceSafety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. high effort. Five attempts per task across100 EHR tasks; share of attempts passed. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| PhysicianBench | Supporting evidence | 35.8%Reported settings & sourceSafety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. xhigh effort. Five attempts per task across100 EHR tasks; share of attempts passed. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| PhysicianBench | Supporting evidence | 43%Reported settings & sourceSafety classifiers enabled; same grader across models; HealthBench/Professional graded with Claude Opus4.8; PhysicianBench with Claude Opus5. max effort. Five attempts per task across100 EHR tasks; share of attempts passed. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.11.1–3; pp.131–134 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| ProgramBench | Supporting evidence | 82%Reported settings & sourceModified 166-task subset after excluding 34 flaky-reference tasks; hidden-test pass rate; mini-swe-agent harness without upstream six-hour limit; no internet or decompilation. Effort is not explicitly stated in this section. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Not the unmodified 200-task benchmark; scores only tests passed by reference binaries. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.7.1; p.117 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| Protein design — Library Ranking | Supporting evidence | 45.3%Reported settings & sourceAPI, biology safeguards disabled; adaptive thinking max effort; five attempts/problem unless explicitly noted. Scores describe underlying model capability, not deployed safeguards. Three attempts/problem; prioritize protein designs for experimental testing. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.13.4; pp.136–139 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |
| Protein design — Sequence Generation | Supporting evidence | 33.1%Reported settings & sourceAPI, biology safeguards disabled; adaptive thinking max effort; five attempts/problem unless explicitly noted. Scores describe underlying model capability, not deployed safeguards. One attempt/problem; novel protein sequences conditioned on design constraints. Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission. Claude Haiku 5.5 System Card · section 8.13.4; pp.136–139 · reviewed 2026-10-07 | Published configuration | lab self-report | Reviewed 2026-10-07 |