RankingDeepSWE · 1.1

DeepSWE · 1.1

Data updated 22 Nov 2026

DeepSWE · 1.1 is published here as its own result table, version 1.1. Scores use percent, and higher values rank better. The display harness is Claude Fable 5.1 and Claude Mythos 5.1 System Card. Data updated 22 Nov 2026.

Bucket
Coding
Unit
percent
Direction
Higher is better
Version
1.1
Display harness
Claude Fable 5.1 and Claude Mythos 5.1 System Card
Board
https://www.anthropic.com/claude-fable-5-1-mythos-5-1-system-card

Compare published benchmark results with category weights →

Models

Published configurations retain their source and harness labels. Missing results remain unknown.

1–15 of 15 entries

ModelScoreHarnessEvidenceSource-recorded date
Claude Fable 5.1Anthropic67.4%
Reported settings & source

Anthropic reported configuration; adaptive thinking at max effort, default sampling, five-trial mean unless section specifies otherwise; context at most 1M. Comparator configuration follows cited prior card or board. 113 tasks; original hidden-test grading.

Fable is the safeguarded deployed configuration; selected tasks may use disclosed Opus fallback, except evaluations explicitly counting safety blocks as failures.

Claude Fable 5.1 and Claude Mythos 5.1 System Card · 8.3 · reviewed 2026-11-22

Claude Fable 5.1 and Claude Mythos 5.1 System Cardlab self-report2026-11-22
Claude Fable 5.1Anthropic67.4%
Reported settings & source

Comparator maximum available thinking/reasoning when reported, otherwise best available result; exact setting not identified by this grid. Argon self-computed with mini-swe agent. Astra from the official public leaderboard; Fable and Opus from their system cards. Highest scoring thinking level per Datacurve.

Provider-published launch claim, not independent measurement by Google for every row. Argon self-computed with mini-swe agent. Astra from the official public leaderboard; Fable and Opus from their system cards. Highest scoring thinking level per Datacurve. Methodology https://deepmind.google/models/evals-methodology/gemini-4-argon, SHA256 ff1df6bdeddc4c0f48840c09a8a7813b10ae7ad9e053892da68dabf64f09bff7. Preserve source rounding. Methodology says capabilities as of September 2026 and results as of October 2026; review date is source retrieval, not evaluation date.

Comparison limit: Provider-published claim. Independent source results are admitted separately with exact checkpoint, effort and compatible protocol; this grid does not establish a matched comparison.

Google Gemini 4 Argon launch and evaluation methodology · Launch grid row 5: DeepSWE 1.1 / Claude Fable 5.1; methodology Additional Details · reviewed 2026-09-30

Google Gemini 4 Argon launch and evaluation methodologylab self-report2026-09-30
Claude Opus 5.5Anthropic74.2%
Reported settings & source

113 long-horizon tasks; five-trial mean. Section 8.3 does not state reasoning effort.

Provider-published lab self-report from the Claude Opus 5.5 system card (SHA256 7311c9c6bbb16d012f1c12c7418b05949fcf7ae3e30d2c40f22050074b2a7378). Not an independent board. Effort is not stated in this section, so it is not copied from the Table 8.1.A max-effort default.

Comparison limit: Provider-published lab self-report. Shown as a labeled claim and not admitted as a matched board comparison.

Claude Opus 5.5 System Card · section 8.3 · reviewed 2026-09-22

Claude Opus 5.5 System Cardlab self-report2026-09-22
Claude Opus 5.5Anthropic74.2%
Reported settings & source

Comparator maximum available thinking/reasoning when reported, otherwise best available result; exact setting not identified by this grid. Argon self-computed with mini-swe agent. Astra from the official public leaderboard; Fable and Opus from their system cards. Highest scoring thinking level per Datacurve.

Provider-published launch claim, not independent measurement by Google for every row. Argon self-computed with mini-swe agent. Astra from the official public leaderboard; Fable and Opus from their system cards. Highest scoring thinking level per Datacurve. Methodology https://deepmind.google/models/evals-methodology/gemini-4-argon, SHA256 ff1df6bdeddc4c0f48840c09a8a7813b10ae7ad9e053892da68dabf64f09bff7. Preserve source rounding. Methodology says capabilities as of September 2026 and results as of October 2026; review date is source retrieval, not evaluation date.

Comparison limit: Provider-published claim. Independent source results are admitted separately with exact checkpoint, effort and compatible protocol; this grid does not establish a matched comparison.

Google Gemini 4 Argon launch and evaluation methodology · Launch grid row 5: DeepSWE 1.1 / Claude Opus 5.5; methodology Additional Details · reviewed 2026-09-30

Google Gemini 4 Argon launch and evaluation methodologylab self-report2026-09-30
Claude Sonnet 5Anthropic53.8%
Reported settings & source

Datacurve highest reported effort; Gemini3.8selfcomputed mini-swe-agent highthinking

Card Opus74 explicitly corrected as erroneous in linkedmethodology; omitted.

Gemini3.8Flash Model Card · Model card page5 Results table · reviewed 2026-09-06

Gemini3.8Flash Model Cardlab self-report2026-09-06
Gemini 3.7 FlashGoogle65.3%
Reported settings & source

Datacurve highest reported effort; Gemini3.8selfcomputed mini-swe-agent highthinking

Card Opus74 explicitly corrected as erroneous in linkedmethodology; omitted.

Gemini3.8Flash Model Card · Model card page5 Results table · reviewed 2026-09-06

Gemini3.8Flash Model Cardlab self-report2026-09-06
Gemini 3.8 FlashGoogle73.7%
Reported settings & source

Datacurve highest reported effort; Gemini3.8selfcomputed mini-swe-agent highthinking

Card Opus74 explicitly corrected as erroneous in linkedmethodology; omitted.

Gemini3.8Flash Model Card · Model card page5 Results table · reviewed 2026-09-06

Gemini3.8Flash Model Cardlab self-report2026-09-06
Gemini 4 ArgonGoogle77.9%
Reported settings & source

Gemini API highest thinking settings; exact named effort not specified by Google. Argon self-computed with mini-swe agent. Astra from the official public leaderboard; Fable and Opus from their system cards. Highest scoring thinking level per Datacurve.

Provider-published launch claim, not independent measurement by Google for every row. Argon self-computed with mini-swe agent. Astra from the official public leaderboard; Fable and Opus from their system cards. Highest scoring thinking level per Datacurve. Methodology https://deepmind.google/models/evals-methodology/gemini-4-argon, SHA256 ff1df6bdeddc4c0f48840c09a8a7813b10ae7ad9e053892da68dabf64f09bff7. Preserve source rounding. Methodology says capabilities as of September 2026 and results as of October 2026; review date is source retrieval, not evaluation date.

Comparison limit: Provider-published claim. Independent source results are admitted separately with exact checkpoint, effort and compatible protocol; this grid does not establish a matched comparison.

Google Gemini 4 Argon launch and evaluation methodology · Launch grid row 5: DeepSWE 1.1 / Gemini 4 Argon; methodology Additional Details · reviewed 2026-09-30

Google Gemini 4 Argon launch and evaluation methodologylab self-report2026-09-30
GPT-5.6 SolOpenAI72.7%
Reported settings & source

Datacurve highest reported effort; Gemini3.8selfcomputed mini-swe-agent highthinking

Card Opus74 explicitly corrected as erroneous in linkedmethodology; omitted.

Gemini3.8Flash Model Card · Model card page5 Results table · reviewed 2026-09-06

Gemini3.8Flash Model Cardlab self-report2026-09-06
GPT-5.6 SolOpenAI73%
Reported settings & source

113tasks; Muse1.3mini-swe-agent; comparators officialDatacurve board; reasoning max

Opus74 omitted because Google current methodology explicitly identifies that board-rounded value as incorrect; underlying precision unresolved.

Muse Spark1.3 evaluation methodology · PDF page4 performance table · reviewed 2026-09-06

Muse Spark1.3 evaluation methodologylab self-report2026-09-06
GPT-5.6 TerraOpenAI69.6%
Reported settings & source

Datacurve highest reported effort; Gemini3.8selfcomputed mini-swe-agent highthinking

Card Opus74 explicitly corrected as erroneous in linkedmethodology; omitted.

Gemini3.8Flash Model Card · Model card page5 Results table · reviewed 2026-09-06

Gemini3.8Flash Model Cardlab self-report2026-09-06
GPT-6 AstraOpenAI74.1%
Reported settings & source

Comparator maximum available thinking/reasoning when reported, otherwise best available result; exact setting not identified by this grid. Argon self-computed with mini-swe agent. Astra from the official public leaderboard; Fable and Opus from their system cards. Highest scoring thinking level per Datacurve.

Provider-published launch claim, not independent measurement by Google for every row. Argon self-computed with mini-swe agent. Astra from the official public leaderboard; Fable and Opus from their system cards. Highest scoring thinking level per Datacurve. Methodology https://deepmind.google/models/evals-methodology/gemini-4-argon, SHA256 ff1df6bdeddc4c0f48840c09a8a7813b10ae7ad9e053892da68dabf64f09bff7. Preserve source rounding. Methodology says capabilities as of September 2026 and results as of October 2026; review date is source retrieval, not evaluation date.

Comparison limit: Provider-published claim. Independent source results are admitted separately with exact checkpoint, effort and compatible protocol; this grid does not establish a matched comparison.

Google Gemini 4 Argon launch and evaluation methodology · Launch grid row 5: DeepSWE 1.1 / GPT-6 Astra; methodology Additional Details · reviewed 2026-09-30

Google Gemini 4 Argon launch and evaluation methodologylab self-report2026-09-30
Mistral Large 4Mistral61.7%
Reported settings & source

Public Mistral Large4 Preview. Effort, sampling and evaluation budgets are not stated. Chart footnote attributes private Artificial Analysis evaluation before public harness launch; provider-republished claim, not a new independent result.

Provider-published lab_self_report; retained as launch evidence, not an independent board admission. Narrative states61.7%; chart rounds to62. Preserve narrative precision. Exact printed chart label; asset https://mistral.ai/_astro/artificial-analysis---deepswe-1.1%201-alt_Z28T39q.webp?dpl=6ac6a8353c2a68969e300fd2; asset SHA256 df82a5aa879ff37b505c514b9b99d73580b194af137fabe772aef827c679d2f0.

Comparison limit: Provider publication; exact benchmark-specific configuration and independent evaluation provenance require separate review before matched-board admission.

Introducing Mistral Large 4 — public preview · Capabilities deep-dive; DeepSWE; chart 0 (mistral-chart-0.webp) · reviewed 2026-10-07

Introducing Mistral Large 4 — public previewlab self-report2026-10-07
Muse Spark 1.2Meta55%
Reported settings & source

113tasks; Muse1.3mini-swe-agent; comparators officialDatacurve board; reasoning xhigh

Opus74 omitted because Google current methodology explicitly identifies that board-rounded value as incorrect; underlying precision unresolved.

Muse Spark1.3 evaluation methodology · PDF page4 performance table · reviewed 2026-09-06

Muse Spark1.3 evaluation methodologylab self-report2026-09-06
Muse Spark 1.3Meta75.4%
Reported settings & source

113tasks; Muse1.3mini-swe-agent; comparators officialDatacurve board; reasoning max

Opus74 omitted because Google current methodology explicitly identifies that board-rounded value as incorrect; underlying precision unresolved.

Muse Spark1.3 evaluation methodology · PDF page4 performance table · reviewed 2026-09-06

Muse Spark1.3 evaluation methodologylab self-report2026-09-06