RankingDeepSWE v1.1 · v1.1

DeepSWE v1.1 · v1.1

Data updated 22 Nov 2026

DeepSWE v1.1 · v1.1 is published here as its own result table, version v1.1. Scores use percent, and higher values rank better. The display harness is Grok 4.6 launch evaluations. Data updated 22 Nov 2026.

Bucket
Coding
Unit
percent
Direction
Higher is better
Version
v1.1
Display harness
Grok 4.6 launch evaluations
Board
https://x.ai/news/grok-4-6

Compare published benchmark results with category weights →

Models

Published configurations retain their source and harness labels. Missing results remain unknown.

1–8 of 8 entries

ModelScoreHarnessEvidenceSource-recorded date
GPT-5.6 SolOpenAI73%
Reported settings & source

Grok High; GPT-5.6 Sol Max; Fable 5 Max. Competitor figures from developer cards/public leaderboards as selected by xAI.

First-party reported result; comparator results retain the source evaluation setup.

Grok 4.6 launch evaluations · Performance table · reviewed 2026-09-06

Grok 4.6 launch evaluationslab self-report2026-09-06
Claude Fable 5Anthropic70%
Reported settings & source

Grok High; GPT-5.6 Sol Max; Fable 5 Max. Competitor figures from developer cards/public leaderboards as selected by xAI.

First-party reported result; comparator results retain the source evaluation setup.

Grok 4.6 launch evaluations · Performance table · reviewed 2026-09-06

Grok 4.6 launch evaluationslab self-report2026-09-06
Grok 4.6xAI65.9%
Reported settings & source

Grok High; GPT-5.6 Sol Max; Fable 5 Max. Competitor figures from developer cards/public leaderboards as selected by xAI.

First-party reported result; comparator results retain the source evaluation setup.

Grok 4.6 launch evaluations · Performance table · reviewed 2026-09-06

Grok 4.6 launch evaluationslab self-report2026-09-06
Grok-4.5xAI54%
Reported settings & source

Grok High; GPT-5.6 Sol Max; Fable 5 Max. Competitor figures from developer cards/public leaderboards as selected by xAI.

First-party reported result; comparator results retain the source evaluation setup.

Grok 4.6 launch evaluations · Performance table · reviewed 2026-09-06

Grok 4.6 launch evaluationslab self-report2026-09-06
Claude Opus 4.8Anthropic59%
Reported settings & source

Meta Model API xhigh; Gemini high; Opus max; GPT xhigh. Best of self-reported or Meta reproduction unless specified. OSWorld GUI only; Terminal bash only5attempts89tasks6CPU8GB; DeepSWE no internet5attempts.

First-party reported result; comparator results retain the source evaluation setup.

Muse Spark 1.1 evaluation report Figure44 · Figure44 physical page101; methodology pages101–105 · reviewed 2026-09-06

Muse Spark 1.1 evaluation report Figure44lab self-report2026-09-06
Gemini 3.1 ProGoogle12%
Reported settings & source

Meta Model API xhigh; Gemini high; Opus max; GPT xhigh. Best of self-reported or Meta reproduction unless specified. OSWorld GUI only; Terminal bash only5attempts89tasks6CPU8GB; DeepSWE no internet5attempts.

First-party reported result; comparator results retain the source evaluation setup.

Muse Spark 1.1 evaluation report Figure44 · Figure44 physical page101; methodology pages101–105 · reviewed 2026-09-06

Muse Spark 1.1 evaluation report Figure44lab self-report2026-09-06
GPT-5.5OpenAI67%
Reported settings & source

Meta Model API xhigh; Gemini high; Opus max; GPT xhigh. Best of self-reported or Meta reproduction unless specified. OSWorld GUI only; Terminal bash only5attempts89tasks6CPU8GB; DeepSWE no internet5attempts.

First-party reported result; comparator results retain the source evaluation setup.

Muse Spark 1.1 evaluation report Figure44 · Figure44 physical page101; methodology pages101–105 · reviewed 2026-09-06

Muse Spark 1.1 evaluation report Figure44lab self-report2026-09-06
Muse Spark 1.1Meta53.3%
Reported settings & source

Meta Model API xhigh; Gemini high; Opus max; GPT xhigh. Best of self-reported or Meta reproduction unless specified. OSWorld GUI only; Terminal bash only5attempts89tasks6CPU8GB; DeepSWE no internet5attempts.

First-party reported result; comparator results retain the source evaluation setup.

Muse Spark 1.1 evaluation report Figure44 · Figure44 physical page101; methodology pages101–105 · reviewed 2026-09-06

Muse Spark 1.1 evaluation report Figure44lab self-report2026-09-06