RankingDeepSWE v1.1 · v1.1
DeepSWE v1.1 · v1.1
- Bucket
- Coding
- Unit
- percent
- Direction
- Higher is better
- Version
- v1.1
- Display harness
- Grok 4.6 launch evaluations
- Board
- https://x.ai/news/grok-4-6
Compare published benchmark results with category weights →
Models
| Model | Score | Harness | Evidence | Source-recorded date |
|---|---|---|---|---|
| GPT-5.6 SolOpenAI | 73%Reported settings & sourceGrok High; GPT-5.6 Sol Max; Fable 5 Max. Competitor figures from developer cards/public leaderboards as selected by xAI. First-party reported result; comparator results retain the source evaluation setup. Grok 4.6 launch evaluations · Performance table · reviewed 2026-09-06 | Grok 4.6 launch evaluations | lab self-report | 2026-09-06 |
| Claude Fable 5Anthropic | 70%Reported settings & sourceGrok High; GPT-5.6 Sol Max; Fable 5 Max. Competitor figures from developer cards/public leaderboards as selected by xAI. First-party reported result; comparator results retain the source evaluation setup. Grok 4.6 launch evaluations · Performance table · reviewed 2026-09-06 | Grok 4.6 launch evaluations | lab self-report | 2026-09-06 |
| Grok 4.6xAI | 65.9%Reported settings & sourceGrok High; GPT-5.6 Sol Max; Fable 5 Max. Competitor figures from developer cards/public leaderboards as selected by xAI. First-party reported result; comparator results retain the source evaluation setup. Grok 4.6 launch evaluations · Performance table · reviewed 2026-09-06 | Grok 4.6 launch evaluations | lab self-report | 2026-09-06 |
| Grok-4.5xAI | 54%Reported settings & sourceGrok High; GPT-5.6 Sol Max; Fable 5 Max. Competitor figures from developer cards/public leaderboards as selected by xAI. First-party reported result; comparator results retain the source evaluation setup. Grok 4.6 launch evaluations · Performance table · reviewed 2026-09-06 | Grok 4.6 launch evaluations | lab self-report | 2026-09-06 |
| Claude Opus 4.8Anthropic | 59%Reported settings & sourceMeta Model API xhigh; Gemini high; Opus max; GPT xhigh. Best of self-reported or Meta reproduction unless specified. OSWorld GUI only; Terminal bash only5attempts89tasks6CPU8GB; DeepSWE no internet5attempts. First-party reported result; comparator results retain the source evaluation setup. Muse Spark 1.1 evaluation report Figure44 · Figure44 physical page101; methodology pages101–105 · reviewed 2026-09-06 | Muse Spark 1.1 evaluation report Figure44 | lab self-report | 2026-09-06 |
| Gemini 3.1 ProGoogle | 12%Reported settings & sourceMeta Model API xhigh; Gemini high; Opus max; GPT xhigh. Best of self-reported or Meta reproduction unless specified. OSWorld GUI only; Terminal bash only5attempts89tasks6CPU8GB; DeepSWE no internet5attempts. First-party reported result; comparator results retain the source evaluation setup. Muse Spark 1.1 evaluation report Figure44 · Figure44 physical page101; methodology pages101–105 · reviewed 2026-09-06 | Muse Spark 1.1 evaluation report Figure44 | lab self-report | 2026-09-06 |
| GPT-5.5OpenAI | 67%Reported settings & sourceMeta Model API xhigh; Gemini high; Opus max; GPT xhigh. Best of self-reported or Meta reproduction unless specified. OSWorld GUI only; Terminal bash only5attempts89tasks6CPU8GB; DeepSWE no internet5attempts. First-party reported result; comparator results retain the source evaluation setup. Muse Spark 1.1 evaluation report Figure44 · Figure44 physical page101; methodology pages101–105 · reviewed 2026-09-06 | Muse Spark 1.1 evaluation report Figure44 | lab self-report | 2026-09-06 |
| Muse Spark 1.1Meta | 53.3%Reported settings & sourceMeta Model API xhigh; Gemini high; Opus max; GPT xhigh. Best of self-reported or Meta reproduction unless specified. OSWorld GUI only; Terminal bash only5attempts89tasks6CPU8GB; DeepSWE no internet5attempts. First-party reported result; comparator results retain the source evaluation setup. Muse Spark 1.1 evaluation report Figure44 · Figure44 physical page101; methodology pages101–105 · reviewed 2026-09-06 | Muse Spark 1.1 evaluation report Figure44 | lab self-report | 2026-09-06 |