RankingMulti-step SWE
Multi-step SWE
Multi-step SWE is published here as its own result table, version 1.0. Scores use percent, and higher values rank better. The display harness is Claude Code. Data updated 22 Nov 2026.
- Bucket
- Supporting evidence
- Unit
- percent
- Direction
- Higher is better
- Version
- 1.0
- Display harness
- Claude Code
- Board
- https://multistepswe.com/
Multi-step SWE is the Klavis AI board of 69 chained tasks in Harbor format. A rollout counts only when every step gets full credit. The table shows the published solve rate from four rollouts, with the paper's 95% task bootstrap interval. These rows stay out of weighted Overall.
Compare published benchmark results with category weights →
Models
1–25 of 593 entries
| Model | Score | Evidence | Source-recorded date |
|---|---|---|---|
| Claude Opus 5.5Anthropic | 18.5% | official board | 2026-11-06 |
| Claude Fable 5.1Anthropic | 17.8% | official board | 2026-11-06 |
| DeepSeek V4.1 FlashDeepSeek | 6.5% | official board | 2026-11-06 |
| Qwen 3.8-MaxQwen | 4% | official board | 2026-11-06 |
| Agnes 2.5 Pro AlphaAgnes AI | — | — | — |
| Agnes 2.5 Pro BetaAgnes AI | — | — | — |
| Agnes 2.5 Pro ThetaAgnes AI | — | — | — |
| Agnes-3.0-Flash PreviewAgnes AI | — | — | — |
| Amazon Nova 2 LiteAmazon | — | — | — |
| Amazon Nova 2 Pro PreviewAmazon | — | — | — |
| Amazon Nova LiteAmazon | — | — | — |
| Amazon Nova MicroAmazon | — | — | — |
| Amazon Nova PremierAmazon | — | — | — |
| Amazon Nova ProAmazon | — | — | — |
| Astra for Law 6.2OpenAI | — | — | — |
| AutoGLM-Phone-9BZ.ai | — | — | — |
| AutoGLM-Phone-9B-MultilingualZ.ai | — | — | — |
| aya-101Cohere | — | — | — |
| aya-23-35BCohere | — | — | — |
| aya-23-8BCohere | — | — | — |
| aya-expanse-8bCohere | — | — | — |
| aya-vision-8bCohere | — | — | — |
| C4Ai Aya Expanse 32BCohere | — | — | — |
| C4Ai Aya Vision 32BCohere | — | — | — |
| c4ai-command-r7b-arabic-02-2025Cohere | — | — | — |
Other harnesses
These runs use other harnesses. Capability scoring matches configurations separately; model details identify which source records contribute.
| Model | Score | Harness | Evidence | Source-recorded date |
|---|---|---|---|---|
| Gemini 4 ArgonGoogle | 10.1% | mini-swe-agent | official board | 2026-11-06 |
| GPT-6 AstraOpenAI | 6.9% | Codex CLI | official board | 2026-11-06 |
| GPT-6.1 SolOpenAI | 10.1% | Codex CLI | official board | 2026-11-06 |
| Grok 4.7xAI | 1.4% | Grok Build | official board | 2026-11-06 |