RankingMulti-step SWE

Multi-step SWE

Data updated 22 Nov 2026

Multi-step SWE is published here as its own result table, version 1.0. Scores use percent, and higher values rank better. The display harness is Claude Code. Data updated 22 Nov 2026.

Bucket
Supporting evidence
Unit
percent
Direction
Higher is better
Version
1.0
Display harness
Claude Code
Board
https://multistepswe.com/

Multi-step SWE is the Klavis AI board of 69 chained tasks in Harbor format. A rollout counts only when every step gets full credit. The table shows the published solve rate from four rollouts, with the paper's 95% task bootstrap interval. These rows stay out of weighted Overall.

Compare published benchmark results with category weights →

Models

Results are ordered by score in the display harness, followed by models with no result. Other harnesses are listed separately below. Missing results remain unknown.

76–100 of 593 entries

ModelScoreEvidenceSource-recorded date
Command A Reasoning 08 2025Cohere———
Command A Translate 08 2025Cohere———
Command A Vision 07 2025Cohere———
Command LightCohere———
Command R 03 2024Cohere———
Command R 08 2024Cohere———
Command R Plus 04 2024Cohere———
Command R Plus 08 2024Cohere———
Command R7B 12 2024Cohere———
Deepseek Coder V2DeepSeek———
DeepSeek Coder V2 Lite InstructDeepSeek———
DeepSeek R1DeepSeek———
Deepseek V2.5DeepSeek———
DeepSeek V3DeepSeek———
DeepSeek V3 0324DeepSeek———
DeepSeek V3.1DeepSeek———
DeepSeek V3.1 TerminusDeepSeek———
DeepSeek V3.2 ExpDeepSeek———
DeepSeek V4 Flash 0424DeepSeek———
DeepSeek V4 Flash VisionDeepSeek———
DeepSeek V4-Pro 0424DeepSeek———
DeepSeek V4-Pro 0813DeepSeek———
DeepSeek V4.1 Flash 1017DeepSeek———
DeepSeek-R1-0528DeepSeek———
DeepSeek-R1-0528-Qwen3-8BDeepSeek———

Other harnesses

These runs use other harnesses. Capability scoring matches configurations separately; model details identify which source records contribute.

ModelScoreHarnessEvidenceSource-recorded date
Gemini 4 ArgonGoogle10.1%mini-swe-agentofficial board2026-11-06
GPT-6 AstraOpenAI6.9%Codex CLIofficial board2026-11-06
GPT-6.1 SolOpenAI10.1%Codex CLIofficial board2026-11-06
Grok 4.7xAI1.4%Grok Buildofficial board2026-11-06