RankingMulti-step SWE

Multi-step SWE

Data updated 22 Nov 2026

Multi-step SWE is published here as its own result table, version 1.0. Scores use percent, and higher values rank better. The display harness is Claude Code. Data updated 22 Nov 2026.

Bucket
Supporting evidence
Unit
percent
Direction
Higher is better
Version
1.0
Display harness
Claude Code
Board
https://multistepswe.com/

Multi-step SWE is the Klavis AI board of 69 chained tasks in Harbor format. A rollout counts only when every step gets full credit. The table shows the published solve rate from four rollouts, with the paper's 95% task bootstrap interval. These rows stay out of weighted Overall.

Compare published benchmark results with category weights →

Models

Results are ordered by score in the display harness, followed by models with no result. Other harnesses are listed separately below. Missing results remain unknown.

376–400 of 593 entries

ModelScoreEvidenceSource-recorded date
Ministral 3 14BMistral———
Ministral 3 3BMistral———
Ministral 3 8BMistral———
Ministral 3BMistral———
Ministral 8BMistral———
Mistral 7B InstructMistral———
Mistral LargeMistral———
Mistral Large 2Mistral———
Mistral Large 2 (July 2024)Mistral———
Mistral Large 2.1Mistral———
Mistral Large 3Mistral———
Mistral Large 4Mistral———
Mistral MediumMistral———
Mistral Medium 3Mistral———
Mistral Medium 3.1Mistral———
Mistral Medium 3.5Mistral———
Mistral Nemo 12BMistral———
Mistral SabaMistral———
Mistral SmallMistral———
Mistral Small 3Mistral———
Mistral Small 3.1Mistral———
Mistral Small 3.2Mistral———
Mistral Small 4Mistral———
Mistral Small CreativeMistral———
Mixtral 8x22B InstructMistral———

Other harnesses

These runs use other harnesses. Capability scoring matches configurations separately; model details identify which source records contribute.

ModelScoreHarnessEvidenceSource-recorded date
Gemini 4 ArgonGoogle10.1%mini-swe-agentofficial board2026-11-06
GPT-6 AstraOpenAI6.9%Codex CLIofficial board2026-11-06
GPT-6.1 SolOpenAI10.1%Codex CLIofficial board2026-11-06
Grok 4.7xAI1.4%Grok Buildofficial board2026-11-06