RankingMulti-step SWE

Multi-step SWE

Data updated 22 Nov 2026

Multi-step SWE is published here as its own result table, version 1.0. Scores use percent, and higher values rank better. The display harness is Claude Code. Data updated 22 Nov 2026.

Bucket
Supporting evidence
Unit
percent
Direction
Higher is better
Version
1.0
Display harness
Claude Code
Board
https://multistepswe.com/

Multi-step SWE is the Klavis AI board of 69 chained tasks in Harbor format. A rollout counts only when every step gets full credit. The table shows the published solve rate from four rollouts, with the paper's 95% task bootstrap interval. These rows stay out of weighted Overall.

Compare published benchmark results with category weights →

Models

Results are ordered by score in the display harness, followed by models with no result. Other harnesses are listed separately below. Missing results remain unknown.

301–325 of 593 entries

ModelScoreEvidenceSource-recorded date
Kimi-Dev-72BMoonshot———
Kimi-K2.6Moonshot———
Kimi-K2.7-CodeMoonshot———
Kimi-Linear-48B-A3B-InstructMoonshot———
Kimi-VL-A3B-InstructMoonshot———
Kimi-VL-A3B-Thinking-2506Moonshot———
Leanstral 1.5Mistral———
Llama 2 Chat 13BMeta———
Llama 2 Chat 70BMeta———
Llama 2 Chat 7BMeta———
Llama 3 Instruct 70BMeta———
Llama 3 Instruct 8BMeta———
Llama 3.1 Instruct 405BMeta———
Llama 3.1 Instruct 70BMeta———
Llama 3.1 Instruct 8BMeta———
Llama 3.1 Nemotron Instruct 70BNVIDIA———
Llama 3.1 Nemotron Nano 4B v1.1NVIDIA———
Llama 3.1 Nemotron Ultra 253B v1NVIDIA———
Llama 3.2 Instruct 11BMeta———
Llama 3.2 Instruct 1BMeta———
Llama 3.2 Instruct 3BMeta———
Llama 3.2 Instruct 90BMeta———
Llama 3.3 Instruct 70BMeta———
Llama 3.3 Nemotron Super 49B v1NVIDIA———
Llama 4 MaverickMeta———

Other harnesses

These runs use other harnesses. Capability scoring matches configurations separately; model details identify which source records contribute.

ModelScoreHarnessEvidenceSource-recorded date
Gemini 4 ArgonGoogle10.1%mini-swe-agentofficial board2026-11-06
GPT-6 AstraOpenAI6.9%Codex CLIofficial board2026-11-06
GPT-6.1 SolOpenAI10.1%Codex CLIofficial board2026-11-06
Grok 4.7xAI1.4%Grok Buildofficial board2026-11-06