RankingMulti-step SWE

Multi-step SWE

Data updated 22 Nov 2026

Multi-step SWE is published here as its own result table, version 1.0. Scores use percent, and higher values rank better. The display harness is Claude Code. Data updated 22 Nov 2026.

Bucket
Supporting evidence
Unit
percent
Direction
Higher is better
Version
1.0
Display harness
Claude Code
Board
https://multistepswe.com/

Multi-step SWE is the Klavis AI board of 69 chained tasks in Harbor format. A rollout counts only when every step gets full credit. The table shows the published solve rate from four rollouts, with the paper's 95% task bootstrap interval. These rows stay out of weighted Overall.

Compare published benchmark results with category weights →

Models

Results are ordered by score in the display harness, followed by models with no result. Other harnesses are listed separately below. Missing results remain unknown.

326–350 of 593 entries

ModelScoreEvidenceSource-recorded date
Llama Nemotron Super 49B v1.5NVIDIA———
Llama-3.1-405B-InstructMeta———
Llama-3.1-70B-InstructMeta———
Llama-3.1-8B-InstructMeta———
Llama-3.2-11B-Vision-InstructMeta———
Llama-3.2-1B-InstructMeta———
Llama-3.2-3B-InstructMeta———
Llama-3.2-90B-Vision-InstructMeta———
Llama-3.3-70B-InstructMeta———
Llama-4-Scout-17B-16E-InstructMeta———
Magistral Medium 1Mistral———
Magistral Medium 1.2Mistral———
Magistral Small 1Mistral———
Magistral Small 1.2Mistral———
MAI-Code-1-FlashMicrosoft———
MAI-Code-1.1-FlashMicrosoft———
MAI-DS-R1Microsoft———
MAI-Thinking-1Microsoft———
Mimo V2 OmniXiaomi———
Mimo V2 Omni 0327Xiaomi———
Mimo V2 ProXiaomi———
MiMo-7B-RLXiaomi———
MiMo-7B-RL-0530Xiaomi———
MiMo-7B-RL-ZeroXiaomi———
MiMo-7B-SFTXiaomi———

Other harnesses

These runs use other harnesses. Capability scoring matches configurations separately; model details identify which source records contribute.

ModelScoreHarnessEvidenceSource-recorded date
Gemini 4 ArgonGoogle10.1%mini-swe-agentofficial board2026-11-06
GPT-6 AstraOpenAI6.9%Codex CLIofficial board2026-11-06
GPT-6.1 SolOpenAI10.1%Codex CLIofficial board2026-11-06
Grok 4.7xAI1.4%Grok Buildofficial board2026-11-06