RankingMulti-step SWE

Multi-step SWE

Data updated 22 Nov 2026

Multi-step SWE is published here as its own result table, version 1.0. Scores use percent, and higher values rank better. The display harness is Claude Code. Data updated 22 Nov 2026.

Bucket
Supporting evidence
Unit
percent
Direction
Higher is better
Version
1.0
Display harness
Claude Code
Board
https://multistepswe.com/

Multi-step SWE is the Klavis AI board of 69 chained tasks in Harbor format. A rollout counts only when every step gets full credit. The table shows the published solve rate from four rollouts, with the paper's 95% task bootstrap interval. These rows stay out of weighted Overall.

Compare published benchmark results with category weights →

Models

Results are ordered by score in the display harness, followed by models with no result. Other harnesses are listed separately below. Missing results remain unknown.

1–25 of 593 entries

ModelScoreEvidenceSource-recorded date
Claude Opus 5.5Anthropic18.5%official board2026-11-06
Claude Fable 5.1Anthropic17.8%official board2026-11-06
DeepSeek V4.1 FlashDeepSeek6.5%official board2026-11-06
Qwen 3.8-MaxQwen4%official board2026-11-06
Agnes 2.5 Pro AlphaAgnes AI———
Agnes 2.5 Pro BetaAgnes AI———
Agnes 2.5 Pro ThetaAgnes AI———
Agnes-3.0-Flash PreviewAgnes AI———
Amazon Nova 2 LiteAmazon———
Amazon Nova 2 Pro PreviewAmazon———
Amazon Nova LiteAmazon———
Amazon Nova MicroAmazon———
Amazon Nova PremierAmazon———
Amazon Nova ProAmazon———
Astra for Law 6.2OpenAI———
AutoGLM-Phone-9BZ.ai———
AutoGLM-Phone-9B-MultilingualZ.ai———
aya-101Cohere———
aya-23-35BCohere———
aya-23-8BCohere———
aya-expanse-8bCohere———
aya-vision-8bCohere———
C4Ai Aya Expanse 32BCohere———
C4Ai Aya Vision 32BCohere———
c4ai-command-r7b-arabic-02-2025Cohere———

Other harnesses

These runs use other harnesses. Capability scoring matches configurations separately; model details identify which source records contribute.

ModelScoreHarnessEvidenceSource-recorded date
Gemini 4 ArgonGoogle10.1%mini-swe-agentofficial board2026-11-06
GPT-6 AstraOpenAI6.9%Codex CLIofficial board2026-11-06
GPT-6.1 SolOpenAI10.1%Codex CLIofficial board2026-11-06
Grok 4.7xAI1.4%Grok Buildofficial board2026-11-06