RankingSWE-bench Verified

SWE-bench Verified

Data updated 22 Oct 2026

Bucket
Coding
Unit
percent
Direction
Higher is better
Version
verified
Display harness
SWE-bench Verified official
Board
https://www.swebench.com/

Compare published benchmark results with category weights →

Models

Results are ordered by score in the display harness, followed by models with no result. Other harnesses are listed separately below. Missing results remain unknown.

1–25 of 471 entries

ModelScoreEvidenceSource-recorded date
Claude Opus 5Anthropic96%lab self-report

The cited result does not establish the same preferred agent and task/runtime configuration; matching benchmark title alone is insufficient.

2026-07-24
DeepSeek V4-Pro 0813DeepSeek80.6%lab self-report

The cited result does not establish the same preferred agent and task/runtime configuration; matching benchmark title alone is insufficient.

2026-08-13
Gemini 3.1 ProGoogle80.6%lab self-report

Google methodology explicitly uses custom scaffolding rather than mini-SWE-agent; Verified also adjusts three tasks after harness fixes.

2026-02-19
Gemini 3 Flash PreviewGoogle75.8%official board2026-02-17
MiniMax-M2.5MiniMax75.8%official board2026-02-17
Claude Opus 4.6Anthropic75.6%official board2026-02-17
Claude Opus 4.5Anthropic74.4%official board2025-11-24
GLM-5Z.ai72.8%official board2026-02-17
NVIDIA Nemotron 3 UltraNVIDIA71.9%lab self-report

Current source table reports70.7 BF16/69.5 NVFP4, not fixture71.9; exact observation and preferred agent cannot be verified.

2026-06-04
Claude Sonnet 4.5Anthropic71.4%official board2026-02-17
DeepSeek-V3.2DeepSeek70%official board2026-02-17
Claude Haiku 4.5Anthropic66.6%official board2026-02-17
MiniMax-M2MiniMax61%official board2025-11-24
o3OpenAI58.4%official board2025-07-26
GLM-4.6Z.ai55.4%official board2025-12-01
Qwen3-Coder-480B-A35B-InstructQwen55.4%official board2025-08-02
GLM-4.5Z.ai54.2%official board2025-08-22
Devstral 2Mistral53.8%official board2025-12-09
Gemini 2.5 ProGoogle53.6%official board2025-07-26
o4-miniOpenAI45%official board2025-07-26
Gemini 2.5 FlashGoogle28.73%official board2025-07-26
gpt-oss-120bOpenAI26%official board2025-08-07
Llama 4 MaverickMeta21.04%official board2025-07-20
Agnes 2.5 Pro AlphaAgnes AI———
Agnes-3.0-Flash PreviewAgnes AI———