UnifyBench

RankingSWE-bench Verified · source release snapshot; version not specified

SWE-bench Verified · source release snapshot; version not specified

Data updated 12 Sept 2026

Bucket
Coding
Unit
percent
Direction
Higher is better
Version
source release snapshot; version not specified
Display harness
MAI-Thinking-1 technical report
Board
https://microsoft.ai/pdf/mai-thinking-1.pdf

Compare published benchmark results with category weights →

Models

ModelScoreHarnessEvidenceSource-recorded date
Claude Opus 4.6Anthropic80.8%
Reported settings & source

MAI avg 4 runs, temperature 1, top-p .97; simple ReAct bash/string-replace; Terminal-Bench ignores predefined timeouts. Comparator configurations from cited reports.

First-party reported result; comparator results retain the source evaluation setup.

MAI-Thinking-1 technical report · Table 11, PDF page 53 · reviewed 2026-09-06

MAI-Thinking-1 technical reportlab self-report2026-09-06
Kimi-K2.6Moonshot80.2%
Reported settings & source

MAI avg 4 runs, temperature 1, top-p .97; simple ReAct bash/string-replace; Terminal-Bench ignores predefined timeouts. Comparator configurations from cited reports.

First-party reported result; comparator results retain the source evaluation setup.

MAI-Thinking-1 technical report · Table 11, PDF page 53 · reviewed 2026-09-06

MAI-Thinking-1 technical reportlab self-report2026-09-06
Claude Sonnet 4.6Anthropic79.6%
Reported settings & source

MAI avg 4 runs, temperature 1, top-p .97; simple ReAct bash/string-replace; Terminal-Bench ignores predefined timeouts. Comparator configurations from cited reports.

First-party reported result; comparator results retain the source evaluation setup.

MAI-Thinking-1 technical report · Table 11, PDF page 53 · reviewed 2026-09-06

MAI-Thinking-1 technical reportlab self-report2026-09-06
MAI-Thinking-1Microsoft73.5%
Reported settings & source

MAI avg 4 runs, temperature 1, top-p .97; simple ReAct bash/string-replace; Terminal-Bench ignores predefined timeouts. Comparator configurations from cited reports.

First-party reported result; comparator results retain the source evaluation setup.

MAI-Thinking-1 technical report · Table 11, PDF page 53 · reviewed 2026-09-06

MAI-Thinking-1 technical reportlab self-report2026-09-06
Amazon Nova 2 LiteAmazon53.6%
Reported settings & source

Nova2 internal agentic scaffold; scaled inference explicitly separate. Comparator setups from cited provider report.

First-party reported result; comparator results retain the source evaluation setup.

Amazon Nova 2 technical report · Table4 · reviewed 2026-09-06

Amazon Nova 2 technical reportlab self-report2026-09-06
Amazon Nova 2 Pro PreviewAmazon61.5%
Reported settings & source

Nova2 internal agentic scaffold; scaled inference explicitly separate. Comparator setups from cited provider report.

First-party reported result; comparator results retain the source evaluation setup.

Amazon Nova 2 technical report · Table4 · reviewed 2026-09-06

Amazon Nova 2 technical reportlab self-report2026-09-06
Amazon Nova PremierAmazon42.4%
Reported settings & source

Nova2 internal agentic scaffold; scaled inference explicitly separate. Comparator setups from cited provider report.

First-party reported result; comparator results retain the source evaluation setup.

Amazon Nova 2 technical report · Table4 · reviewed 2026-09-06

Amazon Nova 2 technical reportlab self-report2026-09-06
Claude Haiku 4.5Anthropic73.3%
Reported settings & source

Nova2 internal agentic scaffold; scaled inference explicitly separate. Comparator setups from cited provider report.

First-party reported result; comparator results retain the source evaluation setup.

Amazon Nova 2 technical report · Table4 · reviewed 2026-09-06

Amazon Nova 2 technical reportlab self-report2026-09-06
Claude Opus 4.7Anthropic87.6%
Reported settings & source

MiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted.

First-party reported result; comparator results retain the source evaluation setup.

MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06

MiniMax M3 model card benchmark figurelab self-report2026-09-06
Claude Sonnet 4.5Anthropic77.2%
Reported settings & source

Nova2 internal agentic scaffold; scaled inference explicitly separate. Comparator setups from cited provider report.

First-party reported result; comparator results retain the source evaluation setup.

Amazon Nova 2 technical report · Table4 · reviewed 2026-09-06

Amazon Nova 2 technical reportlab self-report2026-09-06
Claude Sonnet 4.5Anthropic77.2%
Reported settings & source

Maximum reasoning; tau3 4 trials, GPT5.2 low simulator except Sierra reported comparisons; BrowseComp discard-all context at100k.

First-party reported result; comparator results retain the source evaluation setup.

Mistral Medium 3.5 model card performance charts · images/image4.png · reviewed 2026-09-06

Mistral Medium 3.5 model card performance chartslab self-report2026-09-06
Claude Sonnet 4.6Anthropic79.6%
Reported settings & source

MiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted.

First-party reported result; comparator results retain the source evaluation setup.

MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06

MiniMax M3 model card benchmark figurelab self-report2026-09-06
Claude Sonnet 4.6Anthropic79.6%
Reported settings & source

Maximum reasoning; tau3 4 trials, GPT5.2 low simulator except Sierra reported comparisons; BrowseComp discard-all context at100k.

First-party reported result; comparator results retain the source evaluation setup.

Mistral Medium 3.5 model card performance charts · images/image4.png · reviewed 2026-09-06

Mistral Medium 3.5 model card performance chartslab self-report2026-09-06
Gemini 2.5 FlashGoogle48.9%
Reported settings & source

Nova2 internal agentic scaffold; scaled inference explicitly separate. Comparator setups from cited provider report.

First-party reported result; comparator results retain the source evaluation setup.

Amazon Nova 2 technical report · Table4 · reviewed 2026-09-06

Amazon Nova 2 technical reportlab self-report2026-09-06
Gemini 2.5 ProGoogle59.6%
Reported settings & source

Nova2 internal agentic scaffold; scaled inference explicitly separate. Comparator setups from cited provider report.

First-party reported result; comparator results retain the source evaluation setup.

Amazon Nova 2 technical report · Table4 · reviewed 2026-09-06

Amazon Nova 2 technical reportlab self-report2026-09-06
Gemini 3.1 ProGoogle80.6%
Reported settings & source

MiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted.

First-party reported result; comparator results retain the source evaluation setup.

MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06

MiniMax M3 model card benchmark figurelab self-report2026-09-06
GLM-5.1Z.ai80.2%
Reported settings & source

Maximum reasoning; tau3 4 trials, GPT5.2 low simulator except Sierra reported comparisons; BrowseComp discard-all context at100k.

First-party reported result; comparator results retain the source evaluation setup.

Mistral Medium 3.5 model card performance charts · images/image4.png · reviewed 2026-09-06

Mistral Medium 3.5 model card performance chartslab self-report2026-09-06
GPT-5OpenAI72.8%
Reported settings & source

Nova2 internal agentic scaffold; scaled inference explicitly separate. Comparator setups from cited provider report.

First-party reported result; comparator results retain the source evaluation setup.

Amazon Nova 2 technical report · Table4 · reviewed 2026-09-06

Amazon Nova 2 technical reportlab self-report2026-09-06
GPT-5 MiniOpenAI71%
Reported settings & source

Nova2 internal agentic scaffold; scaled inference explicitly separate. Comparator setups from cited provider report.

First-party reported result; comparator results retain the source evaluation setup.

Amazon Nova 2 technical report · Table4 · reviewed 2026-09-06

Amazon Nova 2 technical reportlab self-report2026-09-06
GPT-5.1OpenAI76.3%
Reported settings & source

Nova2 internal agentic scaffold; scaled inference explicitly separate. Comparator setups from cited provider report.

First-party reported result; comparator results retain the source evaluation setup.

Amazon Nova 2 technical report · Table4 · reviewed 2026-09-06

Amazon Nova 2 technical reportlab self-report2026-09-06
GPT-5.5OpenAI82.9%
Reported settings & source

MiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted.

First-party reported result; comparator results retain the source evaluation setup.

MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06

MiniMax M3 model card benchmark figurelab self-report2026-09-06
Kimi-K2.6Moonshot80.2%
Reported settings & source

MiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted.

First-party reported result; comparator results retain the source evaluation setup.

MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06

MiniMax M3 model card benchmark figurelab self-report2026-09-06
MiniMax-M2.7MiniMax79.9%
Reported settings & source

MiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted.

First-party reported result; comparator results retain the source evaluation setup.

MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06

MiniMax M3 model card benchmark figurelab self-report2026-09-06
MiniMax-M3MiniMax80.5%
Reported settings & source

MiniMax launch figure methodology. SWE internal ClaudeCode 4 runs; Terminal 8CPU16GB2h128k Terminus2; Paper/PostTrain Ralph-loop12h; GDPval internal rubric grader; BrowseComp discard64k; OSWorld361tasks. Comparator provider/leaderboard scores where footnoted.

First-party reported result; comparator results retain the source evaluation setup.

MiniMax M3 model card benchmark figure · figures/benchmark.jpeg · reviewed 2026-09-06

MiniMax M3 model card benchmark figurelab self-report2026-09-06
Mistral Medium 3.5Mistral77.6%
Reported settings & source

Maximum reasoning; tau3 4 trials, GPT5.2 low simulator except Sierra reported comparisons; BrowseComp discard-all context at100k.

First-party reported result; comparator results retain the source evaluation setup.

Mistral Medium 3.5 model card performance charts · images/image4.png · reviewed 2026-09-06

Mistral Medium 3.5 model card performance chartslab self-report2026-09-06
Qwen3.5-397B-A17BQwen76.4%
Reported settings & source

Maximum reasoning; tau3 4 trials, GPT5.2 low simulator except Sierra reported comparisons; BrowseComp discard-all context at100k.

First-party reported result; comparator results retain the source evaluation setup.

Mistral Medium 3.5 model card performance charts · images/image4.png · reviewed 2026-09-06

Mistral Medium 3.5 model card performance chartslab self-report2026-09-06