SWE-bench Multimodal

SWE-bench Multimodal is run by SWE-bench. It has ranked 22 models, of which the catalogue holds 11, scoring from 25.34 to 35.98 on % Resolved.

measured by SWE-bench · the board itself

Place	Model	Metric	Score
1stof 22	o3	% Resolved	35.98
3rdof 22	Claude Sonnet 4	% Resolved	35.59
3rdof 22	Refact.ai Agent	% Resolved	35.59
4thof 22	Claude Sonnet 4	% Resolved (claude-sonnet 4)	34.43
5thof 22	o4-mini	% Resolved	33.85
6thof 22	Claude 3.7 Sonnet	% Resolved	31.33
7thof 22	GPT-4.1	% Resolved	31.14
8thof 22	Zencoder	% Resolved	30.56
9thof 22	GPT-4o	% Resolved	30.37
10thof 22	Code Fixer Agent	% Resolved	29.59
12thof 22	Claude 3.5 Sonnet	% Resolved	25.34

Place

Model

Metric

Score

1stof 22

% Resolved

35.98

3rdof 22

% Resolved

35.59

3rdof 22