Lite

SWE-bench · 50 models

Rank	Model	Score
1	ExpeRepair-v1.0 + Claude 4 Sonnet	60.33
2	Refact.ai Agent	60
3	KGCompass + Claude 4 Sonnet (20250514)	58.33
4	SWE-agent + Claude 4 Sonnet	56.67
5	Isoform	55
6	SemAgent_Multi-v1.0	51.67
7	Isea	51.33
8	EntroPO + R2E + Qwen3-Coder-30B-A3B-Instruct	49.67
9	Blackbox AI Agent	49
9	Codev	49
11	Gru(2024-12-08)	48.67
12	ExpeRepair-v1.0	48.33
12	Globant Code Fixer Agent	48.33
14	SWE-agent + Claude 3.7 Sonnet	48
15	devlo	47.33
16	DARS Agent	47
17	KGCompass + Claude 3.5 Sonnet (20241022)	46
18	EntroPO + R2E + Qwen3-Coder-30B-A3B-Instruct	45
19	Kodu-v1 + Claude-3.5 Sonnet (20241022)	44.67
20	CodeFuse-CGM	44
21	CodeStory Aide + Mixed Models	43
22	Lingxi	42.67
23	Codart AI	41.67
23	OpenHands + CodeAct v2.1 (claude-3-5-sonnet-20241022)	41.67
25	PatchKitty-0.9 + Claude-3.5 Sonnet (20241022)	41.33
26	Composio SWE-Kit (2024-10-30)	41
26	OrcaLoca + Agentless-1.5 + Claude-3.5 Sonnet (20241022)	41
28	Agentless-1.5 + Claude-3.5 Sonnet (20241022)	40.67
29	OpenCSG Starship Agentic Coder + GPT 4 (0806)	39.67
30	Bytedance MarsCode Agent	39.33
31	Moatless Tools + Claude 3.5 Sonnet (20241022)	39
32	Honeycomb	38.33
32	Moatless Tools + Claude 3.5 Sonnet (20241022)	38.33
34	AbanteAI MentatBot + GPT 4o (2024-05-13)	38
35	Patched.Codes Patchwork	37
36	KGCompass + DeepSeek V3	36.67
37	AppMap Navie v2	36
38	CodeFuse-AAIS	35.67
38	Gru(2024-08-11)	35.67
40	Isoform	35
41	Bytedance MarsCode Agent + GPT 4o (2024-05-13)	34
41	SuperCoder2.0	34
43	Alibaba Lingma Agent	33
44	Agentless Lite + O3 Mini (20250214)	32.33
45	Agentless-1.5 + GPT 4o (2024-05-13)	32
46	CodeShellTester + GPT 4o (2024-05-13)	31.33
46	Factory Code Droid	31.33
48	AutoCodeRover (v20240620) + GPT 4o (2024-05-13)	30.67
48	Moatless Tools + Deepseek V3	30.67
50	Aegis - o3-mini_1.0	30.33