Full
SWE-bench · 24 models
| Rank | Model | Score |
|---|---|---|
| 1 | Sonar Foundation Agent + Claude 4.5 Opus | 52.62 |
| 2 | Salesforce AI Research SAGE (bash-only) | 44.25 |
| 3 | Atlassian Rovo Dev (2025-06-05) | 41.98 |
| 4 | Amazon Q Developer Agent (v20250405-dev) | 37.1 |
| 5 | SWE-agent 1.0 (Claude 3.7 Sonnet) | 33.83 |
| 6 | Amazon Q Developer Agent (v20241202-dev) | 29.99 |
| 7 | OpenHands + CodeAct v2.1 (claude-3-5-sonnet-20241022) | 29.38 |
| 8 | AutoCodeRover-v2.0 (Claude-3.5-Sonnet-20241022) | 24.89 |
| 9 | Honeycomb | 22.06 |
| 10 | Amazon Q Developer Agent (v20240719-dev) | 19.75 |
| 11 | Factory Code Droid | 19.27 |
| 12 | AutoCodeRover (v20240620) + GPT 4o (2024-05-13) | 18.83 |
| 13 | SWE-agent + Claude 3.5 Sonnet | 18.13 |
| 14 | AppMap Navie + GPT 4o (2024-05-13) | 14.6 |
| 15 | Amazon Q Developer Agent (v20240430-dev) | 13.82 |
| 16 | SWE-agent + GPT 4 (1106) | 12.47 |
| 17 | SWE-agent + GPT 4o (2024-05-13) | 11.99 |
| 18 | SWE-agent + Claude 3 Opus | 10.51 |
| 19 | RAG + Claude 3 Opus | 3.79 |
| 20 | RAG + Claude 2 | 1.96 |
| 21 | RAG + GPT 4 (1106) | 1.31 |
| 22 | RAG + SWE-Llama 13B | 0.7 |
| 22 | RAG + SWE-Llama 7B | 0.7 |
| 24 | RAG + ChatGPT 3.5 | 0.17 |