Memory KV
BFCL · 50 models
| Rank | Model | Score |
|---|---|---|
| 1 | Claude-Opus-4-5-20251101 (FC) | 70.97 |
| 2 | Gemini-3-Pro-Preview (Prompt) | 59.35 |
| 3 | Grok-4-0709 (FC) | 57.42 |
| 4 | Claude-Sonnet-4-5-20250929 (FC) | 54.19 |
| 5 | Claude-Haiku-4-5-20251001 (FC) | 51.61 |
| 6 | Gemini-3-Pro-Preview (FC) | 50.32 |
| 7 | DeepSeek-V3.2-Exp (Prompt + Thinking) | 46.45 |
| 8 | GLM-4.6 (FC thinking) | 43.87 |
| 8 | Grok-4-0709 (Prompt) | 43.87 |
| 10 | DeepSeek-V3.2-Exp (FC) | 41.94 |
| 11 | Grok-4-1-fast-reasoning (FC) | 41.29 |
| 12 | Nanbeige3.5-Pro-Thinking (FC) | 38.06 |
| 13 | GPT-5-mini-2025-08-07 (FC) | 36.77 |
| 14 | GPT-5.2-2025-12-11 (FC) | 33.55 |
| 14 | o3-2025-04-16 (Prompt) | 33.55 |
| 16 | Nanbeige4-3B-Thinking-2511 (FC) | 31.61 |
| 17 | o3-2025-04-16 (FC) | 24.52 |
| 18 | GPT-4.1-mini-2025-04-14 (FC) | 22.58 |
| 18 | o4-mini-2025-04-16 (Prompt) | 22.58 |
| 20 | Moonshotai-Kimi-K2-Instruct (FC) | 21.94 |
| 21 | GPT-4.1-mini-2025-04-14 (Prompt) | 20.65 |
| 21 | GPT-5-nano-2025-08-07 (Prompt) | 20.65 |
| 21 | Grok-4-1-fast-non-reasoning (FC) | 20.65 |
| 24 | GPT-5-mini-2025-08-07 (Prompt) | 19.35 |
| 24 | Gemini-2.5-Flash (FC) | 19.35 |
| 24 | o4-mini-2025-04-16 (FC) | 19.35 |
| 27 | mistral-large-2411 (FC) | 18.71 |
| 28 | GPT-5-nano-2025-08-07 (FC) | 18.06 |
| 29 | Phi-4 (Prompt) | 17.42 |
| 30 | mistral-large-2411 (Prompt) | 16.77 |
| 31 | Command A Reasoning (FC) | 16.13 |
| 31 | GPT-4.1-2025-04-14 (FC) | 16.13 |
| 31 | Mistral-Medium-2505 | 16.13 |
| 31 | Qwen3-4B-Instruct-2507 (FC) | 16.13 |
| 35 | Mistral-Medium-2505 (FC) | 15.48 |
| 36 | Gemini-2.5-Flash (Prompt) | 13.55 |
| 36 | Qwen3-32B (Prompt) | 13.55 |
| 38 | Qwen3-235B-A22B-Instruct-2507 (Prompt) | 12.9 |
| 38 | Qwen3-4B-Instruct-2507 (Prompt) | 12.9 |
| 40 | Falcon3-10B-Instruct (FC) | 12.26 |
| 40 | Qwen3-32B (FC) | 12.26 |
| 42 | Falcon3-7B-Instruct (FC) | 10.32 |
| 42 | GPT-4.1-nano-2025-04-14 (FC) | 10.32 |
| 44 | GPT-4.1-2025-04-14 (Prompt) | 9.68 |
| 44 | Gemma-3-4b-it (Prompt) | 9.68 |
| 44 | Granite-3.1-8B-Instruct (FC) | 9.68 |
| 44 | MiniCPM3-4B-FC (FC) | 9.68 |
| 44 | Open-Mistral-Nemo-2407 (Prompt) | 9.68 |
| 49 | CoALM-70B | 9.03 |
| 49 | GPT-4.1-nano-2025-04-14 (Prompt) | 9.03 |