Memory Vector

BFCL · 50 models

Rank	Model	Score
1	Claude-Opus-4-5-20251101 (FC)	72.9
2	Gemini-3-Pro-Preview (FC)	63.23
3	Gemini-3-Pro-Preview (Prompt)	62.58
4	DeepSeek-V3.2-Exp (FC)	61.29
5	Grok-4-0709 (Prompt)	59.35
6	Grok-4-0709 (FC)	58.71
7	Nanbeige3.5-Pro-Thinking (FC)	58.06
8	Claude-Sonnet-4-5-20250929 (FC)	57.42
8	Grok-4-1-fast-reasoning (FC)	57.42
10	GLM-4.6 (FC thinking)	56.13
11	Claude-Haiku-4-5-20251001 (FC)	55.48
12	Gemini-2.5-Flash (FC)	50.32
12	o3-2025-04-16 (Prompt)	50.32
14	Gemini-2.5-Flash (Prompt)	47.1
15	DeepSeek-V3.2-Exp (Prompt + Thinking)	46.45
16	o3-2025-04-16 (FC)	44.52
17	GPT-5-mini-2025-08-07 (FC)	43.87
18	GPT-5.2-2025-12-11 (FC)	43.23
19	Nanbeige4-3B-Thinking-2511 (FC)	34.19
20	Llama-4-Maverick-17B-128E-Instruct-FP8 (FC)	32.9
21	GPT-5-nano-2025-08-07 (Prompt)	31.61
22	mistral-large-2411 (Prompt)	30.97
23	GPT-5-mini-2025-08-07 (Prompt)	29.68
24	mistral-large-2411 (FC)	29.03
25	GPT-5-nano-2025-08-07 (FC)	27.1
26	Qwen3-32B (FC)	25.81
27	Gemma-3-12b-it (Prompt)	25.16
27	Phi-4 (Prompt)	25.16
27	o4-mini-2025-04-16 (Prompt)	25.16
30	o4-mini-2025-04-16 (FC)	24.52
31	Command A Reasoning (FC)	23.87
32	Grok-4-1-fast-non-reasoning (FC)	20
32	Mistral-Medium-2505 (FC)	20
32	Moonshotai-Kimi-K2-Instruct (FC)	20
35	Falcon3-10B-Instruct (FC)	19.35
35	GPT-4.1-2025-04-14 (Prompt)	19.35
35	GPT-4.1-nano-2025-04-14 (FC)	19.35
38	Qwen3-235B-A22B-Instruct-2507 (FC)	18.71
39	GPT-4.1-2025-04-14 (FC)	18.06
40	Qwen3-14B (FC)	16.77
41	GPT-4.1-mini-2025-04-14 (FC)	16.13
41	ToolACE-2-8B (FC)	16.13
43	MiniCPM3-4B-FC (FC)	15.48
43	xLAM-2-8b-fc-r (FC)	15.48
45	Mistral-Medium-2505	14.84
46	GPT-4.1-nano-2025-04-14 (Prompt)	14.19
46	Mistral-small-2506 (FC)	14.19
46	Qwen3-32B (Prompt)	14.19
46	Qwen3-4B-Instruct-2507 (Prompt)	14.19
46	palmyra-x-004 (FC)	14.19