Non-Live Simple AST

BFCL · 50 models

Rank	Model	Score
1	Qwen3-30B-A3B-Instruct-2507 (Prompt)	80.67
2	xLAM-2-32b-fc-r (FC)	80.42
3	Qwen3-235B-A22B-Instruct-2507 (Prompt)	79.83
4	Claude-Opus-4-5-20251101 (Prompt)	79.58
4	Gemini-3-Pro-Preview (Prompt)	79.58
6	Hammer2.1-3b (FC)	79.33
6	Open-Mistral-Nemo-2407 (Prompt)	79.33
8	Qwen3-32B (Prompt)	79.08
9	Llama-4-Scout-17B-16E-Instruct (FC)	79
10	Mistral-Small-2506 (Prompt)	78.75
11	Arch-Agent-3B	78.67
12	GPT-4.1-2025-04-14 (Prompt)	78.25
12	xLAM-2-70b-fc-r (FC)	78.25
14	Gemma-3-27b-it (Prompt)	77.67
15	Grok-4-1-fast-reasoning (FC)	77.58
16	Gemini-2.5-Flash (Prompt)	77.33
17	Qwen3-4B-Instruct-2507 (Prompt)	77.25
18	Llama-4-Maverick-17B-128E-Instruct-FP8 (FC)	77.08
19	Claude-Opus-4-5-20251101 (FC)	76.83
19	Qwen3-14B (Prompt)	76.83
21	Arch-Agent-32B	76.67
22	Amazon-Nova-2-Lite-v1:0 (FC)	76.33
22	Mistral-Medium-2505	76.33
24	Gemma-3-12b-it (Prompt)	76.25
25	Llama-3.3-70B-Instruct (FC)	76.08
26	Grok-4-1-fast-non-reasoning (FC)	76
27	Amazon-Nova-Pro-v1:0 (FC)	75.83
28	Command A (FC)	75.75
29	Qwen3-32B (FC)	75.58
30	Gemini-3-Pro-Preview (FC)	75.5
30	Qwen3-4B-Instruct-2507 (FC)	75.5
30	mistral-large-2411 (Prompt)	75.5
33	xLAM-2-3b-fc-r (FC)	75.33
34	Qwen3-8B (Prompt)	75.25
35	GPT-4.1-mini-2025-04-14 (Prompt)	74.92
36	Qwen3-14B (FC)	74.75
37	o3-2025-04-16 (FC)	74.5
38	Gemini-2.5-Flash (FC)	74.33
39	GLM-4.6 (FC thinking)	74.25
39	Phi-4 (Prompt)	74.25
39	o3-2025-04-16 (Prompt)	74.25
42	DeepSeek-V3.2-Exp (Prompt + Thinking)	74.08
43	Hammer2.1-1.5b (FC)	73.92
44	xLAM-2-8b-fc-r (FC)	73.83
45	Command A Reasoning (FC)	73.58
46	Grok-4-0709 (FC)	73.5
47	ToolACE-2-8B (FC)	73.42
48	GPT-4.1-mini-2025-04-14 (FC)	73.33
49	Bielik-11B-v2.3-Instruct (Prompt)	73
50	GPT-5.2-2025-12-11 (FC)	72.92