Python Multiple AST

BFCL · 50 models

Rank	Model	Score
1	BitAgent-Bounty-8B	94.02
2	Gemini-3-Pro-Preview (Prompt)	81.77
3	Qwen3-32B (Prompt)	81.2
4	Qwen3-32B (FC)	80.91
5	mistral-large-2411 (FC)	80.72
6	Gemini-3-Pro-Preview (FC)	80.44
7	Amazon-Nova-2-Lite-v1:0 (FC)	80.15
8	Qwen3-8B (FC)	79.68
9	Mistral-small-2506 (FC)	79.39
9	ToolACE-2-8B (FC)	79.39
11	Arch-Agent-32B	79.11
12	Qwen3-14B (FC)	79.01
13	Claude-Sonnet-4-5-20250929 (FC)	78.92
13	GLM-4.6 (FC thinking)	78.92
13	Qwen3-8B (Prompt)	78.92
16	Mistral-Small-2506 (Prompt)	78.54
17	Command A Reasoning (FC)	78.35
18	Claude-Opus-4-5-20251101 (FC)	78.16
19	Moonshotai-Kimi-K2-Instruct (FC)	78.06
19	Nanbeige4-3B-Thinking-2511 (FC)	78.06
19	Qwen3-14B (Prompt)	78.06
22	Amazon-Nova-Pro-v1:0 (FC)	77.97
22	palmyra-x-004 (FC)	77.97
24	Qwen3-235B-A22B-Instruct-2507 (Prompt)	77.78
25	Claude-Haiku-4-5-20251001 (FC)	77.59
26	Qwen3-30B-A3B-Instruct-2507 (Prompt)	77.49
27	GPT-4.1-2025-04-14 (Prompt)	77.4
28	Grok-4-1-fast-reasoning (FC)	77.3
29	Command A (FC)	76.92
29	Grok-4-1-fast-non-reasoning (FC)	76.92
31	Qwen3-30B-A3B-Instruct-2507 (FC)	76.83
32	Falcon3-10B-Instruct (FC)	76.16
32	Qwen3-4B-Instruct-2507 (FC)	76.16
34	Gemini-2.5-Flash (Prompt)	75.97
35	Llama-3.3-70B-Instruct (FC)	75.5
36	DeepSeek-V3.2-Exp (Prompt + Thinking)	74.74
37	xLAM-2-32b-fc-r (FC)	74.64
38	Qwen3-1.7B (FC)	74.26
39	Claude-Opus-4-5-20251101 (Prompt)	74.17
39	Qwen3-4B-Instruct-2507 (Prompt)	74.17
41	Grok-4-0709 (FC)	73.88
42	GPT-4.1-mini-2025-04-14 (Prompt)	73.31
43	Open-Mistral-Nemo-2407 (Prompt)	73.03
44	Open-Mistral-Nemo-2407 (FC)	72.84
45	Llama-4-Scout-17B-16E-Instruct (FC)	72.74
46	Gemma-3-27b-it (Prompt)	72.46
47	Arch-Agent-3B	72.27
48	Gemma-3-12b-it (Prompt)	71.89
49	Gemini-2.5-Flash (FC)	71.7
50	Qwen3-235B-A22B-Instruct-2507 (FC)	71.6