Python Simple AST

BFCL · 50 models

Rank	Model	Score
1	Claude-Sonnet-4-5-20250929 (FC)	97.75
2	palmyra-x-004 (FC)	97
3	Claude-Opus-4-5-20251101 (FC)	96.5
3	Qwen3-235B-A22B-Instruct-2507 (Prompt)	96.5
5	Llama-3.3-70B-Instruct (FC)	96.25
5	Qwen3-32B (Prompt)	96.25
7	Amazon-Nova-2-Lite-v1:0 (FC)	96
7	Arch-Agent-32B	96
9	Qwen3-8B (Prompt)	95.75
10	Falcon3-10B-Instruct (FC)	95.5
10	Qwen3-4B-Instruct-2507 (FC)	95.5
10	Qwen3-8B (FC)	95.5
13	Command A (FC)	95.25
13	Llama-4-Maverick-17B-128E-Instruct-FP8 (FC)	95.25
13	Mistral-Small-2506 (Prompt)	95.25
13	Qwen3-14B (FC)	95.25
13	mistral-large-2411 (FC)	95.25
13	xLAM-2-32b-fc-r (FC)	95.25
19	Claude-Haiku-4-5-20251001 (FC)	95
19	Qwen3-30B-A3B-Instruct-2507 (Prompt)	95
21	GLM-4.6 (FC thinking)	94.75
21	GPT-4.1-2025-04-14 (Prompt)	94.75
21	Gemini-3-Pro-Preview (Prompt)	94.75
21	Qwen3-30B-A3B-Instruct-2507 (FC)	94.75
21	Qwen3-32B (FC)	94.75
21	xLAM-2-70b-fc-r (FC)	94.75
27	Gemini-3-Pro-Preview (FC)	94.5
27	Nanbeige4-3B-Thinking-2511 (FC)	94.5
27	Qwen3-14B (Prompt)	94.5
30	BitAgent-Bounty-8B	94.25
31	Gemma-3-27b-it (Prompt)	94
31	Grok-4-1-fast-non-reasoning (FC)	94
31	Hammer2.1-3b (FC)	94
31	Llama-3.1-8B-Instruct (Prompt)	94
31	Llama-4-Scout-17B-16E-Instruct (FC)	94
31	Open-Mistral-Nemo-2407 (Prompt)	94
37	Claude-Opus-4-5-20251101 (Prompt)	93.75
37	Command A Reasoning (FC)	93.75
37	Gemma-3-12b-it (Prompt)	93.75
37	Grok-4-1-fast-reasoning (FC)	93.75
37	Phi-4 (Prompt)	93.75
37	Qwen3-4B-Instruct-2507 (Prompt)	93.75
43	Amazon-Nova-Pro-v1:0 (FC)	93.5
43	Hammer2.1-7b (FC)	93.5
45	Falcon3-7B-Instruct (FC)	93.25
45	Open-Mistral-Nemo-2407 (FC)	93.25
47	Arch-Agent-3B	93
48	GPT-5.2-2025-12-11 (FC)	92.75
48	Granite-3.2-8B-Instruct (FC)	92.75
48	Mistral-small-2506 (FC)	92.75