A QLoRA adapter on top of Qwen/Qwen3-4B, fine-tuned for tool-calling against a 20-tool utility catalog. The smaller-model variant of the ToolFlow project — same recipe and training data as the 8B toolflow-qwen3-8b-utility-v2, trained to answer one specific question: is a 4B model good enough on this narrow tool-calling task at half the parameter footprint?

Headline results

Evaluated on the same 270-case eval set as the 8B variant (BFCL v3 prompts re-labeled against the utility catalog by Qwen3-235B):

Metric	Recipe B v2 (this, 4B)	Recipe A v2 (8B)	Sonnet 4.6 few-shot v2
Semantic-valid	50.4%	60.4%	70.4%
Tool match	55.2%	64.4%	71.5%
Schema valid	87.4%	86.3%	98.9%
Mean args F1	0.514	0.614	0.704
Abstain precision	100%	100%	99.4%
Abstain recall	51.8%	61.9%	70.4%
p50 latency	1501 ms	727 ms	1372 ms

Statistical significance vs the 8B variant: Δ semantic-valid = -10.0 pp, 95% CI [-15.2, -5.2], McNemar p = 0.0003. The gap is real, not noise.

When to pick this 4B over the 8B variant

Use case	Pick this 4B	Pick the 8B
Cost is the bottleneck	❌ ~21× cheaper than Sonnet, but only ~$16/mo savings vs 8B at 100K tasks/mo	✅ Better quality-per-dollar at typical cloud serving
Edge/local deployment where 8B doesn't fit	✅ Fits ~30% smaller VRAM	—
Quality matters, latency is fine	❌ 10 pp lower semantic-valid	✅
Tight inference latency budget on small batch	Maybe — 4B is slower without batched serving (1501 vs 727 ms p50 measured), but should win under vLLM batched mode	Maybe

For most cloud-serving production use, prefer the 8B variant — the 10 pp quality gap doesn't justify the ~$16/mo savings at 100K tasks/mo. The 4B becomes the right pick when serving infrastructure is genuinely constrained (memory or hardware compatibility), not when per-task cost is the metric.

What this adapter does

Same as the 8B variant — given a user request and a system prompt enumerating the 20-tool catalog, emits a JSON tool call (or {"tool": "ABSTAIN", "arguments": {}} if no tool fits).

Training details

	Value
Method	QLoRA (4-bit NF4 base, 16-bit LoRA adapter)
LoRA r / alpha / dropout	16 / 32 / 0.05
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Optimizer	AdamW
Learning rate	2e-4 cosine, 3% warmup
Effective batch	16 (per-device 4 × grad_accum 4)
Epochs	3
Max seq length	2048
Compute	Modal H200
Wall time	~20 min
Final train loss	0.018
Final eval loss	0.019
Adapter size	66 MB (smaller than 8B's 87 MB — fewer transformer layers)

Training data

Identical to the 8B variant: 4,461 examples (3,749 positive tool-calls from xlam + Glaive, plus 712 synthesized ABSTAIN cases via Qwen3-235B). See the 8B model card or the project README for the full provenance and licensing.

Per-category breakdown (vs 8B)

Category	n	This (4B)	8B	Δ (4B − 8B)
`abstain`	50	64.0%	76.0%	-12.0 pp
`happy_path`	100	47.0%	56.0%	-9.0 pp
`tool_selection`	80	47.5%	56.3%	-8.8 pp
`multi_call`	40	65.0%	80.0%	-15.0 pp

The 4B is uniformly weaker (8-15 pp drop everywhere); the gap doesn't concentrate in one category. The biggest gap is multi_call, where sustained context + multiple correct args is harder for the smaller model.

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-4B",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("s4um1l/toolflow-qwen3-4b-utility-v2")
model = PeftModel.from_pretrained(base, "s4um1l/toolflow-qwen3-4b-utility-v2")
model.eval()

# (Build SYSTEM prompt with the 20-tool catalog from the repo, see 8B model card for an example)
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user",   "content": "What's the area of a circle with radius 4?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Known limitations

Same distribution-shift issue as the 8B variant — formal/academic-style abstain prompts (BFCL irrelevance category) are under-served by the xlam-shaped training data. Fixable in next iteration by adding ~500 BFCL-style abstain prompts.

The 4B's gap is uniform across categories rather than concentrated in one failure mode, so improvements need to lift overall capability, not just patch one weakness.

License

Apache 2.0 (inherited from Qwen3-4B base model). Adapter weights only — no training data redistributed.

Project + reproducibility

Project repo: https://github.com/s4um1l/toolflow
Recipe B detailed report: reports/recipe_b_report.md
W2 comparison report: reports/week2_comparison_report.md
Project tag: v0.2-week2

Citation

bibtex

@misc{toolflow_qwen3_4b_utility_v2,
  title  = {ToolFlow Qwen3-4B Utility v2 (LoRA adapter)},
  author = {Srivastava, Saumil},
  year   = {2026},
  url    = {https://huggingface.co/s4um1l/toolflow-qwen3-4b-utility-v2}
}