A Qwen3.5-9B model fine-tuned with a custom 5-token control system for adaptive compute allocation in agentic workflows. The model routes every task to one of five compute levels at inference time, emitting a control token at the start of each response before executing structured tool calls.
This is a research artifact, not a production model. The fine-tune successfully taught control token routing and tool call formatting, but it degraded the base model's coding ability by 47 percentage points. The lessons learned from this project are more valuable than the model itself. Read on for the full honest breakdown.
Nerd TL;DR
For engineers who want the raw facts, warts and all.
What this is: A LoRA SFT + DPO fine-tune of Qwen3.5-9B that teaches the model to emit one of five control tokens ([FAST], [THINK], [VERIFY], [RECOVER], [ESCALATE]) and make structured XML+JSON tool calls. Trained on 4,992 SFT examples + 5,000 DPO pairs synthesized from Qwen v3.1 teacher responses, corrected by a pure-Python 5-reviewer orchestrator.
What this is NOT: A coding model. The fine-tune degraded raw coding ability. HumanEval+ pass@1 dropped from 64.0% (base) to 17.1% (fine-tuned, raw completion) to 13.4% (fine-tuned, chat template). The 95.1% / 81.1% HumanEval scores in the benchmark section are the base model with execution-based multi-sample selection (50 samples, T=0.2+0.8), not this fine-tuned model. Those benchmarks establish the ceiling of the base model's coding ability and validate the evaluation methodology.
Should you use this? Probably not for coding. Use the base Qwen3.5-9B for code generation. This model is useful if you need a standalone model that emits structured tool calls without a Python routing layer. If you already have a routing layer (like MAOS, LangChain, etc.), the base model with a good system prompt is better.
Training at a glance:
Stage
Steps
Time
Final Loss
Key Metric
SFT (LoRA r=64, bf16)
1,887
105 min
0.060
99.2% token accuracy
DPO (beta=0.1, 1 epoch)
630
77 min
3.9e-05
100% reward accuracy, margin ~43
The DPO loss went to near-zero. The chosen/rejected pairs were trivially separable -- the teacher narrated tools (950 chars avg) while the synthesized responses emitted structured calls (185 chars avg). The DPO signal is dominated by format differences, not reasoning quality. The model learned "don't narrate, emit structured calls" but did not learn deeper preference signals.
The control token system works but has routing edge cases. The model sometimes routes debugging tasks to [RECOVER] instead of [THINK] when the prompt contains failure-related keywords ("failing", "broken", "error"). Keyword sensitivity from the training data distribution.
Coding SFT LoRA was attempted and abandoned. A separate coding LoRA trained on MBPP + synthetic data (1,256 examples) hurt performance (57.3% vs 64% base). Overfitting on simple patterns. The base model was already strong; fine-tuning on narrow data degraded it.
Infrastructure: Trained on a rented RTX PRO 5000 Blackwell (48GB VRAM) via vast.ai. Total cost ~$3.50 ($2-3 Together AI inference + $0.30 GPU rental). bf16, no quantization, SDPA attention (flash-attn wouldn't build), no packing (cross-contamination risk without flash attention).
Deployment formats: bf16 GGUF (17GB), Q8_0 (8.9GB), Q6_K (6.9GB), Q5_K_M (6.0GB), Q5_K_S (5.9GB), Q4_K_M (5.2GB), Q4_K_S (5.0GB), Q4_0 (4.9GB), Q3_K_M (4.3GB), MLX 4-bit (4.7GB). fp8 is NOT supported by llama.cpp.
Thinking mode gotcha: Qwen3.5 has built-in thinking mode. You MUST pass enable_thinking=False in the chat template, or the model generates a thinking block before the control token. This is why the chat-template benchmark (13.4%) was worse than raw completion (17.1%) -- thinking tokens contaminated the code output.
The Honest Assessment
This project succeeded at what it set out to do (teach control tokens + tool call format) and failed at what we hoped it would do (maintain coding ability while adding agentic behavior). Both outcomes are instructive.
What we got right
The pipeline works. Data generation (5K prompts -> teacher inference -> 5-reviewer orchestrator -> synthesis -> quality filter) produced clean training data in 3 seconds for 5,000 responses, at zero API cost.
The training converged. SFT hit 99.2% token accuracy in 105 minutes. DPO hit 100% reward accuracy in 77 minutes. The format/routing behavior is easy to learn.
The cost was $3.50. Together AI inference ($2-3) + vast.ai GPU rental ($0.30). No expensive API bills, no long training runs.
The benchmarking methodology is sound. Execution-based multi-sample selection (50 samples, T=0.2+0.8) is the standard technique used by DeepSeek-Coder and CodeLlama. We validated it on the base model: 64% greedy -> 81.1% with execution selection.
The deployment pipeline works. fp16 safetensors, MLX 4-bit, and 9 GGUF quantizations all produced and uploaded.
What we got wrong
Fine-tuning degraded coding ability by 47 percentage points. HumanEval+ dropped from 64.0% (base) to 17.1% (fine-tuned). The LoRA adapters optimized for format compliance at the expense of semantic code generation. This is the fundamental trade-off of fine-tuning on format data: you get format, you lose substance.
The DPO signal was format-only. Cross-model DPO pairs (teacher narration vs synthesized structured calls) are trivially separable. The loss went to 3.9e-05. The model learned "don't narrate" but not "reason better." For deeper preference learning, self-preference pairs with multiple temperature samples would have been better.
The coding SFT LoRA overfit. Training on 1,256 MBPP + synthetic examples hurt the base model (57.3% vs 64%). The base model's coding ability was already strong. Fine-tuning on narrow, simple data degraded generalization. Lesson: don't fine-tune a strong model on weak data.
The control token routing has keyword sensitivity. The model routes debugging tasks to [RECOVER] instead of [THINK] when the prompt contains "failing", "broken", "error". The training data distribution biased the routing.
Chat template evaluation was misleading. The thinking mode contamination (13.4% vs 17.1%) took time to diagnose. Should have tested raw completion first.
Flash attention never built. Python 3.12 + CUDA 13.0 had no prebuilt wheel. Used SDPA with packing=False, accepting a 20-30% throughput hit.
Lessons that matter more than the model
These lessons cost $3.50 and 2 days to learn. They're worth more than the model:
Fine-tuning is a trade-off, not an upgrade. If your base model is already good at something, fine-tuning on different data will degrade that ability. The LoRA adapters don't add capability -- they redirect capacity. Know what you're trading away.
DPO is only as good as your pairs. If your chosen/rejected pairs differ in format (not reasoning), the model learns format (not reasoning). Near-zero DPO loss is a red flag, not a success. It means the task was trivial.
Execution-based benchmarking is the correct methodology. Greedy pass@1 underestimates model ability. Multi-sample with execution selection (50 samples, T=0.2+0.8) gives the true ceiling. 28/164 problems had no passing sample at T=0.2; 15 of those were solved at T=0.8. High-temperature diversity is critical.
Test the base model first. We spent time trying to "recover" coding ability with a coding SFT LoRA, not realizing the base model was already strong (64% HumanEval+). If we'd benchmarked the base model first, we'd have known the ceiling and not wasted time on the coding LoRA.
Template-based synthesis is deterministic but shallow. The 5-reviewer orchestrator + template synthesis produced correctly formatted responses at 1,623 responses/sec for $0. But the improved responses were structurally correct and semantically simple. The base model provides the reasoning; the templates provide the format.
$3.50 is enough for a 9B LoRA fine-tune. Together AI inference ($2-3 for 5K responses) + vast.ai RTX PRO 5000 Blackwell ($0.30 for 3 hours). You don't need expensive infrastructure for experimentation.
Model Details
Field
Value
Base model
Qwen/Qwen3.5-9B
Parameters
9B
Training method
LoRA SFT + DPO
LoRA rank
64 (alpha 128, dropout 0.05)
Training precision
bf16 (no quantization during training)
Training data
4,992 SFT examples + 5,000 DPO pairs
Training hardware
RTX PRO 5000 Blackwell (48GB VRAM)
Context length
2048 tokens
Attention implementation
SDPA (PyTorch native)
Packing
Disabled (no flash attention available)
Total training time
~3 hours (SFT 105 min + DPO 77 min)
Total cost
~$3.50
Control Token System
The model emits one of five control tokens at the start of every response:
Token
Usage
Distribution in training data
[FAST]
Direct action, simple tasks
47.4% (2,366)
[THINK]
Multi-step reasoning before acting
41.6% (2,078)
[VERIFY]
Act then confirm result (destructive/irreversible)
2.9% (145)
[RECOVER]
Reassess after failure, try different strategy
7.1% (354)
[ESCALATE]
Surface for human decision (high-risk/security)
1.0% (49)
Tool Call Format
<tool_call>
{"name": "shell", "arguments": {"command": "uv run pytest"}}
</tool_call>```
Supported tools: `shell`, `file_read`, `file_write`, `file_edit`, `file_list`, `grep`, `find_file`, `git`, `web_search`, `web_fetch`, `todo_write`, `ask_user`.
## Usage
### MLX (Apple Silicon)
```python
import mlx_lm
model, tokenizer = mlx_lm.load("davidnichols-ops/adaptive-operator-v4-mlx-4bit")
SYSTEM = """You are an adaptive engineering operator. You inspect systems, reason through problems, take actions, verify results, recover from failures, and decide when deeper thinking is necessary. You work from evidence, not assumptions.
## Compute Modes (control tokens)
Begin every response with exactly one control token to signal how much reasoning the task needs:
[FAST] -- direct, low-latency action. Use for simple tool calls, file reads, status checks, and routine edits.
[THINK] -- extended reasoning before acting. Use for design, debugging, refactoring, multi-step planning, and ambiguous problems.
[VERIFY] -- act, then verify the result before declaring success. Use when a change must be confirmed (tests pass, file written, deploy succeeded).
[RECOVER] -- a previous attempt failed; reassess and try a different strategy. Use after errors, broken builds, or unexpected output.
[ESCALATE] -- the task exceeds safe autonomous scope; stop and surface the situation for a human. Use for irreversible or high-risk actions.
## Tool Use
When a task requires a tool, emit a structured tool call in this exact format:
<tool_call>
{"name": "tool_name", "arguments": {"param": "value"}}
</tool_call>
After each tool call you will receive the result. Use it to decide your next action. When the task is complete, give a concise final answer with no tool calls."""
messages = [
{"role": "system", "content": SYSTEM},
{"role": "user", "content": "Show me the last 10 git commits."},
]
chat = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
response = mlx_lm.generate(model, tokenizer, prompt=chat, max_tokens=256)
print(response)
HuggingFace Transformers (fp16)
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
23model = AutoModelForCausalLM.from_pretrained("davidnichols-ops/adaptive-operator-v4", torch_dtype="auto")4tokenizer = AutoTokenizer.from_pretrained("davidnichols-ops/adaptive-operator-v4")56# CRITICAL: disable thinking mode or control tokens won't appear at response start7chat = tokenizer.apply_chat_template(8 messages, tokenize=False, add_generation_prompt=True, enable_thinking=False9)
Ollama (GGUF)
bash
1ollama create adaptive-operator-v4 -f Modelfile
2ollama run adaptive-operator-v4 "Show me the last 10 git commits."
GGUF Quantizations
All quantizations are in the GGUF repo, generated from the bf16 source with llama.cpp:
Format
Size
Bits/weight
Best for
bf16
17 GB
16.0
Maximum fidelity, CPU inference
Q8_0
8.9 GB
8.5
Near-lossless, good speed/quality balance
Q6_K
6.9 GB
6.6
High quality, moderate size
Q5_K_M
6.0 GB
5.8
Good quality/size trade-off
Q5_K_S
5.9 GB
5.6
Slightly smaller than Q5_K_M
Q4_K_M
5.2 GB
5.0
Recommended default for 4-bit
Q4_K_S
5.0 GB
4.8
Smaller 4-bit variant
Q4_0
4.9 GB
4.7
Fastest 4-bit, legacy format
Q3_K_M
4.3 GB
4.1
Smallest viable, quality degradation
fp8 is not supported by llama.cpp. The highest-fidelity 16-bit option is bf16.
HumanEval / HumanEval+ (EvalPlus) -- Base Qwen3.5-9B
These benchmarks were run on the base model to establish the coding ceiling and validate the evaluation methodology. The fine-tuned model's coding ability is lower (see below).
Method
HumanEval pass@1
HumanEval+ pass@1
Greedy (temp=0)
70.7%
64.0%
20 samples (T=0.2), random pick
69.9%
62.1%
20 samples, execution-selected (base tests)
82.9%
70.7%
20 samples, execution-selected (base+plus tests)
82.9%
72.6%
50 samples (T=0.2+0.8), execution-selected
95.1%
81.1%
pass@10 (20 samples, T=0.2)
81.5%
74.6%
Fine-tuned Model -- Coding Ability Impact
Model
HumanEval+ pass@1 (raw completion)
Base Qwen3.5-9B
64.0%
Fine-tuned (chat template)
13.4%
Fine-tuned (raw completion)
17.1%
Coding SFT LoRA merged
57.3% (abandoned -- overfit)
Fine-tuned Model -- Agentic Metrics
Metric
Value
SFT final loss
0.060
DPO reward accuracy
100%
Control token routing accuracy
86% FAST mode on simple tasks
Tool call format compliance
>99%
Dataset Statistics
SFT Dataset (4,992 examples)
Category
Count
Percentage
tool_use_file_ops
889
17.8%
coding_write
828
16.6%
coding_debug
615
12.3%
tool_use_shell
543
10.9%
tool_use_git
435
8.7%
coding_test
404
8.1%
coding_refactor
376
7.5%
planning
374
7.5%
tool_use_search
298
6.0%
recovery
238
4.8%
Difficulty Distribution
Level
Count
Percentage
1 (simple)
2,374
47.5%
2 (moderate)
2,035
40.8%
3 (complex)
591
11.8%
DPO Dataset (5,000 pairs)
Cross-model preference pairs: original Qwen v3.1 teacher response (rejected) vs synthesized improved response (chosen). The rejected responses are 5x longer (950 chars vs 185 chars) -- the teacher was verbose with narration. The improved responses are concise and structured.
Qwen3.5 has a built-in thinking mode that prepends internal reasoning before the response. To get control tokens at the start of the output, you must pass enable_thinking=False when applying the chat template:
Without this, the model generates a thinking block first, and the control token appears later in the response.
Limitations
Coding ability degradation: The fine-tune degraded raw coding ability (64% -> 17% HumanEval+). Use the base Qwen3.5-9B for coding tasks.
Context length: 2048 tokens (may truncate long conversations)
Tool call accuracy: The model emits correctly formatted tool calls but may select suboptimal tools for ambiguous tasks
Reasoning depth: SFT responses are structurally correct but semantically simple -- the base model provides semantic content, but the synthesis templates produce minimal stubs
Repetition: 4-bit quantization with greedy decoding can cause repetition loops; use temperature sampling (0.7-0.8)
System prompt required: The control token system only activates when the system prompt is provided
Thinking mode: Must disable enable_thinking in the chat template
THINK routing: The model sometimes routes debugging tasks to [RECOVER] instead of [THINK] when the prompt contains failure-related keywords
DPO overfitting: Near-zero DPO loss indicates format-dominated signal, not reasoning quality
Training data bias: 47.5% of training data is difficulty level 1 (simple). The model may underperform on complex, multi-step tasks
