A coding-focused SFT of Qwen 3.6 27B, merged to full-precision BF16. Trained on a mixed
corpus of coding, tool-calling, and agentic (SWE-agent) data in a single supervised
fine-tuning run, with the coding domain deliberately kept dominant (~82.5% of training
tokens) to avoid the smaller tool/agentic buckets degrading coding capability.
Status: public repository. This is the merged, ready-to-use model. For a smaller
adapter-only download see Qwimi-3.6-27B-LoRA; for quantized GGUF files (llama.cpp / Ollama /
LM Studio) see Qwimi-3.6-27B-GGUF.
unsloth/Qwen3.6-27B (finetune of Qwen/Qwen3.6-27B)
Architecture
Dense, 27.8B params. Hybrid: standard attention layers interleaved with GatedDeltaNet linear-attention layers. model_type: qwen3_5, class Qwen3_5ForConditionalGeneration
Modality
Multimodal (image-text-to-text) checkpoint. This SFT is text-only — the vision tower was frozen and untouched (see §9)
Native context
262,144 tokens (base). This SFT trained and validates up to 16,384 tokens
Fine-tuning method
4-bit QLoRA (r=64), merged to BF16
Trainable parameters
466,911,232 / 27,823,639,792 (1.68%) — language layers only (attention + MLP), vision layers frozen
License
Apache 2.0 (inherited from base)
2. Intended use
Coding assistance (generation, debugging, refactoring, explanation) with visible
chain-of-thought reasoning, plus native function/tool calling and basic agentic
(SWE-agent style: multi-turn tool use over a repository) behavior.
Not validated for: image/vision input (untouched by this SFT — see §9), non-English use,
safety-critical or high-stakes deployment, or any task outside code/tool/agentic reasoning.
3. Training configuration
Setting
Value
Precision (training)
4-bit QLoRA (NF4)
Max sequence length
16,384 tokens
LoRA rank / alpha
64 / 64
LoRA dropout
0.0
rsLoRA
off
LoRA scope
language layers only (attention + MLP); vision tower frozen
Unsloth offloaded mode, verified active (measured activation delta 3.1→5.5→9.2 GB across 2k→8k→16.4k token samples)
Hardware
1× A100-SXM4-80GB (Google Colab Pro)
Software
Unsloth 2026.6.9, Transformers 5.5.0, Torch 2.10.0+cu128, CUDA 12.8, Triton 3.6.0, flash-linear-attention + causal-conv1d (required for the GatedDeltaNet fast path)
Checkpoint cadence
every 150 steps, rolling window of 4, persisted to Google Drive
4. Dataset composition
Single mixed SFT run — coding, tool-calling, and agentic examples trained together (not
sequential stages), so the model learns all three without later stages overwriting earlier
ones. Coding was deliberately kept dominant by token share, not just row count.
Coding (5 sources, deduplicated, content- or domain-filtered for code where the source
had no reliable label; one low-quality synthetic source capped hard):
content-filtered; hard cap 600 — synthetically reverse-engineered reasoning traces, lowest-quality source, deliberately minimized
Tool-calling:minpeter/xlam-function-calling-60k-parsed — 6,000 rows (APIGen-generated,
verifiable function calls, native tools schema). No chain-of-thought; direct tool calls.
Agentic (SWE-agent trajectories; the dominant length constraint in this run — most raw
trajectories exceed 16,384 tokens and were dropped whole rather than truncated):
p50 trajectory length ≈ 23,000 tokens — most don't fit
nebius/SWE-agent-trajectories (top-up)
788 unique successful issues
target == True only; 1 trajectory per GitHub issue (deduplicated at the issue level so the later train/eval split can't leak the same issue into both)
Final composition after dedup, 90/10 per-domain split, and whole-fit-or-drop length
filtering at 16,384 tokens:
Domain
Train rows
Eval rows
Train token share
Dropped for length
Coding
16,083
1,778
82.5%
598 (3.6% — accepted as pathological-outlier policy)
Tool
5,360
595
4.8%
0
Agentic
917
102
12.7%
2,321 (multi-turn trajectories that don't fit whole at 16k)
Total
22,360†
2,475
72.5M tokens
—
† One row was dropped by the trainer itself (all labels masked after truncation-safety
filtering) — 22,359 rows actually trained on.
Reasoning-token fraction (coding + agentic, which carry <think> traces) was 95.2% of
all training tokens, comfortably above the 75% floor generally recommended to preserve
reasoning ability when mixing in non-reasoning (tool-calling) data.
5. Training results
Eval loss reported per domain via three held-out splits, evaluated every 150 steps
(subsampled during training for speed: 400/1,778 coding rows, 200/595 tool rows, all 102/102
agentic rows — full-split numbers are pending a re-run and will be added if produced).
Step
Train loss
Eval — Coding
Eval — Tool
Eval — Agentic
150
—
0.5070
0.0246
0.3574
300
—
0.4911
0.0226
0.3257
450
—
0.4853
0.0217
0.3118
600
—
0.4823
0.0214
0.3056
750
0.5004
0.4794
0.0211
0.2981
900
0.4769
0.4781
0.0209
0.2946
1050
0.4931
0.4767
0.0208
0.2915
1200
—
0.4762
0.0208
0.2899
1350
—
0.4761
0.0208
0.2893
1398 (final)
PENDING
0.4761
0.0208
0.2892
All three eval splits decreased monotonically across the entire run with no split turning
up — no overfitting signal at 1 epoch. Coding, the dominant and highest-value domain,
continued improving through the final checkpoint rather than being degraded by the smaller
tool/agentic data — the outcome the mixed-run design targeted. Note that eval-loss
magnitudes are not directly comparable across domains (different sequence structure and
supervised-token counts per split); track each split against itself over time.
Single-batch training loss (0.500, 0.477, 0.493 at steps 750/900/1050) is noisy at
effective batch 16 and is shown only as a secondary signal — eval loss is the trend to read.
6. Chat template & tool-calling format
Standard ChatML markers, with <think> reasoning canonically formatted by the tokenizer's
chat template (verified — the template re-renders think content even if you pass it inline):
Tool calls use Qwen 3.6's native XML-style format (confirmed empirically against the
installed chat template — this is not the JSON-object tool_call format used by earlier
Qwen versions):
<|im_start|>assistant
<think>
</think>
<tool_call>
<function=get_weather>
<parameter=city>
Paris
</parameter>
</function>
</tool_call><|im_end|>
An empty <think></think> block is the canonical "direct response" form — tool calls in
this model are trained to skip reasoning and act directly. Parallel tool calls render as
sibling <tool_call> blocks. Pass your tool definitions via the standard
tools= argument to apply_chat_template; the template injects a # Tools system block
describing the exact call format automatically.
7. How to use
Recommended (tested path) — Unsloth:
python
1from unsloth import FastVisionModel
23model, processor = FastVisionModel.from_pretrained(4"trjxter/Qwimi-3.6-27B-BF16",5 load_in_4bit=False,# this repo is already full-precision BF166 max_seq_length=16384,7)8tokenizer = processor.tokenizer
9FastVisionModel.for_inference(model)1011messages =[{"role":"user","content":"Write a Python function to reverse a linked list."}]12text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)13inputs = tokenizer(text, return_tensors="pt").to(model.device)14out = model.generate(**inputs, max_new_tokens=1024, temperature=0.7, top_p=0.95)15print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Alternative — plain transformers (requires transformers>=5 for qwen3_5 support;
confirm the exact model class name against your installed version's docs before relying on
AutoModelForImageTextToText resolving correctly):
This model was fine-tuned and validated on text-only input; no image inputs were used
during training (see §9).
8. Verification & reproducibility
Adapter integrity: final adapter adapter_model.safetensors = 1,867.8 MB, matching
466,911,232 trainable params × 4 bytes (fp32 save) exactly — confirms the saved file
contains the complete, correctly-scoped trained adapter.
Merge integrity:max |lora_B| (the check that the adapter actually trained before
merging — an all-zero adapter would merge silently into an unchanged copy of the base) —
PENDING, not yet recorded from the merge run. Treat the merge as provisionally verified
pending this number.
Tokenizer/config repair: the merged repo's tokenizer_config.json originally recorded
a runtime-internal tokenizer_class value (TokenizersBackend, from transformers v5's
tokenizer backend) that AutoTokenizer.from_pretrained cannot resolve. This was corrected
in the repo to match the base model's tokenizer class — confirm your load succeeds before
reporting issues; this repair should make plain AutoTokenizer loading work correctly.
Checkpoint recovery: this run survived two Colab VM interruptions via checkpoints
persisted to Drive every 150 steps; the final adapter reflects the complete 1,398-step run,
not a recovered partial checkpoint.
9. Known limitations
Vision tower untouched. The base checkpoint is multimodal; this SFT explicitly froze
the vision layers (finetune_vision_layers=False) and trained on text-only data. No claims
are made about image understanding — it is whatever the base model shipped with, unvalidated
here.
Agentic data is thin and short. After length-filtering, only 917 agentic training rows
survived (12.7% of tokens), and DeepSWE trajectories longer than ~16k tokens were entirely
excluded rather than truncated — this model has seen comparatively little of the longest,
most complex agentic trajectories.
One low-quality source, deliberately capped.Claude-opus-4.7-TraceInversion-5000x
contains reasoning traces reverse-engineered by a small model from compressed summaries, not
genuine teacher traces. Capped at 577 rows (3.5% of the coding split) specifically to limit
this risk.
3.6% of coding examples were dropped for length (598 rows exceeding 16,384 tokens) —
accepted as pathological outliers rather than truncated, since truncating a reasoning trace
teaches the model to stop mid-thought.
Context validated to 16,384 tokens despite the base model's much longer native context;
behavior beyond that length is unvalidated by this training run.
10. Benchmarks
Qwimi was compared against the base Qwen 3.6 27B model using equivalent
Q6 GGUF builds with MTP enabled and the same evaluation harness.
Two benchmark suites were run:
A fully auto-scored 300-task custom benchmark covering coding, tool-calling, and
agentic workflows.
A broader 200-task mixed benchmark containing standard coding tasks and personal
Qwimi-oriented evaluations.
Important: the latency figures below measure average end-to-end wall time per task,
not raw tokens per second. SFT can reduce task completion time by producing shorter, more
direct, better-formatted responses and reaching the stop condition sooner. The results do
not prove that the underlying Q6 inference kernels became faster.
10.1 Custom 300-task benchmark
This benchmark contains 300 custom-authored, fully auto-scored tasks:
Category
Tasks
Coding
150
Tool-calling
75
Agentic workflows
75
Total
300
Both models answered the same 300 tasks, producing 600 total answer rows.
Overall results
Model
Tasks
Pass rate
Average wall time
Total wall time
Base Qwen 3.6 27B Q6 MTP
300
81.33%
53.56 s
267.79 min
Qwimi 3.6 27B Coder Q6 MTP
300
82.67%
27.09 s
135.45 min
Overall change
Result
Absolute pass-rate gain
+1.33 percentage points
Relative pass-rate gain
+1.64%
Average wall-time reduction
49.4%
Effective task-level speedup
~1.98×
Total evaluation time saved
132.34 minutes
The overall result is a small accuracy improvement combined with a very large reduction
in task-completion time.
Category results
Category
Base pass rate
Qwimi pass rate
Accuracy delta
Base avg time
Qwimi avg time
Wall-time reduction
Custom coding
81.33%
85.33%
+4.00 pts
59.33 s
21.16 s
64.3%
Custom tool-calling
88.00%
89.33%
+1.33 pts
31.77 s
27.13 s
14.6%
Custom agentic
74.67%
70.67%
-4.00 pts
63.80 s
38.92 s
39.0%
The strongest result was in custom coding, where Qwimi improved pass rate by four
percentage points while completing tasks with 64.3% lower average wall time.
Tool-calling showed a smaller but positive accuracy gain. Agentic accuracy regressed by four
points, although Qwimi still completed the agentic tasks substantially faster.
Pairwise outcomes
Pairwise result
Tasks
Share
Qwimi passed and base failed
23
7.67%
Base passed and Qwimi failed
19
6.33%
Both passed
225
75.00%
Both failed
33
11.00%
Total
300
100%
Among the 42 decisive tasks where only one model passed:
Decisive result
Count
Share
Qwimi wins
23
54.8%
Base wins
19
45.2%
Category-level pairwise outcomes
Category
Qwimi-only wins
Base-only wins
Both pass
Both fail
Coding
11
5
117
17
Tool-calling
4
3
63
5
Agentic
8
11
45
11
Total
23
19
225
33
300-task verdict: Qwimi achieved a slight overall quality win, driven by stronger
coding and tool-calling performance, while also completing the suite in roughly half the
time. The main remaining weakness was agentic-task accuracy.
10.2 Mixed 200-task benchmark
A separate 200-task evaluation compared the same Q6 MTP builds across standard coding tasks
and personal Qwimi-oriented tests.
The run produced 400 raw answer rows, with 200 answers from each model. Some categories
were omitted from the scored summary because their scorer/export integration was incomplete;
they were not treated as zero-score failures.
Overall auto-scored results
Model
Auto-scored rows
Passed
Pass rate
Average wall time
Base Qwen 3.6 27B Q6 MTP
151
46
30.46%
92.86 s
Qwimi 3.6 27B Coder Q6 MTP
166
51
30.72%
71.30 s
Overall change
Result
Absolute pass-rate gain
+0.26 percentage points
Relative pass-rate gain
+0.85%
Average wall-time reduction
23.2%
Pairwise auto-scored results
On the 146 tasks with a valid paired auto-score:
Pairwise result
Tasks
Share
Qwimi better
7
4.79%
Base better
2
1.37%
Same result
137
93.84%
Total paired tasks
146
100%
Among the nine decisive paired tasks, Qwimi won:
[
\frac{7}{9} = 77.8%
]
This corresponds to a net pairwise edge of +3.42 percentage points across all paired
auto-scored tasks.
Category highlights
Category
Base pass rate
Qwimi pass rate
Delta
HumanEval
80.00%
94.29%
+14.29 pts
Personal XML tool tasks
50.00%
100.00%
+50.00 pts
MBPP
5.77%
1.92%
-3.85 pts
Tool-calling checks
100%
100%
Tie
Architecture checks
100%
100%
Tie
Debugging checks
100%
100%
Tie
200-task verdict: the result was directionally positive for Qwimi, particularly on
HumanEval, XML-style tool behavior, and wall time. However, the overall pass-rate gain was
small, most paired results were ties, and MBPP performance regressed.
10.3 Combined interpretation
Across both benchmark suites, Qwimi demonstrated:
Higher overall pass rate in both runs.
Stronger performance on custom coding, tool-calling, HumanEval, and personal
XML-tool tasks.
A positive pairwise record:
23–19 on the 300-task custom benchmark.
7–2 on decisive paired tasks in the 200-task benchmark.
Substantially lower task-completion time:
49.4% lower on the custom 300-task suite.
23.2% lower on the mixed 200-task suite.
A remaining weakness in some areas:
-4.00 points on custom agentic accuracy.
-3.85 points on MBPP.
The fairest overall conclusion is:
Qwimi is a modest but measurable coding/tool-use improvement over the base model, with
unusually large task-level latency reductions. Its largest quality gains appear in coding
and structured tool usage, while agentic and MBPP performance require further work.
10.4 Interpreting the speed results
Both evaluated models used Q6 quantization, so the speed difference should not be interpreted
as Qwimi having fundamentally faster matrix-multiplication kernels.
Because both models use the same architecture family and quantization level, their raw decode
cost per token should be broadly similar. SFT can still reduce total wall time by changing
generation behavior:
reaching the correct solution with fewer tokens;
producing less unnecessary explanation or repetition;
following the expected output format sooner;
avoiding failed attempts, malformed tool calls, or self-correction loops;
emitting the stop token earlier.
Therefore, the supported claim is:
Qwimi completed the benchmark tasks faster.
A separate controlled measurement of generated-token counts and tokens per second would be
required to claim that Qwimi is intrinsically faster on a per-token basis.
10.5 Benchmark limitations
Only the Q6 GGUF builds were compared; these results should not be assumed to apply
identically to every quant listed in §2.
Wall time depends on hardware, llama.cpp build, prompt length, output length, sampling
settings, MTP support, and stop conditions.
The 300-task suite is custom-authored and is not a replacement for established public
benchmarks.
Some categories in the 200-task run were excluded because their official scorer/export
integration was incomplete; they were not counted as zero.
Agentic tasks used keyword/structural auto-scoring and may not capture every aspect of
repository-level software-engineering quality.
More seeds, repeated runs, and token-level throughput measurements would improve confidence
in the latency conclusions.
