Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16 is a merged BF16 fine-tuned version of unsloth/Qwen3.5-9B.
This model was trained as a reasoning-focused SFT/distillation run using a curated mixture of Kimi K2.6, Qwen reasoning, and Claude Opus TraceInversion-style reasoning datasets. The goal of this run was to improve structured reasoning behavior while preserving Qwen-style chat formatting and <think>...</think> reasoning traces.
Developed by:trjxter
Base model:unsloth/Qwen3.5-9B
Model type: Merged BF16 causal language model
Training method: LoRA / RS-LoRA SFT with Unsloth + TRL
License: Apache 2.0
Language: English
Training Overview
This model was trained with Unsloth and Hugging Face TRL using a LoRA-based supervised fine-tuning setup.
Training Configuration
Setting
Value
Base model
unsloth/Qwen3.5-9B
Sequence length
16,384
Training examples
12,000
Held-out eval examples
366
Trainer eval subset
200
Epochs
1
Effective batch size
16
Per-device batch size
2
Gradient accumulation steps
8
LoRA rank
128
LoRA alpha
128
RS-LoRA
Enabled
Base loading
8-bit
Optimizer
adamw_8bit
Learning rate
2e-5
Scheduler
Linear
Gradient checkpointing
Unsloth
Runtime
~4.37 hours on an 80GB GPU
Final Training Metrics
Metric
Value
Final training loss
0.5517
Final lightweight eval loss
~0.3161
Train runtime
15,728.8s
Train samples/sec
0.763
Train steps/sec
0.048
Total FLOPs
1.45e18
The lightweight eval loss was measured on a 200-example eval subset during training.
Dataset Mix and Curation
This run used a combined reasoning/distillation dataset made from three sources:
trjxter/Kimi-K2.6-Reasoning-3300x-WandB
Jackrong/Qwen3.5-reasoning-700x
Jackrong/Claude-opus-4.6-TraceInversion-9000x
The datasets were normalized into Qwen chat format and assistant reasoning traces were preserved in the following style:
text
1<think>
2...
3</think>
4final answer
After formatting and 16k token filtering, the final usable dataset contained 12,366 examples.
Split
Examples
Notes
Training
12,000
Used for the main SFT run
Held-out eval
366
Reserved from the usable formatted dataset
Trainer eval subset
200
Lightweight eval subset used during training
Dataset Credits
Special thanks to Jackrong and Kyle Hessling for the Opus and Qwen reasoning datasets used in this run. These datasets are not mine, and this model card acknowledges their work and dataset contributions.
The Kimi dataset used in this run was prepared under trjxter/Kimi-K2.6-Reasoning-3300x-WandB, while the Qwen reasoning and Claude Opus TraceInversion datasets came from Jackrong's public dataset releases.
Intended Use
This model is intended for experimentation with:
reasoning-style SFT
synthetic distillation
long-context reasoning behavior
math and structured problem solving
coding and technical reasoning prompts
local inference after conversion to quantized formats
This is an experimental fine-tune and should be evaluated carefully before use in production or high-stakes settings.
Prompt Format
The model follows Qwen-style chat formatting.
Example:
text
1<|im_start|>user
2Solve the problem step by step: If a worker earns $9/hour for 8 hours, plus $15 for each of 7 racquets, $10 for each of 2 grommet replacements, and $1 for each of 5 stencils, how much do they earn?
3<|im_end|>
4<|im_start|>assistant
5<think>
6...
7</think>
8...
9<|im_end|>
When using transformers, prefer applying the tokenizer chat template instead of manually formatting prompts.
Loading with Transformers
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
2import torch
34model_name ="trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16"56tokenizer = AutoTokenizer.from_pretrained(model_name)78model = AutoModelForCausalLM.from_pretrained(9 model_name,10 torch_dtype=torch.bfloat16,11 device_map="auto",12)1314messages =[15{16"role":"user",17"content":"Solve this step by step: A shop earns $72 from hourly pay, $105 from restringing, $20 from grommets, and $5 from stencils. What is the total?"18}19]2021prompt = tokenizer.apply_chat_template(22 messages,23 tokenize=False,24 add_generation_prompt=True,25)2627inputs = tokenizer(prompt, return_tensors="pt").to(model.device)2829outputs = model.generate(30**inputs,31 max_new_tokens=1024,32 temperature=0.6,33 top_p=0.95,34 do_sample=True,35)3637print(tokenizer.decode(outputs[0], skip_special_tokens=False))
Related Releases
This run may also be released in adapter and GGUF formats:
This model was trained using Unsloth for efficient fine-tuning and Hugging Face TRL for SFT training.
Disclaimer
This is an experimental research fine-tune. Outputs may contain mistakes, hallucinations, or incorrect reasoning. Always validate important outputs independently.
