This model was trained with a local staged SFT mixture built for Qwen3.5-4B ChatML formatting.
Some source groups are locally generated or locally transformed training subsets rather than standalone public datasets. The table reflects the local manifest names used during dataset construction.
The SFT file was generated by:
scripts/build_experts_plus_reasoning_sft.py
Variant: qwen35_4b
Format: Qwen ChatML
Seed: 20260707
Total examples: 65,131
Vision / multimodal rows: excluded
Data Mixture
Source
Rows
Share
OpenThoughts-114k reasoning subset
19,073
29.28%
Math reasoning
8,000
12.28%
Code/debug expert data
6,000
9.21%
Experiment-loop reasoning
6,000
9.21%
Python/tool-use expert data
6,000
9.21%
Persona / Lime response data
5,000
7.68%
Time-series reasoning
5,000
7.68%
General repair data
4,000
6.14%
Time-series text prediction
4,000
6.14%
Safety/control data
1,591
2.44%
Dense anchor alignment
391
0.60%
Reasoning trajectory prediction
76
0.12%
Filtering
The dataset was deduplicated and filtered before training.
Skipped rows:
Reason
Count
Duplicate messages
21,280
Requires image
1,405
Dense anchor invalid rows
1,200
Remaining image marker rows
932
Model Changes
This model is derived from a Qwen3.5-4B base model.
The text stack was expanded from 32 decoder layers to 40 decoder layers.
The final staged SFT pass trained the top 8 text layers only:
Trainable layers: 32-39
Frozen base layers: 0-31
Trainable parameters: 892,512,384
Total parameters: 5,431,777,920
This should be understood as a Qwen3.5-4B-derived expanded text-stack model, not a from-scratch model.
Training Notes
The dataset manifest contains 65,131 text-only SFT examples. The final staged training summary records a short top-layer training pass using 5,000 training examples and 500 validation examples.
Training was performed with Qwen ChatML formatting and text-only data. Vision reasoning, image-dependent examples, and remaining image-marker rows were excluded.
The final stage focused on adapting the newly expanded upper text layers to reasoning, expert, tool-use, repair, safety/control, and time-series style data while keeping the original lower Qwen layers frozen.
Prompt / Formatting
All examples were rendered into canonical Qwen ChatML format.
The source system prompts were replaced with a target Qwen3.5-4B system prompt:
You are LimeCore on a Qwen3.5-4B base. Solve reasoning tasks directly and compactly. Use public reasoning summaries, equations, state labels, or tool decisions only when they are part of the requested answer. Do not expose private chain-of-thought; for planner samples, output the requested public planner JSON or state labels exactly.
