Gwimi-4-12B-IT-BF16 is a merged BF16 release of a Gemma 4 12B instruction model that underwent two consecutive post-training stages:
Supervised Fine-Tuning (SFT) on a 20,000-example curated reasoning mixture.
Group Sequence Policy Optimization (GSPO) on 12,000 frozen reinforcement-learning prompts with programmatic correctness, formatting, and anomaly rewards.
The final release contains the original BF16 base weights with the cumulative SFT + GSPO LoRA updates merged into them. No separate adapter is required for inference.
The GSPO phase continued training the same cumulative adapter produced by SFT. This was not a second independent adapter stacked on top of the SFT adapter.
The final merge can be summarized as:
text
1Exact Gemma 4 12B BF16 base
2 +
3Cumulative SFT + GSPO LoRA update
4 =
5Standalone Gwimi-4-12B-IT-BF16 model
The SFT stage trained the model to imitate high-quality reasoning responses across mathematics, science, coding, debugging, technical reasoning, general reasoning, and multilingual STEM material.
The model was trained for one epoch using a large rank-128 rsLoRA adapter while the base model was loaded in 8-bit precision.
SFT dataset composition
The final SFT corpus contained exactly 20,000 examples.
Curated sources
Dataset
Rows
trjxter/Kimi-K2.6-Technical-Reasoning-AddOn-3300x
3,301
trjxter/Kimi-K2.6-Reasoning-3300x-WandB
3,303
trjxter/Gemma-4-31B-Reasoning-1000x
995
Jackrong/Claude-opus-4.7-TraceInversion-5000x
4,761
Curated subtotal
12,360
Kimi K2.5 top-up source
The remaining 7,640 examples were selected from:
Jackrong/Kimi-K2.5-Reasoning-1M-Cleaned
Category
Rows
General-Distillation
4,000
General-Math
1,500
PHD-Science
1,500
MultilingualSTEM
640
Top-up subtotal
7,640
Final SFT dataset arithmetic
text
112,360 curated rows
2+ 7,640 Kimi K2.5 top-up rows
3= 20,000 total rows
The dataset-building process used normalized-prompt deduplication. Curated rows were given priority over top-up rows when a duplicate was found.
The final top-up selection had zero normalized-prompt overlap with the curated set.
SFT train/evaluation split
Split
Rows
Training
18,000
Held-out evaluation
2,000
Total
20,000
SFT hyperparameters
Parameter
Value
Maximum sequence length
32,768
Base loading precision
8-bit
LoRA rank
128
LoRA alpha
256
rsLoRA
True
LoRA dropout
0
Bias
none
Microbatch size
2
Gradient accumulation
8
Effective optimizer batch
16
Epochs
1
Optimizer steps
1,125
Learning rate
2e-5
Warmup ratio
0.03
Scheduler
cosine
Optimizer
adamw_8bit
Maximum gradient norm
1.0
SFT evaluation design
The 2,000 held-out examples were divided into:
100 fixed anchor examples
1,900 examples in a rotating evaluation pool
During training, evaluation was scheduled every 250 optimizer steps using:
The fixed anchor set made checkpoint-to-checkpoint comparisons more consistent. The rotating slices increased coverage without requiring a full 2,000-example evaluation every 250 steps.
A final evaluation was then run over all 2,000 held-out examples.
Recorded SFT evaluation results
Checkpoint-1000 retrospective evaluation
Evaluation slice
Loss
Fixed anchor, 100 rows
0.636239767
Rotation 1, 100 rows
0.699529946
Rotation 2, 100 rows
0.659535289
Rotation 3, 100 rows
0.768394113
Rotation 4, 100 rows
0.665917516
Rotation 5, 100 rows
0.696190119
Mean rotating loss
0.697913396
Combined 600-row mean
0.687634458
These checkpoint-1000 results should not be confused with the final model evaluation.
Final full held-out evaluation
Metric
Result
Held-out rows
2,000
Final evaluation loss
0.6940310597419739
Final perplexity
2.001768540
These are teacher-forced SFT evaluation metrics. They measure prediction quality on held-out target responses and are not equivalent to free-generation benchmark accuracy.
Stage 2: GSPO reinforcement learning
What GSPO means in this run
GSPO stands for Group Sequence Policy Optimization.
For each prompt, the policy generated multiple candidate completions. Programmatic reward functions scored those completions, and the trainer compared their reward values within the prompt group.
The defining GSPO behavior in this run was:
importance_sampling_level = "sequence"
This uses one importance ratio per generated response sequence rather than a separate ratio for every token.
The implementation used TRL's GRPOTrainer and GRPOConfig, but sequence-level importance sampling made the optimization GSPO-style.
Frozen GSPO dataset suite
The RL dataset was frozen before training and stored as reproducible artifacts.
Training split
Source
Rows
dapo_math_en
8,400
openmath_reasoning_mini
1,800
sciq
1,800
Total GSPO training prompts
12,000
Anchor evaluation split
Source
Rows
dapo_math_en
100
openmath_reasoning_mini
100
sciq
100
Anchor total
300
Held-out evaluation split
Source
Rows
dapo_math_en
500
openmath_reasoning_mini
500
sciq
897
Held-out total
1,897
Protected SciQ test split
Split
Rows
Protected SciQ test
991
The frozen dataset suite had zero cross-split prompt overlap.
The GSPO training worker used only the 12,000-row training split. The anchor, held-out, and protected SciQ splits were reserved for evaluation and contamination control.
Reward functions
The training run used three frozen reward functions:
The correctness reward evaluated whether the generated answer matched the expected solution criteria. Mathematical answer comparison used a parser-oriented verification approach rather than relying only on literal string equality.
Format reward
The format reward encouraged the response structure expected by the training pipeline, including a valid final-answer structure and properly formed output.
Anomaly reward
The anomaly reward was designed to detect suspicious, malformed, repetitive, degenerate, or parser-exploiting output patterns.
The reward functions were frozen before the official run so that the optimization target could not silently change during training.
GSPO hyperparameters
Parameter
Value
Training prompts
12,000
Model loading precision
BF16
Learning rate
2e-6
Warmup steps
100
Importance-sampling level
sequence
Loss type
grpo
Reward scaling
group
KL coefficient beta
0.0
Lower clipping epsilon
3e-4
Upper clipping epsilon
4e-4
Generations per prompt
8
Unique prompts per rollout
3
Total completions per rollout
24
Per-device training batch
2
Gradient accumulation
4
Effective optimizer batch
8
Steps per generation
12
Optimizer updates per rollout
3
Maximum completion length
1,280
Maximum runtime sequence length
4,096
Temperature
1.15
Top-p
0.95
Repetition penalty
1.05
Mask truncated completions
True
Maximum gradient norm
1.0
Scheduler
cosine
Checkpoint cadence
every 10 optimizer steps
Final checkpoint
2250
The KL penalty was intentionally disabled. A zero KL value in this configuration does not mean that the policy was identical to a reference model; it means that KL regularization was not part of the objective.
GSPO stopping point
The official run stopped at:
global_step = 2250
The run had originally resumed from checkpoint 450, so the successful continuation added approximately 1,800 optimizer updates.
By the stopping point, the run had processed approximately 15.55 million tokens according to the training telemetry.
The decision to stop at 2250 was based on:
stable entropy rather than continued collapse;
controlled completion lengths;
a learned and saturated format reward;
active but non-saturated sequence clipping;
continued reward variation;
complete resumable checkpoints;
and diminishing expected benefit relative to additional compute cost.
Near-terminal GSPO telemetry
The values below are a near-terminal W&B snapshot around global step 2250. They describe the final observed training batch or local window, not an external benchmark score.
Metric
Near-terminal value
Interpretation
Combined reward
0.2667
Mean aggregate reward for the observed batch
Correctness reward
0.1667
Correctness component for the observed batch
Format reward
0.1000
Format component was effectively saturated
Reward standard deviation
0.3086
The batch retained meaningful reward variation
Fraction of zero-variance reward groups
0.6667
Two of three prompt groups had equal rewards across their eight samples
Entropy
0.0893
Low but stable near the stopping point
Sequence clip ratio, region mean
0.125
Sequence-level clipping was active but not saturated
Mean completion length
762.875 tokens
Completions remained substantial and below the 1,280-token cap
Completion clipped ratio
0.25
Six of 24 completions reached the truncation condition in the observed rollout
Gradient norm
8.3416
Pre-clipping gradient magnitude; updates were constrained by max_grad_norm=1.0
How to interpret these metrics
Reward: Higher is generally better only when the reward implementation correctly reflects the desired behavior. Reward is not equivalent to standardized benchmark accuracy.
Reward standard deviation: Some variation is necessary because GSPO learns from differences among completions in the same prompt group.
Fraction of zero-variance groups: A high value means some prompt groups produced no relative reward signal. The metric was noisy because each rollout contained only three unique prompts, so possible values naturally moved in increments of one third.
Entropy: Entropy is not simply “higher is better.” In this run it declined and then stabilized around 0.09 rather than continuing toward zero. That was treated as evidence against an obvious late-stage policy collapse.
Sequence clip ratio: Some clipping is expected in GSPO. The near-terminal value showed that the trust region was active without clipping the entire rollout.
Completion clipped ratio: This is different from policy-ratio clipping. It measures completions that reached the maximum-length condition. Because truncated completions were masked from policy loss, a high value would reduce training efficiency.
Gradient norm: The logged value was above 1.0 before clipping. The configured maximum gradient norm constrained the actual optimizer update.
Important GSPO evaluation note
The online GSPO worker used:
eval_strategy = "no"
The frozen anchor, held-out, and protected SciQ splits were preserved, but they were not used as periodic online evaluations during the expensive generation loop.
Therefore:
the SFT loss and perplexity results above are official recorded evaluation results;
the GSPO values above are training telemetry;
no claim is made here that the final model has already beaten the base model or SFT-only checkpoint on a standardized external benchmark;
a proper comparison should evaluate the base, SFT-only checkpoint, and final SFT+GSPO model under the same prompts, decoding settings, and scoring implementation.
This distinction is intentional. Reward curves are useful for diagnosing RL training, but they are not substitutes for independent benchmark evaluation.
The final adapter was merged into the exact untouched BF16 base revision:
0f03bc8c5a5d620f219daea1ef93e05775ae9b74
The original base and merged model had the same byte size because the merge changed existing BF16 tensor values rather than adding a second set of parameters:
This verifies that the uploaded BF16 model contains modified cumulative SFT + GSPO weights and is not an unchanged copy of the original base model.
Intended uses
This model is intended for experimentation with:
mathematical and scientific reasoning;
technical question answering;
coding and debugging assistance;
long-form structured reasoning;
instruction following;
local inference research;
comparison of SFT-only and SFT+RL post-training pipelines.
The model is best treated as a research and development release rather than a guaranteed production system.
Limitations
No independent final benchmark suite is reported yet.
The README separates SFT evaluation loss, GSPO training telemetry, and future independent evaluation.
Reward optimization can inherit reward-function blind spots.
Correctness, formatting, and anomaly rewards cannot measure every aspect of answer quality.
The GSPO stage focused heavily on verifiable mathematics, science, and structured reasoning prompts.
Performance may vary outside those distributions.
Long-context behavior was not independently benchmarked after GSPO.
SFT used a maximum sequence length of 32,768, while GSPO used a 4,096-token runtime window with a 1,280-token completion cap.
Text behavior was the focus of this post-training project.
Any other capabilities inherited from the base model were not independently evaluated here.
The model can still hallucinate, make calculation errors, produce unsafe advice, or follow incorrect premises.
Outputs should be verified for high-stakes use.
Training reward should not be interpreted as benchmark accuracy.
The terminal reward values describe the model under the training reward system and rollout distribution.
Loading the BF16 model
This model was exported through Unsloth's Gemma 4 unified implementation.
A Transformers installation that does not recognize the gemma4_unified architecture may fail to load the model. Use a current compatible Unsloth and Transformers environment.
Example with Unsloth
python
1import torch
2from unsloth import FastLanguageModel
34model_id ="trjxter/Gwimi-4-12B-IT-BF16"56model, processor = FastLanguageModel.from_pretrained(7 model_name=model_id,8 max_seq_length=4096,9 dtype=torch.bfloat16,10 load_in_4bit=False,11)1213FastLanguageModel.for_inference(model)1415messages =[16{17"role":"user",18"content":[19{20"type":"text",21"text":"Solve the problem carefully and provide a clear final answer: What is 17% of 240?"22}23],24}25]2627inputs = processor.apply_chat_template(28 messages,29 add_generation_prompt=True,30 tokenize=True,31 return_dict=True,32 return_tensors="pt",33).to(model.device)3435with torch.inference_mode():36 output_ids = model.generate(37**inputs,38 max_new_tokens=512,39 do_sample=True,40 temperature=0.7,41 top_p=0.95,42)4344generated_ids = output_ids[0, inputs["input_ids"].shape[-1]:]4546print(47 processor.decode(48 generated_ids,49 skip_special_tokens=True,50)51)
For deterministic evaluation, use greedy decoding or a low-temperature configuration and keep decoding parameters identical across compared models.
Recommended evaluation approach
For a meaningful comparison, evaluate all three checkpoints under identical conditions:
text
11. Original Gemma 4 12B instruction base
22. Gwimi SFT-only checkpoint
33. Final Gwimi SFT + GSPO model
Keep the following fixed:
prompt set;
prompt formatting;
chat template;
maximum generated tokens;
temperature and top-p;
random seeds;
answer extraction;
reward and benchmark scoring;
hardware and inference backend where practical.
Useful evaluation categories include:
exact-answer mathematics;
scientific multiple choice;
code generation and debugging;
instruction-format compliance;
long-response termination behavior;
repetition and anomaly rate;
response length;
pass@1 and sampled pass@k;
qualitative reasoning review.
Training and release hardware
Stage
Hardware
SFT
NVIDIA RTX PRO 6000 Blackwell Server Edition
GSPO
NVIDIA H200 on Modal
Final BF16 merge
NVIDIA A100-SXM4-80GB
Acknowledgements
This release builds on:
the Gemma model family;
Unsloth;
Hugging Face Transformers, PEFT, TRL, and Hub;
Math-Verify;
the dataset authors and teacher-model outputs represented in the SFT and GSPO data mixtures.
License
This model is a derivative of Gemma and remains subject to the applicable Gemma license and terms of use.
Users are responsible for reviewing the upstream license and ensuring that their intended use complies with it.
