[!Note]
This repository contains the Quantization-Aware Training (QAT) checkpoint of Opt.Gear-1B: master weights adapted to the deployment INT4 grid, together with the learned channel-wise quantization scales.
The deployment format is W4A16 (per-channel symmetric INT4 weights, 16-bit activations). The checkpoint performs best with its learned QAT scales, and also re-quantizes stably with other INT4 PTQ methods (RTN, AWQ, GPTQ).
Making a model small is not enough for real devices. Beyond parameter count, the memory occupied by weights, memory bandwidth, arithmetic precision, and the kernels supported by the accelerator determine final inference performance. The question is: how much of the original quality survives when the model is squeezed into 4 bits?
Opt.Gear-1B-QAT adapts the trained model directly to the deployment INT4 grid via Quantization-Aware Training. On six English/Korean benchmarks, the QAT model loses only ~5.8 point on average from the Base model — under the exact per-channel symmetric W4A16 grid used at deployment time.
We recommend using this model if your deployment framework lacks support for group-wise PTQ or if you prioritize inference speed over accuracy; otherwise, group-wise QAT weights are preferable for maximum accuracy.
QAT Highlights
Group-size-32 symmetric INT4 weight quantization with a W4A16 deployment format
Hessian-based scale initialization: channel-wise INT4 scales initialized by a layer-wise output reconstruction objective measured on calibration activations — not simple max-based statistics
Learned Step Size Quantization (LSQ): weights and step sizes jointly optimized under the full language-modeling objective, with straight-through gradient estimation
No quantizer mismatch: the same channel-wise INT4 grid is used during initialization, QAT, and final weight conversion, so the learned quantization structure survives deployment export
Quantization-friendly weights: the checkpoint re-quantizes stably with RTN/AWQ/GPTQ, giving deployment pipelines freedom in quantizer choice
For more details, please refer to our tech report and blog post.
Quantization Configuration
Item
Setting
Weight precision
INT4
Activation precision
INT16
Quantization format
W4A16
Weight quantization
Symmetric (zero-point fixed at 0)
Scale initialization
Hessian-based output reconstruction
Scale optimization
Learned Step Size Quantization
QAT length
1,000 steps
The scale for each quantization group is initialized as
Δ* = argmin_Δ ‖ WX − Q_Δ(W)X ‖²_F
where W is the full-precision weight matrix, X contains the calibration inputs, and Q_Δ(W) denotes the weights quantized with scale Δ. This objective accounts for the relative importance of different input directions instead of treating all weight perturbations equally. Starting from this initialization, QAT jointly refines the weights and the LSQ step sizes under the global training objective, while the forward pass consistently emulates the final W4A16 deployment format.
Benchmark Results
Base, QAT, and PTQ results of Opt.Gear-1B and Gemma3-1B on six English/Korean benchmarks. RTN, AWQ, and GPTQ rows re-quantize the master weights of the QAT checkpoint with each PTQ method; the QAT row evaluates the final QAT quantizer including learned scales. All results are obtained using the LM Evaluation Harness.
Method
MMLU
HellaSwag
WinoGrande
KMMLU
KoBEST
HAERAE
Avg.
Δ
Opt.Gear-1B
Base
43.2
56.0
58.2
36.0
60.3
44.2
49.7
–
QAT
37.1
51.0
55.9
30.4
55.4
33.9
43.9
-5.8
RTN
35.7
51.3
56.5
30.8
55.0
33.0
43.7
-6.0
GPTQ
35.3
51.7
55.3
25.5
54.8
33.8
42.7
-7.0
Gemma3-1B (reference)
Base
39.8
60.0
58.7
30.7
59.5
35.3
47.3
–
QAT
35.8
51.4
55.0
26.9
56.2
31.2
42.8
-4.6
RTN
32.8
49.9
57.1
24.8
54.8
23.0
40.4
-6.9
GPTQ
31.9
48.1
55.2
24.9
53.1
26.6
40.0
-7.3
Re-quantization Stability
The QAT checkpoint performs best with its learned scales, and the QAT-trained weights themselves respond stably to other INT4 quantizers:
Model
Quantizer
Avg
Δ vs QAT
Opt.Gear-1B-QAT-per-channel
Learned QAT scale
43.9
–
RTN
43.7
-0.2
GPTQ
42.7
-1.2
Gemma3-1B-QAT
Learned QAT scale
42.8
–
RTN
40.4
-2.4
GPTQ
40.0
-2.8
Under identical RTN re-quantization, Opt.Gear-1B loses only 0.2 points on average, versus 2.4 points for Gemma3-1B. The Opt.Gear-1B-QAT checkpoint is not merely a set of learned scale parameters — the weights themselves have been adapted to be quantization-friendly, giving deployment pipelines freedom in quantizer choice.
Quickstart
[!Important]
This checkpoint is intended for INT4 deployment. For best quality, export with the included learned channel-wise scales (the same per-channel symmetric W4A16 grid used during QAT). Re-quantizing with RTN/AWQ/GPTQ is supported and remains stable, at a small cost in accuracy (see table above).
Use the learned scales when possible: The learned QAT quantizer gives the best accuracy (Avg. 43.9). Switch to RTN/GPTQ only when your deployment toolchain requires it — the accuracy cost is small but non-zero.
Keep the grid consistent: The checkpoint was trained against a per-channel symmetric INT4 grid with 16-bit activations (W4A16). Exporting to a different group size or asymmetric grid discards the learned quantization structure.
Termination tokens: Same as the full-precision model — the Instruction model terminates with <EOT>, the Base model with <EOS>.
Model capabilities: Task capability follows Opt.Gear-1B — Korean-English bilingual understanding and instruction following; not intended for code generation. See the base model card for full benchmark results and architecture details.
Citation
If you find our work helpful, feel free to give us a cite.
