Qwen 3.5 4B — reasoning-directed mixed-precision IQ2_XS GGUF
1---
2base_model: Qwen/Qwen3.5-4B
3language:
4- multilingual
5pipeline_tag: text-generation
6license: apache-2.0
7tags:
8- qwen3.5
9- gguf
10- llama.cpp
11- mixed-precision
12- task-aware-quantization
13- reasoning
14- cada
15---
Qwen 3.5 4B — reasoning-directed mixed-precision IQ2_XS GGUF
This is an experimental, reasoning-directed mixed-precision GGUF derived from Qwen3.5-4B.
It was produced by QLAB, which measures quantization damage and redistributes precision at the tensor level under an approximately fixed byte budget.
This release is important because it is the first QLAB result outside the Gemma family.
On the reasoning evaluation, the stock IQ2_XS + category-imatrix quant scored 46.875%. The QLAB allocation using the same calibration strategy scored 54.688%.
That is:
- +7.813 percentage points
- +16.67% relative reasoning performance
- at only +0.412% model size
The BF16 source scored 78.125%, giving this artifact 70.0% reasoning retention versus 60.0% for the stock IQ2_XS comparator.
This is a targeted quantization experiment, not a claim that the resulting model is universally better. Eight of eleven measured suites improved over the stock comparator, while knowledge QA, structured output, and coherence regressed.
See Interpretation and limitations before using this model.
Model details
| Property | Value |
|---|
| Architecture | Qwen3.5 4B |
| Format | GGUF |
| Quantization | Per-tensor mixed precision searched around an IQ2_XS byte budget |
| File size | 1,637,318,816 bytes (1.525 GiB) |
| Stock comparator | 1,630,594,336 bytes (1.519 GiB) |
| BF16 source | 8,424,393,632 bytes (7.846 GiB) |
| Size reduction from BF16 | 80.56% |
| Candidate vs stock size | +0.412% |
| Candidate SHA-256 | [CANDIDATE SHA-256] |
| Stock SHA-256 | [STOCK SHA-256] |
| Source SHA-256 | [SOURCE SHA-256] |
| Target category | Reasoning |
| Base model | Qwen/Qwen3.5-4B |
The upstream Qwen3.5-4B model is released under the Apache License 2.0.
What QLAB changed
This is not a uniform IQ2_XS quantization.
QLAB begins with a category-calibrated quantization and then searches for tensor-level precision reallocations intended to recover capability lost during quantization.
Some tensors are assigned more precision while others give precision back so that the resulting artifact remains near the original byte budget.
For this experiment:
- Stock IQ2_XS + imatrix: 1,630,594,336 bytes
- QLAB allocation + imatrix: 1,637,318,816 bytes
- Difference: 6,724,480 bytes
- Relative increase: 0.412%
There is no LoRA, post-training, pruning, distillation, or weight optimization involved.
The measured difference comes from changing where the quantization budget is spent.
Usage
Use a recent build of llama.cpp:
1llama-cli \
2 -m qwen3.5-4b-CADA-IQ2_XS.gguf \
3 -cnv
Or start an OpenAI-compatible server:
1llama-server \
2 -m qwen3.5-4b-CADA-IQ2_XS.gguf \
3 --host 127.0.0.1 \
4 --port 8080
Qwen3.5 is natively multimodal upstream. This release contains the GGUF language-model artifact only unless a matching projector is explicitly included in this repository.
Multimodal behavior was not evaluated as part of this QLAB release.
Evaluation
Target result: reasoning
| Artifact | Bytes | Reasoning score | Retention vs BF16 |
|---|
| BF16 source | 8,424,393,632 | 78.125% | 100.0% |
| Stock IQ2_XS + imatrix | 1,630,594,336 | 46.875% | 60.0% |
| QLAB allocation + imatrix | 1,637,318,816 | 54.688% | 70.0% |
Allocation effect
54.688 − 46.875 = +7.813 percentage points
Relative improvement over the stock comparator:
+16.67%
The candidate uses only 0.412% more bytes than the stock comparator.
Absolute scores across all measured suites
| Suite | BF16 | Stock IQ2_XS + imatrix | QLAB allocation + imatrix | QLAB − stock |
|---|
| Reasoning | 78.125 | 46.875 | 54.688 | +7.813 |
| Math | 32.812 | 4.688 | 13.281 | +8.594 |
| Coding | 44.531 | 2.344 | 4.688 | +2.344 |
| Knowledge QA | 65.625 | 54.688 | 39.062 | −15.625 |
| Instruction following | 26.562 | 19.048 | 23.438 | +4.390 |
| Structured output | 57.812 | 51.562 | 42.188 | −9.375 |
| Summarization / extraction | 68.750 | 32.812 | 45.312 | +12.500 |
| General fidelity | 36.500 | 30.515 | 30.813 | +0.299 |
| Coherence | 81.250 | 85.938 | 67.188 | −18.750 |
| Stability | 41.546 | 35.888 | 38.139 | +2.251 |
| Context | 100.000 | 86.458 | 92.708 | +6.250 |
The QLAB candidate improved 8 of 11 measured suites by point estimate.
What improved
The largest positive allocation effects were:
- Summarization / extraction: +12.500 points
- Math: +8.594
- Reasoning: +7.813
- Context: +6.250
- Instruction following: +4.390
- Stability: +2.251
- Coding: +2.344
- General fidelity: +0.299
Reasoning was the optimization target.
The improvements outside reasoning are secondary observations and should not be interpreted as independently optimized capabilities.
What regressed
Three measured suites scored below the stock comparator:
- Knowledge QA: −15.625 points
- Structured output: −9.375
- Coherence: −18.750
These are meaningful tradeoffs.
This artifact should therefore be treated as a reasoning-directed specialized quant, not as a universally superior IQ2_XS replacement.
Why this result matters
Earlier QLAB experiments demonstrated allocation effects on Gemma-family models.
This run demonstrates the same basic mechanism on Qwen3.5, a different model family and architecture.
At an essentially fixed IQ2_XS storage budget, tensor-level allocation increased reasoning performance from:
46.875 → 54.688
while increasing model size by only:
0.412%
This reduces the likelihood that the allocation behavior observed in earlier experiments is specific to Gemma.
It does not establish universal transfer across architectures. Additional models and larger scales still need to be tested.
Evaluation protocol
| Property | Value |
|---|
| Target | Reasoning |
| Evaluation system | MLAB |
| Source artifact | [SOURCE ARTIFACT] |
| Stock comparator | IQ2_XS + same category imatrix |
| Candidate | QLAB tensor-level allocation + same imatrix |
| MLAB run | [MLAB RUN ID] |
| Evaluation pack | [PACK NAME / VERSION] |
| Bootstrap resamples | [BOOTSTRAP COUNT] |
| Bootstrap seed | [BOOTSTRAP SEED] |
| MLAB report SHA-256 | [REPORT SHA-256] |
| llama.cpp commit | [LLAMA.CPP COMMIT] |
The candidate and comparator should be interpreted as a matched-budget comparison. The candidate is 0.412% larger, which remains inside QLAB's ±0.5% byte-parity window.
Interpretation and limitations
This is an experimental research artifact.
The strongest supported claim from this release is narrow:
At approximately the same IQ2_XS byte budget, QLAB tensor-level allocation improved measured reasoning performance from 46.875% to 54.688%, a +7.813-point / +16.67% relative improvement over the stock category-imatrix comparator.
Do not interpret this as:
- IQ2_XS matching BF16 generally
- a universally better Qwen3.5 quant
- proof that the same allocation strategy works on every architecture
- proof that every category benefits
- evidence of multimodal preservation
Known weaknesses
Coding remains heavily degraded.
The candidate improves coding from 2.344% to 4.688%, but the BF16 source scores 44.531%. The relative improvement should not be confused with practical coding parity.
Intended use
This model is intended for:
- quantization research
- tensor-level mixed-precision experimentation
- studying capability recovery near aggressive quantization regimes
- reasoning-focused local inference where a roughly 1.5 GiB model is useful
- community evaluation of QLAB/CADA
Evaluate the model against your own workload before relying on it.
It should not be used as the sole source of truth for medical, legal, financial, safety-critical, or other high-stakes decisions.
License and attribution
This derivative is distributed under the Apache License 2.0, consistent with the upstream Qwen3.5-4B release.
Upstream model:
License:
Qwen3.5 is developed by the Qwen team.
QLAB produced the task-directed tensor-level mixed-precision allocation described in this model card.
MLAB performed the independent evaluation.
Project links
QLAB / ByteOtter updates:
Support larger QLAB experiments:
Community support goes toward compute for larger experimental runs, including Qwen 27B-class models. Funding an experiment does not guarantee a successful result. Failed, neutral, and negative experiments are part of the research process.
Citation
For the upstream model, follow the citation guidance provided by the Qwen team:
1@misc{qwen3_5,
2 title = {Qwen3.5},
3 author = {{Qwen Team}},
4 year = {2026},
5 howpublished = {\url{https://huggingface.co/Qwen/Qwen3.5-4B}}
6}
If referencing this quantization artifact specifically, please also link directly to this Hugging Face repository.
