Qwen3.5 4B Dflash, self-quantized to GGUF by Atomic Chat. Built straight from Z Lab's original weights with a per-tensor importance matrix, so this is not a repack of somebody else's files. Runs fully offline.

Highlights

0.6B parameters: the weights this repo quantizes.
Context length: 262,144 tokens (256K), as published by Z Lab.
6 layers: Dense decoder, hybrid sliding-window (4096) and global attention.
Full imatrix ladder: every quant is calibrated with an importance matrix.

[!NOTE] These GGUFs are self-quantized from the original weights, not a repack. The importance matrix keeps low-bit quants closer to the full-precision model.

[!IMPORTANT] Always pass --jinja so the Qwen3.5 4B Dflash chat template is applied. Without it the model can emit malformed turns.

Model Overview

Property	Value
Base model	`z-lab/Qwen3.5-4B-DFlash`
Parameters	0.6B
Layers	6
Sliding window	4096 tokens
Context length	262,144 tokens (256K)
Vocabulary	248,320
Modalities	Text
Architecture	Dense decoder, hybrid sliding-window (4096) and global attention, 32 attention heads over 8 KV heads, `DFlashDraftModel`
This repo	GGUF quants (imatrix). Quants: `Q8_0`

Choosing a quant

Quant	Size	Notes
`Q8_0`	0.7 GB	Effectively lossless, reference quality.

[!TIP] Pick the largest file that fits your (V)RAM with room for context. Q8_0 is the sweet spot for most setups; Q6_K or Q8_0 for maximum fidelity.

Get started

Run Qwen3.5 4B Dflash locally with:

Atomic Chat: the easiest path. Open the app, search AtomicChat/Qwen3.5-4B-DFlash-GGUF, pick a quant, hit Use this model.
llama.cpp: llama-server -hf AtomicChat/Qwen3.5-4B-DFlash-GGUF:Q8_0 --jinja -c 8192
Ollama: ollama run hf.co/AtomicChat/Qwen3.5-4B-DFlash-GGUF:Q8_0
LM Studio / Jan: search the repo id, download any quant.

Best practices

Parameter	Value
sampling defaults	not stated

The base model card does not state sampling defaults.

Run in llama.cpp

bash

git clone https://github.com/ggml-org/llama.cpp
cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --target llama-cli llama-server

bash

./llama.cpp/build/bin/llama-server \
    -hf AtomicChat/Qwen3.5-4B-DFlash-GGUF:Q8_0 \
    --jinja -ngl 99 -c 8192 -fa on

How these were made

Download z-lab/Qwen3.5-4B-DFlash (original weights).
Convert to f16 GGUF with llama.cpp.
Build an importance matrix over our calibration corpus.
Quantize the ladder with --imatrix.

License

Original model by Z Lab, released under the Apache 2.0 license. Quantized by Atomic Chat.