DeepSeek V4 Flash IQ2_XXS, MoEspresso package
This repository holds a MoEspresso package, and MoEspresso is the only engine that serves it:
https://github.com/steadfastgaze/MoEspresso.
llama.cpp, transformers, and mlx-lm will not load these files.
Install and serving instructions live in the MoEspresso repository and are deliberately kept out of this card.
Quality
| check | scope | result |
|---|
| Official next-token agreement | Five fixed prompts and 17 greedy next-token decisions, compared with captures from the DeepSeek-V4-Flash API | 17/17 on one tested MLX build; 16/17 on another. The sole difference was lowercase versus uppercase C; the reference token was the runner-up locally. |
| Official continuation loss (lower is better) | 100 prompts, 2,290 reference tokens | Average target-token NLL: 0.3765–0.3783; first-token agreement: 66/100. |
| Long-context recall | 16 fixed facts in a 30,474-token prompt | 16/16 exact match. |
The expert-weight source was compared on the same 100-prompt check:
| expert weights | average NLL | first-token agreement |
|---|
| Re-encoded from the bf16 release using the same codec assignments | 0.3915 | 53/100 |
| Byte-copied from antirez's GGUF | 0.3765 | 66/100 |
Speed
Measured on an Apple M3 Max with 128 GB of unified memory - MoEspresso engine.
| metric | value |
|---|
| decode | 26.06 and 26.24 tok/s, two independent runs on a 3844-token anchor, greedy |
| time to first token | 14.667 s median on the same anchor, about 262 t/s of prefill |
| longest tested context | 113,855 tokens grown in one session; the live cache holds 4.47 GB at that depth |
Recipe
The quantization scheme is a faithful reproduction of the IQ2_XXS recipe antirez designed for his ds4 engine, recorded as such in the package's own recipe report. The inputs:
The routed experts ship byte for byte from antirez's GGUF: gate and up projections in iq2_xxs (86 tensors), down projections in q2_k (43). The 345 dense targets are encoded fresh from the bf16 release into q8_0, following the recipe's assignments: all five attention projection classes, the shared experts, and the output head. The DSv4-specific auxiliary blocks (indexer, compressor, HC) are stored in MoEspresso's affine and mxfp8 formats, and norms pass through in f32.
One clarification the recipe report makes explicit: the imatrix file plays no role in quantization here. The expert bytes are copied and the q8_0 dense encoding takes no importance weighting. Its only use is ranking experts by observed routing frequency to build expert_hotlist.json, the cold-start prewarm list.
DeepSeek_V4.pdf is the base model's paper and ships because the manifest declares it. package_manifest.json records the sha256 of every file; after downloading, moespresso-verify checks the package against it in one command.
Files
Forty-seven safetensors shards (80.0 GiB, 1328 tensors) plus the sidecars the engine reads: package_manifest.json, package_plan.json, expert_hotlist.json, source_inventory.json, and the recipe report. Tokenizer files come from the original release.
Credits
This quantization is derived from antirez's quantization work for his ds4 engine. The base model is DeepSeek V4 Flash by DeepSeek. Everything in the chain is MIT: the original release, antirez's GGUFs, and this package.
