This is an unofficial REAP K160 compact variant of DeepSeek V4 Flash 0731.
Quality evaluation is still in progress, and some capabilities are significantly degraded compared with the full model.
What is this?
This repository contains a REAP K160 / 160-expert compact variant of DeepSeek V4 Flash 0731, created by pruning routed experts directly from an already-working quantized GGUF.
The goal is to reduce memory usage while preserving as much of the behavior of the known-working Q2 quantization as possible.
Why does this exist?
There are already quantized REAP K160 variants available.
However, this model takes a different conversion path.
Instead of starting from an already-pruned K160 checkpoint and quantizing its remaining weights again, this variant starts from an existing working Q2 GGUF and only removes the experts excluded by the K160 pruning map.
This avoids an additional:
quantized / mixed-precision weights
↓
dequantization
↓
requantization to Q2
step for the retained expert weights.
The practical motivation is simple: I want a K160 model small enough to run comfortably on memory-constrained local hardware while remaining useful for real agentic workloads.
Current status
The model is still under evaluation.
Confirmed
Loads and runs with REAP-compatible DS4 forks.
Tested with the ROCm REAP runtime on AMD Strix Halo.
Produces coherent English output.
Has successfully completed a real agentic task involving tool use in my testing.
Uses only 160 routed experts per MoE layer instead of the original 256.
Not yet fully evaluated
General benchmark quality
Coding benchmark performance
Agentic task success rate across a larger test set
Long-context reliability
Comparison against the full Q2 model
Comparison against other K160 quantizations
ROCmFP2 variants
More results will be added as testing progresses.
Important limitation: Japanese
Japanese capability is effectively unusable in my testing after REAP K160 pruning.
Normal Japanese conversation cannot be reliably sustained.
This is not a small quality regression; for practical purposes, do not use this model for Japanese-language workloads.
English agentic use is the primary target of my current testing.
Q2 quantization
The current Q2 variant is derived from the imatrix quantization in:
Additional quantization variants may be added to this repository later.
In particular, a ROCmFP2 variant is under consideration.
Inference
This model uses a compact REAP K160 layout.
The routed expert tensors physically contain fewer expert slots than the original DeepSeek V4 Flash model, so a runtime with explicit compact REAP support is required.
However, upstream DS4 currently does not directly load this compact K160 GGUF.
The original model stores 256 routed experts, while this model physically stores only 160. Attempting to load this compact model with an incompatible upstream build results in a tensor shape validation error such as:
ds4: tensor blk.0.ffn_gate_inp.weight has dim[1]=160, expected 256
Use one of the REAP-compatible forks listed above instead.
Hardware target
My primary test system is:
AMD Ryzen AI MAX+ 395 (Strix Halo)
96 GB unified memory
ROCm
Linux
The broader goal is to make DeepSeek V4 Flash practically usable on hardware where the full-size model leaves too little memory headroom for context, KV cache, agent workloads, and other system processes.
Planned evaluation
I plan to compare:
Full asymmetric Q2
vs
This REAP K160 Q2
vs
Other K160 quantizations
Across several dimensions:
Model size
Peak memory usage
Prefill speed
Decode speed
General quality benchmarks
Coding capability
Tool-call reliability
Agentic task completion
Repetition / generation failures
Long-context behavior
The purpose is not only to ask whether the model "runs", but whether the resulting compressed model remains practically usable.
The K160 expert selection is used to determine which routed experts are retained.
The actual retained expert weights in this Q2 release come from the already-quantized antirez Q2 GGUF rather than from quantizing the 0xSero K160 checkpoint.
DeepSeek V4 Flash 0731 is released under the MIT License.
This repository is an unofficial derivative built using resources from the projects referenced above. See each upstream repository for its original license notices and attribution information.
Disclaimer
This is an unofficial experimental derivative.
It is not an official release from DeepSeek or the upstream projects referenced above.
Compatibility information, benchmark results, model quality, and recommended runtimes may change as the model and inference software continue to evolve.
