[!WARNING]
Line status (2026-08-12): use with caution. This first-generation REAM release passed its structural and smoke checks at publication time, but the cascaded descendants built from it (REAM128, REAM96) later showed severe multi-turn degradation, and the whole REAM line is being rebuilt with a revised methodology (single-step prune-first from the base model, longer multi-turn calibration). A REAM144 replacement targeting the same memory tier is planned; this repo may be superseded.
DeepSeek V4 Flash 0731 REAM160-180B DS4 Q2
⚠️ This checkpoint requires a patched inference engine. The upstream
antirez/ds4 does not recognize the 160-expert topology.
This is a 2-bit DS4-specific GGUF of the REAM160 variant of DeepSeek V4 Flash.
REAM160 reduces the routed-expert count from 256 to 160 via calibration-guided,
training-free expert merge, then quantizes to IQ2_XXS / Q2_K for routed tensors
and Q8 for attention, shared experts, and output projections.
Quantization layout
- Routed gate/up: IQ2_XXS.
- Routed down: Q2_K.
- Attention projections, shared experts, and output: Q8.
- MTP and DSpark tensors and metadata are absent.
- File size: 53.4 GiB.
Native REAM and quantization methodology
REAM160 is a calibration-guided, training-free expert merge: it does not apply
gradient updates or add training examples to the checkpoint.
Calibration composition: 3,072 samples × 512 tokens.
| Slice | Samples | Share | Pinned sources and focus |
|---|
| Code | 614 | 20.0% | sahil2801/CodeAlpaca-20k and bigcode/self-oss-instruct-sc2-exec-filter-50k; instruction following and executable code |
| Math | 307 | 10.0% | openai/gsm8k and AI-MO/NuminaMath-CoT; arithmetic and mathematical reasoning |
| Agent/tool | 1,075 | 35.0% | NousResearch/hermes-function-calling-v1; function selection and structured tool calls |
| Russian/Cyrillic | 1,076 | 35.0% | ZeroAgency/ru-big-russian-dataset; Russian and Cyrillic language coverage |
A disjoint 512-sample heldout split was excluded from calibration.
At each layer, REAP saliency is the routed activation norm multiplied by router
probability. The 160 highest-saliency source experts become centroids; the
remaining source experts are assigned by their mean output and router-gate
similarity, with group capacity 32. Each group is merged by saliency-weighted
deterministic FP32 accumulation, router rows are remapped, and top-k 6 routing
is preserved. Calibration therefore determines which source experts are retained
and combined for the code, math, agent/tool, and Russian/Cyrillic workload mix;
it is not fine-tuning.
The DS4 importance matrix reuses the exact REAM160 calibration activations with
the packed-expert mapping. Quantization consumes it with --imatrix and
--imatrix-strict: routed gate/up tensors use IQ2_XXS, routed down tensors use
Q2_K, and attention projections, shared experts, and output use Q8. Conversion
is direct from Safetensors to GGUF without an intermediate F16 checkpoint.
Usage
Clone the fork and build:
1git clone https://github.com/iamwavecut/ds4
2cd ds4
3make
Full residency (requires ≥56 GiB GPU wired memory on macOS):
1sudo sysctl iogpu.wired_limit_mb=62976
2./ds4 -m DeepSeek-V4-Flash-0731-REAM160-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf \
3 --nothink
SSD streaming (for machines that cannot hold the full 53.4 GiB model):
1./ds4 -m DeepSeek-V4-Flash-0731-REAM160-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf \
2 --nothink --ssd-streaming
--nothink selects bounded direct replies. Remove it only when enabling thinking
with a completion budget suitable for longer reasoning.
This is a DS4-specific GGUF, not a generic llama.cpp or Transformers artifact.
Measured quality
- Comparative heldout NLL/PPL: not measured yet.
- DS4 continuation suite: 100/100 pass.
- Quantization and validation wall time: 9.304 hours.
Semantic request modes: direct=75, thinking=25.
Performance benchmarks
Hardware: Apple M2 Max, 64 GB unified memory, macOS, Metal backend.
Full model residency (53.4 GiB mapped), iogpu.wired_limit_mb=62976.
Greedy decoding (--temp 0 --seed 42 --nothink), 128 generated tokens,
10 runs per context size, prompts padded to ~70% of context.
| Context | Prompt size | Prefill t/s (median) | Generation t/s (median) |
|---|
| 4K | ~2K tok | 214.37 | 19.02 |
| 8K | ~5K tok | 184.15 | 11.45 |
| 16K | ~10K tok | 131.62 | 10.32 |
| 32K | ~22K tok | 153.99 | 9.56 |
Detailed per-run statistics:
| Context | Runs | Prefill avg | Prefill min–max | Gen avg | Gen min–max |
|---|
| 4K | 10 | 211.33 | 175.70–236.50 | 18.87 | 15.96–20.86 |
| 8K | 10 | 183.13 | 159.51–197.52 | 10.60 | 3.27–13.51 |
| 16K | 10 | 130.86 | 90.48–161.60 | 8.63 | 2.59–11.03 |
| 32K | 10 | 150.38 | 128.17–162.35 | 8.19 | 2.45–10.71 |
SSD streaming mode (same hardware, for comparison): ~6–7 t/s prefill,
~8–9 t/s generation at 4K context.
License
MIT License — same as the upstream DeepSeek-V4 weights.
