Hy3-REAP-48e — REAP-pruned Hunyuan Hy3 (experimental)

⚠️ Experimental research artifact. This is an aggressive prune (75% of experts removed) calibrated on a small, code-only set. It generates fluent text and is genuinely strong at code, but general/factual accuracy is degraded (see below). Not intended for production. A better-calibrated version is planned — this repo will be updated in place.

How it was made

• Method: REAP, one-shot (no retraining). Each expert is scored by the mean of router_weight × activation_norm over a calibration set; the least-salient are dropped.
• Compression: compression_ratio 0.75 → 192 → 48 experts kept per MoE layer.
• Calibration: 64 samples from theblackcat102/evol-codealpaca-v1 (code).
• Streaming: the 557 GB model was built on meta and each of the 80 layers streamed from the on-disk safetensors shards (~7 GB resident at a time), for both calibration and the prune. See the How To Spark training notes for the recipe.

Quality profile (sanity check)

Running it

• Two GPUs / two nodes — e.g. vllm serve <repo> --pipeline-parallel-size 2 (vLLM supports HYV3ForCausalLM). This model was validated running pipeline-parallel across two DGX Sparks.
• 4-bit quantization (~40 GB) to fit a single device (tooling for hy_v3 quantization is still maturing).

License