[!TIP]
Support this work: donate.sybilsolutions.ai
REAP surfaces: GLM | MiniMax | Qwen | Gemma | Paper | Code | PR17 | Cerebras Collection
Benchmark-free calibration dataset for REAP (Routing-Enhanced Activation Pruning) of Mixture-of-Experts language models.
REAP prunes MoE models by removing experts that rarely activate. To decide which experts are safe to remove, REAP needs to observe which experts fire on diverse… See the full description on the dataset page:
https://huggingface.co/datasets/0xSero/reap-calibration-data-v1.