This physically pruned, recovery-tuned, and MXFP4-packed checkpoint is specialized for English-to-Simplified-Chinese translation. It is a CONTRASTIVE Chinese ESTS checkpoint for the WMT 2026 Model Compression shared task.
1git clone https://github.com/oceanusm/wmt26-ests-model-compression.git
2cd wmt26-ests-model-compression
3
4SUB=submissions/ests-gptoss-zho-k28
5MODEL_CACHE="$HOME/.cache/ests-models" \
6MODELZIP_SOURCE="$PWD" \
7bash "$SUB/setup.sh"
8
9printf '%s\n' 'The meeting begins at nine tomorrow morning.' > /tmp/ests-input.txt
10CUDA_VISIBLE_DEVICES=0 bash "$SUB/run.sh" \
11 --lang-pair eng-zho_Hans \
12 --batch-size 1 \
13 --input /tmp/ests-input.txt \
14 --output /tmp/ests-output.txt
15cat /tmp/ests-output.txt
This checkpoint requires the custom variable-capacity MoE/MXFP4 implementation provided by the linked runner. It is not a drop-in stock Transformers model.
Experts were ranked within each layer by accumulated routing mass, and language-specific layerwise allocation determined how many experts to retain. Router and expert tensors were physically sliced and compactly remapped, the pruned model was recovery-tuned, and the routed expert weights were packed to MXFP4.
This checkpoint is intended for English-to-Simplified-Chinese translation, not general assistant use. Aggressive expert removal may degrade unrelated capabilities, synthetic recovery targets may introduce bias or errors, translations can still be inaccurate or incomplete, and no additional safety evaluation was performed.
1@article{martin2026extracting,
2 title={Extracting Small Translation Specialists from LLMs by Aggressively Pruning Experts},
3 author={Martin, Liu O. and Bandarkar, Lucas and Peng, Nanyun},
4 journal={arXiv preprint arXiv:2605.28042},
5 year={2026}
6}
Apache 2.0, following the base model.
