You can deploy this model and start earning money today!

HaithamalWaisy/
kimi-k3-merged

Name: kimi-k3-merged
Availability: InStock

Views

No views yet

Kimi K3 — Merged 2-Expert Model (66.3 GB)

Fully dense, 62.23 B-parameter model produced by collapsing all 896 routed experts of each MoE layer of Kimi K3 into 2 super-experts using the MergeMoE expert-output-merging technique.

Derived from: Moonshot AI's Kimi K3 (open weights, Apache 2.0) via Unsloth's 1-bit GGUF conversion (UD-IQ1_S, 14 shards).

Paper: Kimi K3 Under Compute Constraints: Implementing MergeMoE to Take 14 Shards to 5GB on Consumer Hardware

Author: Haitham al-Waisy · Contact: haithamalwaisy@gmail.com · X/Twitter · YouTube

Code: https://github.com/Haitham-alwaisy/Kimi-K3-MergeMoE

Model anatomy

93 blocks, all 896 experts per layer → 2 super-experts (usage-frequency-weighted average)
All-Q8_0/F32 weights (1,392 Q8_0 + 1,181 F32 tensors)
62.23 B parameters, 100% active per token

Intended use

This model is intended as a base for fine-tuning and Reinforcement Learning. It is recommended to be run inside the Prime Agent harness.

Usage

hf download HaithamalWaisy/kimi-k3-merged kimi-k3-merged-2exp.gguf --local-dir .

License

Apache 2.0 (inherited from Kimi K3). Quality was not benchmarked by this project.