Moonshot AI announced Kimi K3 on July 16: a 2.8-trillion-parameter open model with a 1-million-token context window, native vision, and two architectural bets that go beyond scaling up the usual recipe. The weights land on Hugging Face by July 27. This page is our deep-dive, prepared in advance: the benchmarks and the architecture story now, the traced graph the moment the checkpoint drops.
Announced release time: July 27, 2026 at 00:00 UTC.
Kimi K3 specifications: 2.8 trillion parameters, top-16 of 896 experts, one-million-token context, a three-to-one KDA to Gated MLA cycle, MXFP4 weights with MXFP8 activations, and a July 27 weights release
The architecture: four ideas carry it
K3 is not a scaled-up K2. Moonshot credits the jump to roughly 2.5× better scaling efficiency than Kimi K2, and the launch post names the mechanisms plainly. Each one lands on a trend line we have been tracking across the 2,400+ models traced on hfviewer.
Kimi Delta Attention · the workhorse
KDA is Moonshot’s delta-rule take on linear attention: short convolutions, L2-normalized keys, and gated state updates instead of a quadratic attention matrix. Three of every four attention layers in K3 are KDA. That is the industry’s loudest trend right now: linear-attention layers appear in 29% of new models in our catalog this quarter, up from near zero a year ago.
Every fourth attention layer is full multi-head latent attention with output gating, keeping global, lossless retrieval in the stack while KDA does the bulk work. MLA itself just had its breakout quarter: 23% of new models now ship it, and with K3, Moonshot pairs it with linear attention rather than choosing between them.
AttnRes is the genuinely novel piece: instead of every block adding uniformly onto one residual stream, blocks selectively retrieve representations from earlier depths (the α operators in Moonshot’s diagram). Think of it as attention over depth, not just sequence. No open model we have traced does this; it will be a new shape on the graph.
Feed-forward blocks route top-16 of 896 experts, the largest expert pool ever shipped in the open. Balancing at that sparsity is its own research problem: Moonshot’s Quantile Balancing derives allocation from router-score quantiles instead of an auxiliary loss, alongside a SiTU activation and Per-Head Muon optimizer. See MoE routing in the glossary for the family this extends.
Animated top-to-bottom Kimi K3 architecture cycle with KDA, Gated MLA, Stable LatentMoE, and Attention Residuals
Left to right: the 3:1 KDA-to-MLA cycle every token flows through · the expert pool one LatentMoE routes over · the α operators retrieving from earlier depths.
The sparsity race, on one chart
Every dot below comes from a graph traced from real weights on hfviewer; K3 is plotted from the launch post. The direction is unmistakable: expert pools keep growing while the active fraction falls. K3 pushes the pool to 896 but activates 16, more than recent frontier MoEs, landing at 1.8% active.
The original hfviewer sparsity chart showing routed expert count against the share active per token
Active share counts routed experts only (shared always-on experts excluded). Sources: traced configs on hfviewer; Kimi K3 from Moonshot’s launch post.
Benchmarks
Moonshot’s own framing is unusually candid: K3 “still trails the most powerful proprietary models” overall while leading everything else. The numbers back that up. Across the 33 public rows in the launch table, K3 is first or tied-first on 8, ahead of every other open model on essentially all of them, and usually within a few points of the frontier. Standouts: SWE Marathon (42.0, two points clear of the field and triple the GPT 5.5 score), BrowseComp (91.2), DeepSearchQA (95.0), and OmniDocBench (91.1).
Animated bars comparing four highlighted Kimi K3 benchmark scores with the strongest comparison shown in Moonshot AI's table
All K3 scores at max reasoning effort. Harnesses differ per benchmark (KimiCode, Claude Code, Codex) and several third-party scores are cited from external leaderboards or release blogs; Claude Fable 5 was evaluated by a third party and may include fallback behavior. Rows marked cited mix first-party and cited numbers; internal rows are Moonshot-internal. Elo rows use a different scale. Full footnotes in the launch post.
What it will look like on hfviewer
Until July 27, the closest traced relative is Kimi K2.7-Code: an MLA MoE decoder routing top-8 of 384 experts plus one always-on shared expert. Explore it below, then picture the K3 diff: three quarters of the attention layers turning into KDA blocks with their convolutions and gates, the expert pool growing to top-16 of 896, and a web of AttnRes edges that no current graph has.
