ZAYA1-8B-JANGTQ4

Important Runtime Note

Runtime Pin Required

Architecture Summary

• 80 decoder layers: alternating CCA attention and top-1 MoE
• Hidden size 2048, 16 query heads, 2 KV heads, head dim 128
• CCA state per attention layer: standard KV plus conv_state [B,1280,2] and prev_hs [B,2048]
• 16 routed experts per MoE layer, top-1 routing with MOD skip route
• Context length 131072, rope_theta=5000000

Quantization

• conv_qk.*, temp, norms, residual scaling, router path, biases, and balancing biases are preserved as float tensors.
• Embeddings and lm_head use 8-bit affine in the prepared bundles.
• jangtq_runtime.safetensors is included: true.

Bundle Verification

• Safetensor headers scanned.
• Source tensor coverage checked.
• Converted bundles checked for local_experts removal.
• Converted expert tensors checked for pre-stacked switch_mlp layout.
• JANGTQ sidecars checked for the Swift runtime contract.
• Capabilities verified: family=zaya, supports_thinking=False, tool_parser=zaya_xml.

Korean Summary

Files

• config.json carries weight_format=mxtq, zaya_expert_layout=split_switch_mlp.
• jang_config.json carries cache_subtype=zaya_cca.
• Tokenizer files and chat_template.jinja are preserved from the upstream source snapshot.