A GGUF quantization (Q4_K_M) of kakaocorp/kanana-2-30b-a3b-instruct-2601, so it runs in llama.cpp — e.g. as the Korean endpoint of local-llm-toolkit on a 16 GB GPU.

Kanana-2 uses a DeepSeek-V3-style MLA + MoE architecture that ExLlamaV3 (EXL3) does not support, so GGUF via llama.cpp is the practical path to run it locally on consumer hardware. Only the file was quantized — no weights were retrained or altered.

This is a Derivative Work of Kanana-2, redistributed under the KANANA LICENSE AGREEMENT (see license_link). Per the agreement's redistribution terms: this notice and the KANANA license are included, the base model is attributed above, and this is identified as a modified (quantized) version. Use must follow Kakao's Guidelines for Responsible AI. All rights in the underlying model remain with Kakao Corp.