2026/07/27: 🤗 Released kanana-2-3b, kanana-2-1.3b HF model weights.
2026/07/27: 📕 Published a blog post about the development of the Kanana-2 SLM series.
Introduction
We present Kanana-2 SLM, Kakao's second series of Small Language Models (SLMs), designed to deliver strong language capabilities while remaining compact and efficient for practical deployment. The series includes 3B, 1.3B, and 0.9B models. This release publicly includes the 3B model and the compressed 1.3B model, providing a balance between capability and efficiency for a wide range of applications.
Kanana-2-3B was pretrained from scratch on TPU clusters and further improved through post-training with supervised fine-tuning and reinforcement learning, resulting in strong instruction-following and reasoning capabilities.
Kanana-2-1.3B models are derived from Kanana-2-3B through a cascade pruning and distillation pipeline. To further improve deployment efficiency, they adopt Sliding Window Attention (SWA), enabling memory-efficient long-context inference with support for context lengths of up to 32K tokens while substantially reducing KV-cache memory requirements.
The Kanana-2 SLM release consists of the following four publicly available models:
Kanana-2-3B-Base — 3B pretrained base
Kanana-2-3B-Instruct — instruction-tuned 3B model
Kanana-2-1.3B-Base — compressed 1.3B pretrained base
Kanana-2-1.3B-Instruct — instruction-tuned 1.3B model for on-device deployment
[!NOTE]
No Kakao user data was used for either pre-training or post-training.
Highlights
Cascade Pruning & Distillation: Kanana-2-1.3B is built by progressively compressing Kanana-2-3B-Base (3B → 2B → 1.3B → 0.9B) through a cascade pruning and distillation pipeline.
Sliding Window Attention (SWA): Uses a 3:1 hybrid layout of sliding-window and full-attention layers. A sliding-window size of 1024 reduces per-token KV-cache reads, cutting KV-cache usage by up to ~72.7% at a 32K context length compared to a full-attention-only model. YaRN is applied to full-attention layers, while SWA layers retain RoPE, preserving long-range context without sacrificing local-attention efficiency.
Kanana-2 tokenizer: Improves Korean tokenization efficiency by over 30% compared to the previous generation.
Long context: Natively supports context lengths of up to 32,768 tokens.
Instruction-following, chat, tool-calling, code, math, and knowledge benchmarks for the Kanana-2 SLM series. Scores use greedy decoding (temperature 0.0, top-p 1.0, max 4096 tokens); the metric for each benchmark is listed in the Metric column.
Benchmark
Metric
kanana-2-3b-instruct
kanana-2-1.3b-instruct
Qwen3.5-2B
Qwen3-1.7B
Chat
MT-Bench†
judge
7.15
6.83
6.87
6.98
KoMT-Bench†
judge
6.92
6.54
5.21
5.29
Instruction Following
IFBench
prompt strict
33.33
34.69
24.83
18.33
IFEval
prompt strict
80.96
77.63
66.91
68.39
IHEval
pass@1
35.96
27.06
38.52
42.09
Tool Calling
BFCL-v3 (Live)‡
pass@1
71.94
69.64
66.96
65.48
BFCL-v3 (Multi-Turn)‡
pass@1
17.12
5.50
6.27
4.38
Code Generation
MBPP
pass@1
70.63
69.05
55.56
62.17
MBPP+
pass@1
60.05
60.85
46.83
52.65
Mathematics
GSM-Plus
pass@1
61.17
58.27
61.29
63.10
MATH-500
pass@1
61.20
61.40
67.80
72.00
Minerva Math
pass@1
24.44
22.43
35.18
27.21
Reasoning & Knowledge
MMLU-CoT
acc
61.09
60.37
69.01
66.02
KMMLU-CoT
acc
43.32
42.79
41.75
37.84
HAERAE-Bench (v1.0)-CoT
acc
43.75
44.89
27.84
27.84
KoSimpleQA
acc
22.29
17.81
3.21
2.82
† Evaluated using gpt-4o-2024-08-06 as the judge model.
‡ Live denotes the average score of 6 live benchmarks, and Multi-Turn the average score of 4 multi-turn benchmarks.
Quickstart
[!NOTE]
kanana-2-1.3b-base uses a custom hybrid attention architecture (Kanana2TinyForCausalLM — a Qwen3 backbone with a 3:1 SWA/full-attention layout and per-layer-type RoPE), shipped as remote code. Loading requires trust_remote_code=True and transformers >= 4.57.
python
1import torch
2from transformers import AutoModelForCausalLM, AutoTokenizer
34model_name ="kakaocorp/kanana-2-1.3b-base"56tokenizer = AutoTokenizer.from_pretrained(7 model_name,8 trust_remote_code=True,9)10model = AutoModelForCausalLM.from_pretrained(11 model_name,12 trust_remote_code=True,13 dtype=torch.bfloat16,14)15device = torch.device("cuda"if torch.cuda.is_available()else"cpu")16model.to(device)17model.eval()1819prompt ="Kakao is a leading company in South Korea, and it is known for"2021input_ids = tokenizer(22[prompt],23 return_tensors="pt",24)["input_ids"].to(device)2526with torch.no_grad():27 output = model.generate(28 input_ids,29 max_new_tokens=32,30 do_sample=False,31)3233decoded = tokenizer.decode(output[0], skip_special_tokens=True)34print(decoded)
