Views
No views yet
| Property | Value |
|---|---|
| Parameters | 12B |
| Architecture | Mistral (Transformer) |
| Context Length | 8K tokens |
| Vocabulary | Extended (~20K Georgian tokens) |
| Languages | Georgian (ka), English (en), other (limited) |
| Training Tokens | ~30B |
| Training | Continue pre-training (embeddings + high-rank LoRA) |
| Base Model | mistralai/Mistral-Nemo-Base-2407 |
pip install transformers torch accelerate1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model = AutoModelForCausalLM.from_pretrained(
4 "tbilisi-ai-lab/kona2-12B-Base",
5 torch_dtype="auto",
6 device_map="auto"
7)
8tokenizer = AutoTokenizer.from_pretrained("tbilisi-ai-lab/kona2-12B-Base")
9
10# Text completion
11text = "საქართველო არის"
12inputs = tokenizer(text, return_tensors="pt").to(model.device)
13outputs = model.generate(**inputs, max_new_tokens=100)
14print(tokenizer.decode(outputs[0], skip_special_tokens=True))| Model | Description |
|---|---|
| kona2-12B-Instruct | Instruction-tuned version (SFT) |
| kona2-12B | Preference-aligned version (DPO) |
| kona2-small-3.8B | Smaller 3.8B model |
Mistral-Nemo-Base-2407
│
├── Expand Vocabulary (+20K Georgian tokens)
│ └── Initialize with token average
│
└── Continue Pre-training (~30B tokens)
└── Full embeddings training + high-rank LoRA
│
└── kona2-12B-Base ← YOU ARE HERE
│
└── SFT (~2.8M examples)
│
└── kona2-12B-Instruct
│
└── DPO (387K pairs)
│
└── kona2-12B1@misc{tbilisi2025kona2base,
2 title = {Kona2-12B-Base: A Georgian Language Model},
3 author = {Tbilisi AI Lab Team},
4 year = {2025},
5 publisher = {Hugging Face},
6 howpublished = {\url{https://huggingface.co/tbilisi-ai-lab/kona2-12B-Base}}
7}