🇨🇿 Czech Language & Personality LoRA for Qwen3.5-9B
A Czech LoRA trained for Qwen3.5-9B using Unsloth.
The purpose of this LoRA is not to increase the model's knowledge.
Instead, it teaches the model how to communicate naturally in Czech, improving sentence structure, personality consistency and conversational quality.
✨ Features
- 🇨🇿 More natural Czech
- 📝 Improved Czech word order
- 💬 Better conversational flow
- ❤️ Consistent personality
- 😊 More natural emotional responses
- 🤝 Better long conversations
- ❓ Improved question asking
- 🙋 Better handling of uncertainty and apologies
🧠 Base Model
| Parameter | Value |
|---|
| Base Model | Qwen3.5-9B |
| Framework | Unsloth |
| Format | LoRA |
⚙️ Training Parameters
| Parameter | Value |
|---|
| Rank (r) | 64 |
| Alpha | 128 |
| Dropout | 0.05 |
| Epochs | 20 |
| Batch Size | 16 |
| Learning Rate | 8e-5 |
| Scheduler | Cosine |
| Warmup Steps | 10 |
| Weight Decay | 0.01 |
| Gradient Clipping | 1.0 |
| Seed | 3407 |
| Context Length | 8192 |
📚 Dataset
Approximately 483 manually written conversations focused on natural Czech dialogue.
The dataset includes:
- everyday conversations
- emotional dialogue
- personality consistency
- natural Czech grammar
- Czech word order
- asking questions
- admitting uncertainty
- apologies
- long conversational context
🎯 Recommended Settings
During testing the best results were achieved with:
| Setting | Value |
|---|
| LoRA Multiplier | 0.6 |
Higher multipliers tended to over-emphasize the personality and occasionally produced overly philosophical responses.
📈 Goal
This LoRA focuses on improving:
- natural Czech sentence construction
- dialogue quality
- conversational realism
- personality consistency
rather than increasing factual knowledge.
🙏 Acknowledgements
Special thanks to the developers of:
- Qwen Team
- Unsloth
- HuggingFace
for making this project possible.
👤 Author
Ondřej Šimek
Czech Republic 🇨🇿
