Views
No views yet
| Parameter | Nilai |
|---|---|
| Parameters | 58.5M |
| Training Samples | 50,047 |
| Context Length | 512 tokens |
| Epochs | 3 |
| Validation Loss | 1.236 |
| Validation PPL | 3.44 |
| Validation Accuracy | 71.0% |
| Metric | Value |
|---|---|
| Avg tokens | 13.21 |
| Distinct-2 | 73.8% |
| Distinct-3 | 89.0% |
| Degeneration Rate | 3.0% |
| Topic Drift | 15.0% |
vocab_size: 8196
n_layers: 20
n_heads: 8
n_kv_heads: 2
hidden_dim: 512
ffn_dim: 1408
context_length: 512
rope_theta: 10000
activation: SwiGLU
normalization: RMSNorm| Epoch | Val Loss | PPL | Accuracy |
|---|---|---|---|
| 1 | 2.296 | 9.94 | 49.0% |
| 2 | 1.665 | 5.28 | 61.1% |
| 3 | 1.236 | 3.44 | 71.0% |
| Turns | Percentage |
|---|---|
| 2 turns | 64.5% |
| 3 turns | 7.0% |
| 4 turns | 22.0% |
| 5+ turns | 6.5% |
Teach the model how to converse naturally in Indonesian, not how to answer knowledge questions.
<|user|>\nHalo\n<|assistant|>\n
Output: Halo, aku baik. Kamu?<|user|>\nUdah makan?\n<|assistant|>\n
Output: Udah, kamu?<|user|>\nGue capek banget\n<|assistant|>\n
Output: Parah juga ya. Banyak kerjaan?<|user|>\nWeekend mau ngapain?\n<|assistant|>\n
Output: Nggak tau nih. Lo?