You can deploy this model and start earning money today!

Thermostatic/
rosettia-chanka-4b-base

Name: rosettia-chanka-4b-base
Availability: InStock

Views

No views yet

YouTube entry: Rosettia video

rosettia-chanka-4b-base

The 4B Chanka-specialized base model used as the starting point for the Rosettia compact-mixed SFT chain that produced Thermostatic/rosettia-chanka-4b-alpha160 (chrF++ 56.94 on held-out clean Chanka).

This is not the strongest deployable artifact — it scores chrF++ 43.49 on the 158-row held-out, well below the final champion. It is published so other researchers can train alternative LoRA adapters on the same Chanka-specialized base without having to redo the multi-hour broad+full-FT chain.

Held-out result

Metric (158-row clean Chanka held-out)	This base	Final champion	Δ
chrF++	43.49	56.94	+13.45
BLEU	16.14	30.76	+14.62
token F1	28.94	46.43	+17.49
TER (↓)	82.49	62.21	−20.28

How this base was produced

Stage	Recipe
0	Raw `unsloth/Qwen3.5-4B`
1	Broad Quechua LoRA SFT (~768 steps on ~169k AmericasNLP + SomosNLP `quy-spa` pairs, LoRA r=64/α=128, LR 5e-6)
2	Merge the broad LoRA into the full base
3	Full-parameter FT on the clean Chanka subset (48 steps, LR=2e-6, `paged_adamw_8bit`) — produces this checkpoint

How to use

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("Thermostatic/rosettia-chanka-4b-base")
model = AutoModelForCausalLM.from_pretrained(
    "Thermostatic/rosettia-chanka-4b-base",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
# Use the same prompt format as the champion model card.

Or attach the compact-mixed LoRA chain on top with PEFT for the champion behavior (see the champion model card for the full recipe).

Intended use

Starting point for further Chanka SFT / LoRA experiments
Reproducing the Rosettia compact-mixed chain (v11→v12→v13)
Domain-adapting to other Chanka subdomains (judicial training data is the source here)

Limitations

Trained on 1,055 reviewed Chanka pairs from a single judicial manual — out-of-domain coverage is unknown
Chanka variety only (quy_Latn); not appropriate for Cuzco-Collao (quz), Bolivian (quh), or other Quechua varieties without further adaptation
Tokenizer-level: inherits Qwen3.5-4B vocabulary (no Quechua-specific tokens added)

Citation / attribution

Built for #HACKATHONSomosNLP 2026 by the Thermostatic team. Data: Thermostatic/rosettia-chanka-data. The judicial-manual source PDF is in the public domain.