Qwen3-14B-sk Alpaca Slovak IT is an instruction-tuned Slovak assistant model
derived from slovak-nlp/Qwen3-14B-sk. It was trained with a LoRA supervised
fine-tuning recipe on Slovak Alpaca-style instruction data and then merged back
into the base model, so the released artifact can be loaded directly with
transformers.
The model is designed to answer in Slovak by default while still attempting to
follow explicit user requests for another language or a specific output format.
Intended Use
This model is intended for Slovak instruction following, Slovak question
answering, drafting, rewriting, summarization-style prompts, and general
assistant workflows where Slovak is the primary language.
It can also respond to English prompts and translation-style requests, but
language control is not guaranteed. Use additional application-level checks for
strict formatting, policy compliance, or high-reliability translation.
Do not use this model as the sole source for medical, legal, financial, safety,
or other high-stakes decisions. It has not been safety aligned, red-teamed, or
moderated for production deployment.
Dataset preparation converted each example into a chat-style conversation with
system, user, and assistant messages. Empty instruction/output examples were
excluded, and duplicate instruction/input/output triples were removed across
the prepared splits.
Split
Rows
Train
41,601
Held-out
10,401
The data preparation used dataset revision
058172466eb1d6a28b161f29c74350911d154161.
Each training conversation used this system prompt:
Si užitočný asistent. Riaď sa jazykom a požadovaným formátom používateľa. Ak používateľ nežiada iný jazyk, odpovedaj po slovensky.
Training Recipe
The model was trained with supervised fine-tuning using PEFT LoRA. Only the
assistant turns were included in the training loss.
Setting
Value
LoRA rank / alpha / dropout
32 / 64 / 0.05
Target modules
linear projection modules
Sequence length
4096
Sample packing
enabled
Epochs
1
Effective batch size
16
Micro batch size
1
Gradient accumulation
16
Learning rate
1e-4
Scheduler
cosine
Warmup ratio
0.06
Optimizer
fused AdamW
Precision
bfloat16
Gradient checkpointing
enabled
Seed
42
After training, the LoRA adapter was merged into the base model with PEFT and
saved as a standalone safetensors checkpoint.
Usage
python
1import torch
2from transformers import AutoModelForCausalLM, AutoTokenizer
34model_id ="mrshu/qwen3-14b-sk-alpaca-slovak-it"56tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)7model = AutoModelForCausalLM.from_pretrained(8 model_id,9 torch_dtype=torch.bfloat16,10 device_map="auto",11 trust_remote_code=True,12)1314messages =[15{16"role":"system",17"content":(18"Si užitočný asistent. Riaď sa jazykom a požadovaným formátom "19"používateľa. Ak používateľ nežiada iný jazyk, odpovedaj po slovensky."20),21},22{"role":"user","content":"Stručne vysvetli, čo je LoRA."},23]2425inputs = tokenizer.apply_chat_template(26 messages,27 add_generation_prompt=True,28 return_tensors="pt",29).to(model.device)3031outputs = model.generate(32 inputs,33 max_new_tokens=512,34 do_sample=True,35 temperature=0.6,36 top_p=0.95,37)3839print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
Limitations
The supervised fine-tuning data is translated instruction data, so the model
may inherit translation artifacts, unnatural phrasing, or source-dataset
biases.
The model is biased toward Slovak responses. Explicit requests for another
language may work, but they are not guaranteed.
Strict JSON, exact labels, citations, and other constrained formats should be
validated outside the model.
The model may hallucinate facts, produce unsafe content, or follow malicious
instructions without additional safeguards.
