from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("emilcw/gpt-sw3-1.3b-da-saga-dpo", torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained("emilcw/gpt-sw3-1.3b-da-saga-dpo")

prompt = "Dansk er"
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=60, temperature=0.8, do_sample=True)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training details

Base model: GPT-SW3 1.3B (Nordic pretraining, base DA PS 86%)
Method: DPO
Oracle: SpaCy da_core_news_lg (Danish dependency parser)
LoRA: rank 16, α=32, all linear layers, bfloat16 (merged into full weights)

Citation

bibtex

@article{fakhar2025saga,
  title={SAGA: Syntax-Aware Grammar Alignment for Low-Resource Nordic Languages},
  author={Fakhar, Hoda and others},
  year={2025},
  note={Under review}
}

License

Inherits the AI Sweden LLM License from the base model.