A Maltese-specialised model built from utter-project/EuroLLM-1.7B in two stages on a single consumer GPU (RTX 5060 Ti, 16 GB).

It is fluent in Maltese but hallucinates and does not always reply with anything related to the prompt. It's able to translate english sentences into a related sentence in Maltese. It's quite bad at it but the Maltese translation is related somehow.

The prompt template and sampling parameters are stored in the repo, so nothing else is needed. Pick a build with a tag:

There is no chat template. Use the Mistoqsija:/Tweġiba: framing above, which is what the model was trained on. A low temperature helps: what factual knowledge it has is held with very little margin and does not survive sampling at 0.7.

Both stages were full fine-tunes (all 1.657B parameters), bf16 weights and gradients with 8-bit AdamW and gradient checkpointing, ~14 GB VRAM.

The model was trained on this exact framing, with loss computed only on the answer:

Mistoqsija: {question}
Tweġiba: {answer}

35% of training questions had their diacritics stripped (ħ→h, ġ→g, ċ→c, ż→z, so għ→gh) while every answer was left correct. Because loss lands only on the answer, the model is shown informal spelling and never rewarded for producing it.

Q: X'inhu n-numru medju ta' sighat ta' rqad li persuna ghandha tiehu?
A: In-numru medju ta' sigħat ta' rqad li persuna għandha tieħu huwa madwar
   7.5 sigħat kuljum.

Note sighat → sigħat, ghandha → għandha, tiehu → tieħu.

Perplexity on held-out documents, by register, against the untrained base:

A 57-item Malta factual eval (greedy decoding, an upper bound on knowledge):

Do whatever you want with it