RoQLlama is a new lightweight Romanian language-adapted LLM with 7 billion parameters and quantized to 4 bits by employing the state-of-the-art quantized LoRA (QLoRA) training technique.

Language: Romanian
License: Llama2 Community License Agreement
Finetuned from model: Meta's Llama2 7B

Model Sources

Paper: https://arxiv.org/abs/2410.04269

How to Get Started with the Model

python

from transformers import AutoTokenizer, AutoModelForCausalLM
MODEL_NAME = "andreidima/Llama-2-7b-Romanian-qlora"
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, device_map="auto")
model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto")

input_text = """Eu răspund la întrebări pe baza contextului.
Context: În anul 1600, Mihai Viteazul a realizat prima unire a Țărilor Române: Țara Românească, Transilvania și Moldova. Această unire a fost un moment important în istoria României.
Întrebare: În ce an a realizat Mihai Viteazul prima unire a Țărilor Române?
Răspuns: """
input_ids = tokenizer(input_text, return_tensors="pt")

outputs = model.generate(
    **input_ids, 
    max_new_tokens=100, 
    eos_token_id=[13] # 13 is the token ID for a newline character at the end of a non-empty line
)
print(tokenizer.decode(outputs[0]))

Note: Adding a space at the end of the prompt has been observed to significantly improve the model's output quality.

Training Details and Evaluation

Please refer to the paper for details on the model's training and evaluation.

Citation

BibTeX:

@inproceedings{dima2024roqllama,
      title={RoQLlama: A Lightweight Romanian Adapted Language Model}, 
      author={George-Andrei Dima and Andrei-Marius Avram and Cristian-George Crăciun and Dumitru-Clementin Cercel},
      booktitle={Findings of the Association for Computational Linguistics: EMNLP 2024}
      year={2024},
      url={https://arxiv.org/abs/2410.04269}, 
}

APA:

Dima, G. A., Avram, A. M., Crăciun, C. G., & Cercel, D. C. (2024). RoQLlama: A lightweight Romanian adapted language model. In Findings of the Association for Computational Linguistics: EMNLP 2024.