You can deploy this model and start earning money today!

NYTK/
PULI-LlumiX-Llama-3.1

Name: PULI-LlumiX-Llama-3.1
Availability: InStock

Views

No views yet

PULI-LlumiX-Llama-3.1 8B base (8.03B billion parameter)

Trained with LLaMA-Factory github
The Llama 3.1 8B Instruct model were continual pretrained on Hungarian dataset

Dataset for continued pretraining

Hungarian (8.7 billion words): documents (763K) that exceed 5000 words in length + Hungarian Wikipedia and news
English: Long Context QA (1 billion words), BookSum (42 million words)

Limitations

max_seq_length = 16 384
bfloat16

Usage with pipeline

python

from transformers import pipeline, LlamaForCausalLM, AutoTokenizer

model = LlamaForCausalLM.from_pretrained("NYTK/PULI-LlumiX-Llama-3.1")
tokenizer = AutoTokenizer.from_pretrained("NYTK/PULI-LlumiX-Llama-3.1")
prompt = "Elmesélek egy történetet a nyelvtechnológiáról."
generator = pipeline(task="text-generation", model=model, tokenizer=tokenizer, device=0)

print(generator(prompt, max_new_tokens=30)[0]["generated_text"])

Since the model was continuously pre-trained from Llama 3.1 8B Instruct, it can be used as a chat model.

python

import torch
from transformers import pipeline

model_id = "NYTK/PULI-LlumiX-Llama-3.1"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
messages = [
    {"role": "system", "content": "You are a helpful assistant"},
    {"role": "user", "content": "Mit gondolsz a nyelvtechnológiáról?"},
]
outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

Citation

If you use this model, please cite the following paper:

@article{chatpuli,
  title={ChatPULI: Enhancement to the first Hungarian conversational model},
  author={Yang, Zijian Győző and Bánfi, Ágnes and Dodé, Réka and Ferenczi, Gergő and Földesi, Flóra and Hatvani, Péter and Héja, Enikő and Lengyel, Mariann and Madarász, Gábor and Osváth, Mátyás and Sárossy, Bence and  Varga, Kristóf and  Váradi, Tamás and  Prószéky, Gábor and  Ligeti-Nagy, Noémi},
  journal={Annales Mathematicae et Informaticae},
  doi = {https://doi.org/10.33039/ami.2025.10.010},
  url = { https://ami.uni-eszterhazy.hu},
  year={2025},
  volume={61},
  pages={261–-274}
}