You can deploy this model and start earning money today!

LumiOpen/
llama-hpltv2-edu-classifier-xlm-roberta-large-kat-Geor

Name: llama-hpltv2-edu-classifier-xlm-roberta-large-kat-Geor
Availability: InStock

safetensors

xlm-roberta

kat

LumiOpen/hpltv2-llama33-edu-annotation

apache-2.0

Views

No views yet

Llama-HPLT-edu-Georgian classifier

Model summary

This is a classifier for judging the educational content of Georgian (kat-Geor) web pages. It was developed to filter educational content from HPLT v2 and was trained on 450k annotations generated by LLama3.3-70B-instruct. The web pages were sampled randomly from Georgian subset of the corpus.

How to load in transformers

To load the Llama-HPLT-Edu classifier, use the following code:

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("LumiOpen/llama-hpltv2-edu-classifier-xlm-roberta-large-kat-Geor")
model = AutoModelForSequenceClassification.from_pretrained("LumiOpen/llama-hpltv2-edu-classifier-xlm-roberta-large-kat-Geor")
text = "I'm non-educational web page containing nothing useful"
inputs = tokenizer(text, return_tensors="pt", padding="longest", truncation=True)
outputs = model(**inputs)
logits = outputs.logits.squeeze(-1).float().detach().numpy()
score = logits.item()
result = {
    "text": text,
    "score": score,
    "int_score": int(round(max(0, min(score, 5)))),
}
print(result)
#results from a model trained with Welsh annotations
#{'text': "I'm non-educational web page containing nothing useful", 'score': 0.8145455718040466, 'int_score': 1}
#{'text': 'what are most common animals found in farm? there are cows, sheeps', 'score': 1.6858888864517212, 'int_score': 2}

Training

Model: FacebookAI/xlm-roberta-large with a classification head
Dataset: 500,000 samples from Llama3.3 annotations split into 450,000 train, 25,000 validation, and 25,000 test splits.
Epochs: 20
Learning Rate: 3e-4
Evaluation Metric: F1 score

Test Metrics


              precision    recall  f1-score   support

           0       0.75      0.56      0.64       7293
           1       0.63      0.71      0.67       10553
           2       0.42      0.57      0.49       3726
           3       0.42      0.46      0.44       2063
           4       0.68      0.25      0.37       1304
           5       0.05      0.02      0.03       61

    accuracy                           0.60      25000
   macro avg       0.49      0.43      0.44      25000
weighted avg       0.62      0.60      0.60      25000

Citing

Preprint coming soon. If you need to cite this work, please use the citation below:

@misc {llama_hplt_edu_classifiers_2025,
    author       = { Tarkka, Otto, Reunamo, Akseli, Vitiugin, Fedor and Pyysalo, Sampo }
    title        = { Llama-HPLT-edu classifiers },
    year         = 2025,
    url          = {https://huggingface.co/collections/LumiOpen/hplt-edu-classifiers-68a85a78f9710426320e7cbb},
    publisher    = { Hugging Face }
}