from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

model_name = "xlm-r-hsd-span"
tok = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForTokenClassification.from_pretrained(model_name)
text = "Ví dụ câu tiếng Việt có nội dung thù ghét ..."
enc = tok(text, return_tensors="pt", truncation=True, max_length=256, is_split_into_words=False)
with torch.no_grad():
    logits = model(**enc).logits
    pred_ids = logits.argmax(-1)[0].tolist()
# TODO: chuyển pred_ids -> spans theo scheme nhãn của bạn (BIO/BILOU/char-offset)

License

Apache-2.0

Acknowledgments

Base model: xlm-r

visolex/xlm-r-hsd-span

xlm-r-hsd-span: Hate Speech Span Detection (Vietnamese)

Model Details

Hyperparameters

Results

Usage

License

Acknowledgments

visolex/
xlm-r-hsd-span