Primary Application: Language classification for Cantonese and Traditional Chinese texts.
Users: NLP researchers, developers working with Chinese language data.

Training Data

Utilizes the "raptorkwok/cantonese-traditional-chinese-parallel-corpus" from Hugging Face Datasets.

Training Procedure

Base Model: bert-base-chinese
Epochs: 3
Learning Rate: 2e-5

How to Use

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained("ming030890/chinese-langid")
model = AutoModelForSequenceClassification.from_pretrained("ming030890/chinese-langid")
text = "係唔係廣東話？"
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)
# 0 for Cantonese, 1 for Traditional Chinese
prediction = outputs.logits.argmax(-1).item()