Views
No views yet
| Level | Description | Recall |
|---|---|---|
| A1 | Beginner | 96.6% |
| A2 | Elementary | 90.0% |
| B1 | Intermediate | 90.0% |
| B2 | Upper-Intermediate | 86.7% |
| C1 | Advanced | 86.7% |
| C2 | Mastery | 60.0% |
Note: C2/C1 confusion is expected — the boundary between mastery and advanced is inherently subtle, even for human annotators.
1from transformers import AutoModelForSequenceClassification, AutoTokenizer
2import torch
3import torch.nn.functional as F
4
5model_id = "yanou16/cefr-english-classifier"
6
7tokenizer = AutoTokenizer.from_pretrained(model_id)
8model = AutoModelForSequenceClassification.from_pretrained(model_id)
9model.eval()
10
11LABELS = ["A1", "A2", "B1", "B2", "C1", "C2"]
12
13def predict(text: str) -> dict:
14 inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256)
15 with torch.no_grad():
16 logits = model(**inputs).logits
17 probs = F.softmax(logits, dim=-1)[0].tolist()
18 pred_idx = logits.argmax().item()
19 return {
20 "level": LABELS[pred_idx],
21 "confidence": round(probs[pred_idx], 4),
22 "probabilities": {LABELS[i]: round(p, 4) for i, p in enumerate(probs)},
23 }
24
25# Examples
26print(predict("I have dog. I like it very much."))
27# → {"level": "A1", "confidence": 0.97, ...}
28
29print(predict("Despite the challenging circumstances, she managed to articulate her concerns with remarkable clarity."))
30# → {"level": "C1", "confidence": 0.89, ...}| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen2.5-1.5B |
| Method | QLoRA (4-bit NF4) |
| LoRA rank | 32 |
| LoRA alpha | 64 |
| Target modules | q_proj, v_proj |
| Epochs | 5 |
| Learning rate | 2e-4 |
| Scheduler | Cosine |
| Batch size | 8 |
| Max length | 256 tokens |
| Training samples | 1,605 |
| Test samples | 179 |