The model achieves strong overall performance with a Macro-F1 of ~0.80 on the test set.
OTHER and NONE classes show the highest performance.
The AFTER class has lower precision but strong recall, indicating some over-prediction.
The drop from validation (0.83 F1) to test (0.79 F1) suggests a mild generalization gap, but overall stable performance.
Errors are mainly observed in:
- BEFORE vs AFTER confusion
- NONE vs OTHER boundary ambiguity

🚀 Usage

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_id = "AnirbanSaha/llama32-3b-tlink-full-finetune"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "The patient developed fever before taking the medication."

inputs = tokenizer(text, return_tensors="pt", truncation=True)

outputs = model(**inputs)
pred = outputs.logits.argmax(dim=-1).item()

labels = ["BEFORE", "AFTER", "OTHER", "NONE"]
print(labels[pred])

⚠️ Important Notice (LLaMA License)

This model is based on:

👉 meta-llama/Llama-3.2-3B-Instruct

You must request access and accept the license:

https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct

Otherwise, loading this model will fail.

📊 Training Logs

Weights & Biases run: 👉 https://wandb.ai/anirbansaha002-univeristy-of-north-texas/llama32-3b-full-finetune/runs/mrdwp04g

🧩 Limitations

Performance depends on dataset distribution
May not generalize well to unseen domains
Sensitive to long input truncation
Some confusion between temporally similar classes (BEFORE vs AFTER)

🔬 Research Context

This model was developed as part of research on:

Temporal reasoning in NLP
Relation classification using large language models

📬 Contact

Author: Anirban Saha Anik
Affiliation: University of North Texas

⭐ Citation

If you use this model in your research, please cite appropriately.