This model is based on microsoft/deberta-v3-large and has been fine-tuned on the LIAR dataset for fact-checking tasks. It classifies statements into 6 truthfulness categories.

Performance

Validation Accuracy: 40.81%
Test Accuracy: 37.57%
F1 Score (macro): 36.89%

Labels

The model predicts one of six truthfulness labels:

true (0): The statement is accurate
mostly-true (1): The statement is mostly accurate
half-true (2): The statement has some truth but is incomplete/misleading
barely-true (3): The statement has minimal truth
false (4): The statement is inaccurate
pants-fire (5): The statement is completely false and ridiculous

Usage

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("Arko007/fact-check-v1")
model = AutoModelForSequenceClassification.from_pretrained("Arko007/fact-check-v1")

# Example usage
text = "The economy is doing great under this administration"
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=384)

with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.softmax(outputs.logits, dim=-1)
    predicted_class = torch.argmax(predictions, dim=-1)

# Map prediction to label
labels = ["true", "mostly-true", "half-true", "barely-true", "false", "pants-fire"]
print(f"Prediction: {labels[predicted_class.item()]}")
print(f"Confidence: {predictions[0][predicted_class].item():.4f}")

Training Details

Training Data

Dataset: LIAR dataset
Training samples: 10,240
Validation samples: 1,284
Test samples: 1,267

Training Configuration

Base Model: microsoft/deberta-v3-large (435M parameters)
Hardware: NVIDIA A100 80GB
Training Time: 7 minutes 21 seconds
Batch Size: 64
Learning Rate: 1e-5
Epochs: 4
Optimizer: AdamW with cosine scheduling
Class Weighting: Balanced for imbalanced dataset

Features Used

The model was trained with enhanced features including:

Original statement text
Speaker information and credibility scores
Political party affiliation
Historical claim statistics
Context and subject matter

Limitations

The model was trained specifically on political statements and may not generalize well to other domains
Performance is limited by the inherent difficulty of the 6-class fact-checking task
May exhibit bias present in the training data
Should not be used as the sole source for fact-checking decisions

Citation

If you use this model, please cite:

bibtex

@misc{fact-check-v1,
  title={Fact-Check Model v1: DeBERTa-based Fake News Detection},
  author={Arko007},
  year={2025},
  url={https://huggingface.co/Arko007/fact-check-v1}
}

License

Apache 2.0