Views
No views yet
distilbert-base-uncased| Epoch | Training Loss | Validation Loss |
|---|---|---|
| 1 | 0.1200 | 0.0857 |
| 2 | 0.0322 | 0.0258 |
| 3 | 0.0165 | 0.0129 |
| 4 | 0.0335 | 0.0084 |
| 5 | 0.0079 | 0.0067 |
| 6 | 0.0066 | 0.0056 |
| 7 | 0.0311 | 0.0048 |
| 8 | 0.0523 | 0.0045 |
| 9 | 0.0051 | 0.0044 |
| 10 | 0.0278 | 0.0043 |
1- Epochs: 10
2- Batch Size: 8
3- Evaluation Strategy: Per epoch
4- Optimizer: AdamW (default)
5- Max Sequence Length: 128
6- Device: GPU (Tesla T4)1from transformers import AutoTokenizer, AutoModelForSequenceClassification
2import torch
3
4# Load model and tokenizer
5model = AutoModelForSequenceClassification.from_pretrained("./cbt_model_final")
6tokenizer = AutoTokenizer.from_pretrained("./cbt_model_final")
7
8# Load label mappings
9import json
10with open("./cbt_model_final/label_config.json", "r") as f:
11 label_config = json.load(f)
12
13id2label = label_config["id2label"]1def predict_distortions(text, threshold=0.5):
2 # Tokenize input
3 inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
4
5 # Get predictions
6 with torch.no_grad():
7 outputs = model(**inputs)
8 probabilities = torch.sigmoid(outputs.logits).squeeze()
9
10 # Extract distortions above threshold
11 detected = []
12 for idx, prob in enumerate(probabilities):
13 if prob > threshold:
14 label = id2label[str(idx)]
15 detected.append({
16 "distortion": label,
17 "confidence": f"{prob.item():.2%}"
18 })
19
20 return detected
21
22# Example usage
23text = "I always mess everything up. This is a disaster!"
24distortions = predict_distortions(text)
25
26for d in distortions:
27 print(f"{d['distortion']}: {d['confidence']}")Input: "I always mess everything up. This is a disaster!"
Detected distortions:
overgeneralization: 73.45%
catastrophizing: 68.92%cbt_model_final/
├── config.json # Model configuration
├── model.safetensors # Model weights
├── tokenizer_config.json # Tokenizer configuration
├── vocab.txt # Vocabulary
├── special_tokens_map.json # Special tokens
├── tokenizer.json # Tokenizer data
└── label_config.json # Label mappingsBase Model: DistilBERT
Original Paper: Sanh et al. (2019) - DistilBERT, a distilled version of BERT
Fine-tuning: Custom CBT distortion detection