Views
No views yet

1from transformers import pipeline
2
3# Load model
4classifier = pipeline('text-classification', model='tabularisai/ModernFinBERT')
5
6# Test sentences
7sentences = [
8 "The company reported strong quarterly earnings with revenue growth of 15% year-over-year, exceeding analyst expectations.",
9 "Due to rising inflation and supply chain disruptions, the Federal Reserve decided to increase interest rates by 0.75 basis points.",
10 "The merger between the two pharmaceutical giants is expected to create significant synergies and reduce operational costs by $2 billion annually."
11]
12
13# Evaluate
14for i, sentence in enumerate(sentences, 1):
15 result = classifier(sentence)
16 print(f"Sentence {i}: {result[0]['label']} ({result[0]['score']:.3f})")| Dataset | Model | Accuracy | F1-Score | Precision | Recall | ROC-AUC |
|---|---|---|---|---|---|---|
| FIQA | ModernFinBERT | 0.80 | 0.61 | 0.64 | 0.88 | 0.96 |
| FIQA | distilroberta_financial | 0.54 | 0.47 | 0.61 | 0.71 | 0.71 |
| FIQA | finbert | 0.48 | 0.43 | 0.59 | 0.66 | 0.76 |
| FIQA | finbert-tone | 0.36 | 0.36 | 0.62 | 0.58 | 0.77 |
| FIQA | roberta_sentiment | 0.36 | 0.35 | 0.60 | 0.58 | 0.89 |
| ModernFinBERT | 0.71 | 0.70 | 0.68 | 0.81 | 0.94 | |
| distilroberta_financial | 0.75 | 0.71 | 0.68 | 0.75 | 0.87 | |
| finbert-tone | 0.75 | 0.66 | 0.68 | 0.64 | 0.83 | |
| finbert | 0.73 | 0.67 | 0.65 | 0.70 | 0.86 | |
| roberta_sentiment | 0.70 | 0.61 | 0.63 | 0.60 | 0.82 | |
| JeanBaptiste | ModernFinBERT | 0.74 | 0.58 | 0.71 | 0.56 | 0.84 |
| JeanBaptiste | distilroberta_financial | 0.88 | 0.79 | 0.92 | 0.74 | 0.86 |
| JeanBaptiste | finbert | 0.77 | 0.68 | 0.70 | 0.67 | 0.88 |
| JeanBaptiste | finbert-tone | 0.74 | 0.60 | 0.72 | 0.56 | 0.86 |
| JeanBaptiste | roberta_sentiment | 0.70 | 0.55 | 0.79 | 0.51 | 0.83 |
| Model | Accuracy | F1-Score | Precision | Recall | ROC-AUC |
|---|---|---|---|---|---|
| ModernFinBERT | 0.75 | 0.63 | 0.68 | 0.75 | 0.91 |
| distilroberta_financial | 0.73 | 0.66 | 0.73 | 0.73 | 0.82 |
| finbert | 0.66 | 0.59 | 0.65 | 0.68 | 0.84 |
| finbert-tone | 0.62 | 0.54 | 0.68 | 0.59 | 0.82 |
| roberta_sentiment | 0.59 | 0.50 | 0.67 | 0.56 | 0.84 |