CryptoBERT was trained on retail social media posts (StockTwits, Twitter, Reddit), which gives it strong crypto-specific vocabulary but a poor match for journalistic headline text. Zero-shot, it scores 0.39 macro-F1 on Bitcoin headlines, close to random for a 3-class task. This model fine-tunes CryptoBERT on ~41,400 headlines relabeled with a directional market-impact rubric, reaching 0.78 macro-F1 on an independent held-out set.

Developed by: Hiyaw Ertiro
Model type: BERT-based sequence classification (3-class)
Language: English
License: MIT
Finetuned from model: ElKulako/cryptobert

Labels

ID	Label
0	Bearish (negative)
1	Neutral
2	Bullish (positive)

Labels are directional (does the event plausibly move price), not tonal (does the headline read positive/negative). See the dataset card for the full labeling rubric.

Direct Use

Sentiment scoring of English-language Bitcoin news headlines, as a feature for downstream pipelines (price prediction, trading signal aggregation, market monitoring dashboards).

How to Get Started

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

model = AutoModelForSequenceClassification.from_pretrained("hiyawtaken/cryptobert-finetuned_on_news_headlines")
tokenizer = AutoTokenizer.from_pretrained("hiyawtaken/cryptobert-finetuned_on_news_headlines")

headline = "Bitcoin surges to new all-time high as ETF inflows accelerate"
inputs = tokenizer(headline, return_tensors="pt", truncation=True, max_length=128)
with torch.no_grad():
    logits = model(**inputs).logits
pred = torch.argmax(logits, dim=-1).item()
print(model.config.id2label[pred])

Training Details

Training Data

labeled-bitcoin-news-headlines, built on top of Bitcoin News Sentiment Dataset by filipemunizz (2011-2024) + headlines pulled from google news(2024-July 2026). The original tone-based labels were relabeled with DeepSeek V4 Flash using an explicit directional rubric, with noise removed (ads, listicles, evergreen explainers, non-English headlines, generic roundups). ~41,400 real headlines plus ~500 synthetic headlines (fictional companies, clearly disclosed as synthetic) added to correct two specific model failure patterns found during validation: reasoning about transaction magnitude ($2 vs. $2M) and disambiguating near-identical dollar figures referring to different underlying events.

Training Procedure

Full fine-tune (all parameters updated, not a frozen-encoder head-only tune), warm-started from CryptoBERT's existing 3-class head.

Training Hyperparameters

Base model: ElKulako/cryptobert
Epochs: 3
Batch size: 32
Learning rate: Dynamic
Training regime: fp16 mixed precision
Metric for best checkpoint: macro-F1 (not accuracy, due to class imbalance)

Testing Data

An independent, hand-labeled set of 72 live-scraped 2026 Bitcoin headlines, never seen during labeling or training.

Metrics

Macro-F1 was used instead of accuracy because the classes are imbalanced (neutral is the largest class), and accuracy alone would reward a model that defaults to the majority class.

Results

Model	Macro-F1
CryptoBERT (zero-shot)	0.39
This model (fine-tuned)	0.78

Class	Precision	Recall	F1
Bearish	0.85	0.73	0.79
Neutral	0.80	0.69	0.74
Bullish	0.74	0.89	0.81

Summary

Fine-tuning improves macro-F1 by +38.5 points over zero-shot CryptoBERT on headline-length text, confirming that CryptoBERT's social-media training data transfers poorly to journalistic headlines without adaptation.

Bias, Risks, and Limitations

Training labels are LLM-generated (DeepSeek V4 Flash), not human-annotated. Treat as strong weak supervision, not ground truth.
The neutral/positive boundary is inherently fuzzy for headlines with implied but unstated direction (price predictions, analyst commentary). Some disagreement with human judgment is expected.
The model has limited ability to reason about transaction magnitude and to disambiguate near-identical numeric details referring to different underlying events, even after targeted synthetic data augmentation. This is a known limitation of a 110M-parameter text classifier rather than something expected to fully resolve with more data of the same kind.

Recommendations

Use as one signal among several in a larger pipeline, not as a standalone decision-maker. Review model predictions on headlines involving specific dollar amounts or company financial actions with extra scrutiny given the documented magnitude-reasoning limitation.

Model Architecture and Objective

BERT-based (RoBERTa-family) sequence classification, 3-class softmax output.

Hardware

NVIDIA RTX 3090 (24GB VRAM)

Software

PyTorch, Hugging Face Transformers, Datasets

Model Card Contact

Hiyaw Ertiro — https://huggingface.co/datasets/HiyawErtiro/bitcoin-news-sentiments-latest

HiyawErtiro/cryptobert-finetuned_on_news_headlines

Update

CryptoBERT Fine-Tuned on Bitcoin News Headlines

Model Description