Model Card for seifelzeiny/finetuned-tinyllama-finmarba
Model Description
TinyLlama fine-tuned to answer financial news headlines sentiment Q&A, using the FinMarBa dataset.
This model predicts sentiment (Negative/Neutral) from financial headlines in English via causal language modeling.
Developed by: Seifeldin Elzeiny (@seifelzeiny)
Model type: Causal language model (LLM, LoRA/PEFT)
Research on financial text understanding by small language models
Downstream Use
Can be extended for broader financial NLP tasks:
Financial chatbot assistants
Automated news classification
Sentiment-based trading signals (research only)
Out-of-Scope Use
Do not use for medical, legal, or high-stakes banking advice.
Not suitable for real-time trading decisions without thorough validation.
May not generalize to non-financial or multi-lingual content.
Bias, Risks, and Limitations
Model trained only on English headlines, mostly US/EU market news.
May reflect dataset biases (e.g., over/under-representation of neutral or negative market moves).
Performance in non-news or conversational contexts is untested.
Recommendations
For academic or research use.
Validate outputs before applying to production finance systems.
Future work should expand to multi-class and multi-lingual sentiment coverage.
How to Get Started with the Model
Use the code below to get started with the model.
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("seifelzeiny/finetuned-tinyllama-finmarba")
tokenizer = AutoTokenizer.from_pretrained("seifelzeiny/finetuned-tinyllama-finmarba")
prompt = "### Question: What is the sentiment of this news?\nFederal Reserve raises rates.\n### Answer:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=32)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Model achieves near-equal training and validation loss, indicating good generalization without overfitting (see loss table above).
Direct accuracy scores are not used due to causal LM format, but manual validation shows plausible sentiment answers.
Environmental Impact
Hardware Type: Nvidia A10G GPU (Modal cloud), or local CUDA device
