Llama3-8B fine-tuned on financial sentiment (100% agreement threshold). This model was fine-tuned using LoRA (Low-Rank Adaptation) on the Financial PhraseBank dataset with 100% annotator agreement threshold.

Model Details

Base Model: meta-llama/Meta-Llama-3-8B
Fine-tuning Method: LoRA (Low-Rank Adaptation)
Dataset: Financial PhraseBank (sentences with 100% annotator agreement)
Task: Financial Sentiment Analysis (3-class: positive, negative, neutral)
Language: English

Performance

Metric	Value
Accuracy	83.0%
F1 Score	82.5%
Precision	83.5%
Recall	82.0%

Training Details

This model was fine-tuned as part of a Final Year Project on Financial Sentiment Analysis and Stock Prediction. The training used:

Training Framework: Transformers + PEFT
Quantization: 4-bit quantization using BitsAndBytes
Hardware: CUDA-enabled GPU
Hyperparameter Optimization: Extensive Optuna-based tuning

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Meta-Llama-3-8B",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B")

# Load fine-tuned model
model = PeftModel.from_pretrained(base_model, "jengyang/trained-llama3-sentences_allagree-financial-sentiment")

# Prepare input
text = "The company reported strong quarterly earnings, exceeding analyst expectations."
prompt = f"Classify the sentiment of this financial text as positive, negative, or neutral: {text}\n\nSentiment:"

# Tokenize and generate
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=10,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Training Data

The model was trained on the Financial PhraseBank dataset, specifically using sentences where 100% of annotators agreed on the sentiment label. This ensures higher quality and consistency in the training data.

The Financial PhraseBank contains financial news headlines categorized into:

Positive: Favorable financial news
Negative: Unfavorable financial news
Neutral: Factual financial information without clear sentiment

Evaluation

The model was evaluated on a held-out test set from the Financial PhraseBank dataset. The evaluation metrics reflect performance on financial sentiment classification with the 100% agreement threshold.

Note: This Llama3 model was fine-tuned with optimized hyperparameters determined through extensive Optuna studies.

Limitations and Bias

The model is specifically designed for financial text sentiment analysis
Performance may vary on non-financial text or different domains
The model reflects the biases present in the Financial PhraseBank dataset
Results should be interpreted within the context of financial sentiment analysis
The model may not capture nuanced sentiment in complex financial scenarios

Intended Use

Intended Use Cases:

Financial news sentiment analysis
Investment research and analysis
Automated financial content classification
Academic research in financial NLP

Out-of-Scope Use Cases:

General-purpose sentiment analysis
Medical or legal text analysis
Real-time trading decisions without human oversight

Citation

If you use this model, please cite:

bibtex

@misc{trained_llama3_sentences_allagree,
  title={Trained Llama3 Sentences_Allagree: Fine-tuned Meta-Llama-3-8B for Financial Sentiment Analysis},
  author={Final Year Project},
  year={2024},
  howpublished={\url{https://huggingface.co/jengyang/trained-llama3-sentences_allagree-financial-sentiment}}
}

Model Card Authors

This model card was generated as part of a Final Year Project on Financial Sentiment Analysis and Stock Prediction.