This model is a fine-tuned version of
distilbert/distilroberta-base, trained on a
financial news headlines dataset from Kaggle. The headlines were assigned a classification based on the movement of the S&P 500 index on the next trading day following the publication of each headline. This movement classification was determined using S&P 500 closing prices obtained separately from Yahoo Finance and is defined as follows:
This model is based on the
distilroberta-base architecture, a distilled version of
RoBERTa-base. DistilRoBERTa retains 97% of the language understanding capabilities of RoBERTa-base while being twice as fast. It has 6 layers, 768 dimensions, and 12 attention heads, totaling 82 million parameters (compared to 125 million for RoBERTa-base). The model is case-sensitive and distinguishes between different cases in text.
The goal of this model is to predict the movement of the S&P 500 index based on financial news headlines. However, this initial version is not intended for production use. While it may provide insights into the potential impact of news on market movements, fine-tuning on more specific and extensive datasets will be necessary to improve its accuracy and reliability.
The training data consists of financial news headlines sourced from Kaggle. The headlines were classified based on the movement of the S&P 500 index on the following trading day, with closing prices gathered separately from Yahoo Finance. Movements were classified into three categories: increase, decrease, or maintain, based on the criteria mentioned above.
