This distilbert model was fine-tuned on 50.000 stock news articles using the HuggingFace adapter from Kern AI refinery. The articles consisted of the headlines plus abstract of the article.
For the finetuning, a single NVidia K80 was used for about four hours.
Join our Discord if you have questions about this model:
https://discord.gg/MdZyqSxKbeDistilBERT is a smaller, faster and lighter version of BERT. It was trained by distilling BERT base and has 40% less parameters than bert-base-uncased.
It runs 60% faster while preserving over 95% of BERT’s performances as measured on the GLUE language understanding benchmark.
DistilBERT does not have token-type embeddings, pooler and retains only half of the layers from Google’s BERT.
1from transformers import AutoModelForSequenceClassification, AutoTokenizer
2
3model = AutoModelForSequenceClassification.from_pretrained("KernAI/stock-news-distilbert")
4tokenizer = AutoTokenizer.from_pretrained("KernAI/stock-news-distilbert")
To classify a single sentence or a sentence pair, you can use the HuggingFace Pipeline API:
1from transformers import pipeline
2
3classifier = pipeline("text-classification", model=model, tokenizer=tokenizer)
4result = classifier("This is a positive sentence.")
5print(result)
6# [{'label': 'POSITIVE', 'score': 0.9998656511306763}]
