This model is a fine-tuned version of
gpt2 on the writings of W. E. Burghardt Du Bois.
The model is designed to be finned tunning with writting from Historical black black writers who wrote on freedom and emancipation.
This first version has GPT2 fintunned with the writings of W. E. Burghardt Du Bois.
This can be used to complete sentences where historical context advocating for black freedom and emancipation is required.
The data used in the training consist of the writings of W. E. Burghardt Du Bois. The DarkWater written by Du Bois was downloaded from project Gutenberg using the link
https://www.gutenberg.org/files/15210/15210-h/15210-h.htm
Specifiically, the chapters used are below THE SHADOW OF the YEAR(12,515 word token), Litany at Atlanta(6,378 word token), THE SOULS OF WHITE FOLK(7301 word token), The Riddle of the Sphinx, THE HANDS OF ETHIOPIA(6378 word Token), The Princess of the Hither Isles(1508 word Token) OF WORK AND WEALTH(7301 word token), Second Coming(1033 word Token), THE SERVANT IN THE HOUSE(6508 word Token), Jesus Christ in Texas(3372 word Token), OF THE RULING OF MEN(7096 word Token),
The Call and THE DAMNATION OF WOMEN(6508 word Token). About 50,000 word token was used in the training.
After corpus was put together, the text was preprocessed to remove extra text and license information added by Gutenberg organization. Also the word token was kept below 50,000 words so that it could be trained on basic package provided by Google Colab.
It was then tokenized using GPT2Tokenizer and afterwards finned tunned on GPT2.
