In this repository we release (yet another) GPT-2 model, that was trained on various texts for German.
The model is meant to be an entry point for fine-tuning on other texts, and it is definitely not as good or "dangerous" as the English GPT-3 model. We do not plan extensive PR or staged releases for this model 😉
Note: The model was initially released under an anonymous alias (anonymous-german-nlp/german-gpt2) so we now "de-anonymize" it.
More details about GPT-2 can be found in the great Hugging Face documentation.
German GPT-2 fine-tuned on Faust I and II
We fine-tuned our German GPT-2 model on "Faust I and II" from Johann Wolfgang Goethe. These texts can be obtained from Deutsches Textarchiv (DTA). We use the "normalized" version of both texts (to avoid out-of-vocabulary problems with e.g. "ſ")
Fine-Tuning was done for 100 epochs, using a batch size of 4 with half precision on a RTX 3090. Total time was around 12 minutes (it is really fast!).
We also open source this fine-tuned model. Text can be generated with:
python
1from transformers import pipeline
23pipe = pipeline('text-generation', model="dbmdz/german-gpt2-faust",4 tokenizer="dbmdz/german-gpt2-faust")56text = pipe("Schon um die Liebe", max_length=100)[0]["generated_text"]78print(text)
and could output:
Schon um die Liebe bitte ich, Herr! Wer mag sich die dreifach Ermächtigen?
Sei mir ein Held!
Und daß die Stunde kommt spreche ich nicht aus.
Faust (schaudernd).
Den schönen Boten finde' ich verwirrend;
