ruGPT-Neo 1.3B is a transformer model designed using EleutherAI's replication of the GPT-3 architecture. ruGPT-Neo refers to the class of models, while 1.3B represents the number of parameters of this particular pre-trained model.
This model was trained on the wiki, gazeta summorization, for 38k steps, on 1*v100 gpu, still training . It was trained as a masked autoregressive language model, using cross-entropy loss.
You can use this model directly with a pipeline for text generation. This example generates a different sequence each time it's run:
1>>> from transformers import pipeline
2>>> generator = pipeline('text-generation', model='AlexWortega/rugpt-neo-1.3b')
3>>> generator("Как какать? Ответ:", do_sample=True, min_length=50)
4
5[{'generated_text': 'Как какать? Ответ: Cпустите штаны и покакайте, затем воспользуйтесь бумагой'}]
