This is a ruRoberta-large model trained on the RuCoLa dataset. It can be used to classify Russian sentences into fluent or non-fluent ones, where fluency is understood as linguistic acceptability.
Training notebook: task_oriented_TST/fluency/rucola_classifier_v1.ipynb (in a private repo).
Training parameters:
optimizer: Adam
lr=2e-6
batch_size=32
epochs=10
clip_grad_norm=1.0
Test accuracy (on the leaderboard this model is submitted as ruroberta-base-cased-rucola-v1): 0.81.
