The model corrects spelling and punctuation errors and typos by bringing all the words in the text to the norm of the Russian language.
Corrector is a distilled version of the original model that had been trained based on the
FRED-T5-1.7B architecture.
An extensive dataset with “artificial” errors was taken as a training corpus: the corpus was assembled on the basis of the Russian-language Wikipedia and transcripts of Russian-language videos, then typos and spelling errors were automatically introduced into it using the library
SAGE.
Below are automatic metrics for determining the correctness of the spell checkers.
We compare our solution with both open automatic spell checkers and the ChatGPT family of models on all four available datasets:
1from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
2
3tokenizer = AutoTokenizer.from_pretrained("ai-forever/sage-fredt5-distilled-95m")
4model = AutoModelForSeq2SeqLM.from_pretrained("ai-forever/sage-fredt5-distilled-95m")
5
6model.to("cuda")
7
8sentence = "И не чсно прохожим в этот день непогожйи почему я веселый такйо"
9inputs = tokenizer(sentence, max_length=None, padding="longest", truncation=False, return_tensors="pt")
10outputs = model.generate(**inputs.to(model.device), max_length = inputs["input_ids"].size(1) * 1.5)
11print(tokenizer.batch_decode(outputs, skip_special_tokens=True))
12
13# ["И не ясно прохожим в этот день непогожий, почему я весёлый такой?"]
14
Model
FRED-T5-1.7B, on the basis of which our solution is made, and its source code are supplied under the MIT license.
Our solution comes with MIT license also.
