This model is fine-tuned on the task of text correction: it fixes English text with the wrong grammar, punctuation and order of words. Significantly larger model that supports Russian language as well can be found here.

Training and evaluation data

Around 10% of the data from this dataset was used.

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 100
num_epochs: 1

Framework versions

Transformers 4.41.0
Pytorch 2.2.2
Datasets 2.19.1
Tokenizers 0.19.1

gudleifrr/text-correction-en-small

text-correction-en-small

Model description

Training and evaluation data

Training hyperparameters

Framework versions

gudleifrr/
text-correction-en-small