xtremedistil-l6-h256-OCR-quality – AI Model by pszemraj | AlphaNeural AI

You can deploy this model and start earning money today!

pszemraj/
xtremedistil-l6-h256-OCR-quality

like

0

text-classification

generated_from_trainer

pszemraj/OCR-quality-classification

microsoft/xtremedistil-l6-h256-uncased

autotrain_compatible

endpoints_compatible

Views

No views yet

xtremedistil-l6-h256-uncased: OCR-quality-classification

This model is a fine-tuned version of microsoft/xtremedistil-l6-h256-uncased on pszemraj/OCR-quality-classification

It achieves the following results on the evaluation set:

Loss: 0.0316
Accuracy: 0.994
Num Input Tokens Seen: 57341952

Intended uses & limitations

predict whether a document is clean or noisy

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 3e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 64
optimizer: Adam with betas=(0.9,0.99) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.05
num_epochs: 2.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Input Tokens Seen
0.0812	0.2660	250	0.0860	0.986	8192000
0.0637	0.5321	500	0.0532	0.988	16384000
0.031	0.7981	750	0.0463	0.99	24576000
0.0315	1.0641	1000	0.0343	0.992	32765952
0.0223	1.3301	1250	0.0337	0.994	40957952
0.0137	1.5962	1500	0.0423	0.99	49149952
0.0186	1.8622	1750	0.0316	0.994	57341952

Framework versions

Transformers 4.40.2
Pytorch 2.2.0+cu121
Datasets 2.19.1
Tokenizers 0.19.1