layoutlmv3-classification-finetuned-no-pretrain
This model is a fine-tuned version of
microsoft/layoutlmv3-base on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0001
- Accuracy: 1.0
Model Description: Legal Document Classifier (LayoutLMv3)
Overview
This model is a fine-tuned version of microsoft/layoutlmv3-base designed for the automated classification of administrative and legal documents. It distinguishes between document types based on visual layout, textual content, and spatial structure. This model was developed to facilitate digital transformation pipelines, enabling the automated archiving of scanned records.
Intended Use
The model classifies documents into five standard legal categories:
- Birth Certificate: Official identification documents.
- Power of Attorney: Legal authorization and representation forms.
- Title Deed: Real estate ownership and warranty deeds.
- Affidavit of Heirship: Legal determination of inheritance.
- Other / Bill of Sale: Miscellaneous administrative records.
Intended Use:
This model is designed for private use.
Limitations:
- Synthetic Training Domain: The model was trained entirely on synthetic data. While it performs well on augmented test sets, it may exhibit reduced accuracy on real-world documents with extreme degradation or handwritten annotations that were not simulated during training.
- Language/Region Specificity: The visual layouts and terminology are specific to Syrian legal contexts and may not generalize to documents from other jurisdictions.
Training and evaluation data
Training Data:
To ensure privacy compliance, no real citizen data was used. The training dataset consists of synthetic documents generated using a custom Python pipeline (Faker + PIL). These documents mimic the authentic layout and typography of official forms but contain randomly generated information.
Augmentation (Sim-to-Real):
The training data was subjected to aggressive augmentation to simulate real-world scanning conditions, including:
- Gaussian Blur (simulating defocus).
- Random Rotation/Skew (simulating physical paper feed errors).
- Salt-and-Pepper Noise (simulating dust and scanner artifacts).
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 16
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 5
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|
| 0.2551 | 0.2 | 50 | 0.0022 | 1.0 |
| 0.0025 | 0.4 | 100 | 0.0009 | 1.0 |
| 0.0014 | 0.6 | 150 | 0.0006 | 1.0 |
| 0.001 | 0.8 | 200 | 0.0005 | 1.0 |
| 0.0008 | 1.0 | 250 | 0.0004 | 1.0 |
| 0.0006 | 1.2 | 300 | 0.0003 | 1.0 |
| 0.0005 | 1.4 | 350 | 0.0002 | 1.0 |
| 0.0004 | 1.6 | 400 | 0.0002 | 1.0 |
| 0.0004 | 1.8 | 450 | 0.0002 | 1.0 |
| 0.0003 | 2.0 | 500 | 0.0002 | 1.0 |
| 0.0003 | 2.2 | 550 | 0.0002 | 1.0 |
| 0.0003 | 2.4 | 600 | 0.0001 | 1.0 |
| 0.0002 | 2.6 | 650 | 0.0001 | 1.0 |
| 0.0002 | 2.8 | 700 | 0.0001 | 1.0 |
| 0.0002 | 3.0 | 750 | 0.0001 | 1.0 |
| 0.0002 | 3.2 | 800 | 0.0001 | 1.0 |
| 0.0002 | 3.4 | 850 | 0.0001 | 1.0 |
| 0.0002 | 3.6 | 900 | 0.0001 | 1.0 |
| 0.0002 | 3.8 | 950 | 0.0001 | 1.0 |
| 0.0002 | 4.0 | 1000 | 0.0001 | 1.0 |
| 0.0002 | 4.2 | 1050 | 0.0001 | 1.0 |
| 0.0002 | 4.4 | 1100 | 0.0001 | 1.0 |
| 0.0002 | 4.6 | 1150 | 0.0001 | 1.0 |
| 0.0001 | 4.8 | 1200 | 0.0001 | 1.0 |
| 0.0001 | 5.0 | 1250 | 0.0001 | 1.0 |
Framework versions
- Transformers 4.57.2
- Pytorch 2.9.0+cu126
- Datasets 4.0.0
- Tokenizers 0.22.1
