RUKOPYS Qwen3-VL 8B Page LoRA
Page-level LoRA adapter for Qwen/Qwen3-VL-8B-Instruct, fine-tuned for Ukrainian handwritten document understanding.
This model takes a full scanned page as input and predicts structured JSON regions: bounding boxes, region type, language metadata, and transcribed text. It was built as part of an end-to-end HTR pipeline for the RUKOPYS handwritten document dataset: data curation, supervised fine-tuning dataset generation, QLoRA training, inference, evaluation, and Kaggle-style submission export.
What It Does
- Parses full-page Ukrainian handwritten documents.
- Returns structured region-level JSON instead of plain OCR text.
- Supports document AI workflows where layout and transcription must be evaluated together.
- Serves as the first public baseline LoRA adapter from this project.
Training Data
Trained on the curated RUKOPYS MVP dataset:
The dataset is a cleaned derivative of UkrainianCatholicUniversity/rukopys, prepared for page-level vision-language fine-tuning, crop-level transcription, and layout detection experiments.
Output Format
The expected output is JSON similar to:
1[
2 {
3 "bbox": [10, 20, 300, 80],
4 "type": "handwritten",
5 "language": "uk",
6 "text": "..."
7 }
8]
Notes
This is the original public 8B page-level adapter. For the improved checkpoint, use:
Limitations
Optimized for Ukrainian handwriting and RUKOPYS-style documents.
Sensitive to image quality, scan resolution, and page layout.
Should be validated on target documents before production use.
The adapter is Apache-2.0, but the training data derives from a CC BY-NC-SA 4.0 dataset.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 8
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- training_steps: 600
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|
| 0.4863 | 0.1907 | 50 | 0.5126 |
| 0.4714 | 0.3815 | 100 | 0.4933 |
| 0.4369 | 0.5722 | 150 | 0.4835 |
| 0.4201 | 0.7630 | 200 | 0.4751 |
| 0.4059 | 0.9537 | 250 | 0.4701 |
| 0.4296 | 1.1412 | 300 | 0.4664 |
| 0.4154 | 1.3319 | 350 | 0.4646 |
| 0.3869 | 1.5227 | 400 | 0.4618 |
| 0.3784 | 1.7134 | 450 | 0.4617 |
| 0.3631 | 1.9041 | 500 | 0.4578 |
| 0.3904 | 2.0916 | 550 | 0.4569 |
| 0.3874 | 2.2823 | 600 | 0.4558 |
Framework versions
- PEFT 0.19.1
- Transformers 5.0.0
- Pytorch 2.11.0+cu128
- Datasets 4.8.5
- Tokenizers 0.22.2
