WriteViT is a one-shot handwritten text synthesis framework that learns a writer's style from a small set of reference images and generates handwritten text in that style. The model combines a ViT-based writer encoder, a multi-scale Transformer generator with conditional positional encoding, and a lightweight ViT recognizer.

arXiv

Open in Colab

License

Model Summary

Field	Description
Task	Handwritten text image generation
Framework	PyTorch
Architecture	ViT writer encoder, Transformer generator, ViT recognizer
Supported Data	IAM English handwriting and Vietnamese handwriting data
Image Height	32px
Checkpoints	English and Vietnamese checkpoints included
Intended Use	Research, reproducibility, handwriting synthesis, and data augmentation experiments

Highlights

Learns writer style from a small set of reference handwriting images.
Generates handwritten text conditioned on target text and writer style.
Supports English IAM-style handwriting and Vietnamese handwriting generation.
Includes prepared dataset pickles, lexicons, font templates, and released checkpoints.
Provides training code and qualitative result figures in one reproducible PyTorch release.

Qualitative Results

Handwriting Generation

Handwriting Reconstruction

Released Artifacts

File	Purpose
`File/eng_ckpt.pth`	Released English/IAM checkpoint
`File/vn_ckpt.pth`	Released Vietnamese checkpoint
`File/vgg19.pth`	VGG19 backbone checkpoint/resource used by the project
`File/IAM.pickle`	Prepared IAM handwriting dataset pickle
`File/VN.pickle`	Prepared Vietnamese handwriting dataset pickle
`File/unifont.pickle`	Font/template data for query rendering
`File/english_words.txt`	English lexicon
`File/vn_words.txt`	Vietnamese lexicon

Repository Structure

text

.
|-- data/                 # Dataset loading and preparation utilities
|-- Figures/              # Architecture and qualitative result figures
|-- File/                 # Datasets, checkpoints, lexicons, and font resources
|-- models/               # Generator, discriminators, recognizer, and writer encoder
|-- util/                 # Shared model and training utilities
|-- params.py             # Experiment and dataset configuration
|-- train.py              # Training entry point
`-- requirements.txt

Installation

Python 3.7 or newer and a CUDA-capable GPU are recommended for training.

Install PyTorch for your CUDA version first, then install the project dependencies:

pip install -r requirements.txt

Download Checkpoints and Artifacts

The released checkpoints, prepared data files, lexicons, and font resources are hosted in this Hugging Face repository. Clone the full release with Git LFS:

bash

git lfs install
git clone https://huggingface.co/DAIR-Group/WriteViT
cd WriteViT

If you want to download the artifacts into another local copy of this codebase, use huggingface_hub:

bash

pip install -U huggingface_hub
python - <<'PY'
from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="DAIR-Group/WriteViT",
    repo_type="model",
    local_dir=".",
    allow_patterns=[
        "File/*.pth",
        "File/*.pickle",
        "File/*.txt",
        "config.json",
        "README.md",
    ],
)
PY

The default IAM setup expects these files under File/:

text

File/
├── IAM.pickle
├── VN.pickle
├── eng_ckpt.pth
├── vn_ckpt.pth
├── vgg19.pth
├── english_words.txt
├── vn_words.txt
└── unifont.pickle

Configuration

Experiment settings are defined in params.py.

The default configuration uses IAM:

python

DATASET = 'IAM'
DATASET_PATHS = './File/IAM.pickle'
NUM_WRITERS = 339
WORDS_PATH = './File/english_words.txt'

To train or evaluate with Vietnamese data, switch to:

python

DATASET = 'VNDB'
DATASET_PATHS = './File/VN.pickle'
NUM_WRITERS = 106
WORDS_PATH = './File/vn_words.txt'

The available recognizer backbones are resnet18, vgg11, and vgg19.

Training

Review params.py, especially DATASET, DATASET_PATHS, NUM_WRITERS, WORDS_PATH, BACKBONE, learning rates, batch size, and RESUME.

Start training:

CUDA_VISIBLE_DEVICES=0 python train.py

Outputs are written to:

text

saved_models/<EXP_NAME>/
saved_images/<EXP_NAME>/

When RESUME = True, the training script loads:

saved_models/<EXP_NAME>/model.pth

Data Format

Prepared dataset pickle files contain writer-split handwriting samples:

python

{
    "train": {
        "writer_id": [
            {"img": PIL.Image.Image, "label": "handwritten text"}
        ]
    },
    "test": {
        "writer_id": [
            {"img": PIL.Image.Image, "label": "handwritten text"}
        ]
    }
}

Resources

Paper: https://arxiv.org/abs/2505.13235
Interactive demo: https://colab.research.google.com/drive/15Lswqr-aQwI-fF6yRoGYt-2pxSlC2L-R
Original datasets/checkpoints folder: https://drive.google.com/drive/folders/1ZgYS6-6l6fjKY75RJipONBByujIgf-uE

Citation

If you use WriteViT in your research, please cite:

bibtex

@article{nam2025writevit,
  title   = {WriteViT: Handwritten Text Generation with Vision Transformer},
  author  = {Dang Hoai Nam and Huynh Tong Dang Khoa and Vo Nguyen Le Duy},
  journal = {arXiv preprint arXiv:2505.13235},
  year    = {2025}
}

Acknowledgements

This repository builds on Handwriting Transformers by Ankan Kumar Bhunia et al. We thank the authors for making their work publicly available.