This repository provides the DPText-DETR text detection experiments from the MonkeyOCRv2 paper. The visual encoder from MonkeyOCRv2-AS (ViTAEv2-S, 21M parameters) is integrated into DPText-DETR as a drop-in detectron2 backbone. The last three ViTAEv2 stages (strides 8/16/32) are exposed as res3–res5 and feed the standard deformable-DETR input projections, so the transformer encoder/decoder and the detection head are unchanged.

Training and evaluation follow the official DPText-DETR protocols on Total-Text, CTW1500, ICDAR19-ArT, Rotated Total-Text and Inverse-Text.

For each benchmark, three visual backbones are compared under identical settings: the original ImageNet-pretrained ResNet-50, the text-specific oCLIP ResNet-50, and MonkeyOCRv2. MonkeyOCRv2 consistently improves F-score across all five benchmarks.

All models are trained directly on the target dataset (no SynthText/MLT pre-training), for 200k iterations with a total batch size of 8, using the positional label form and the rotated training images released with DPText-DETR (*_poly_train_rotate_pos).

Rotated Total-Text and Inverse-Text are test-only benchmarks: they reuse the Total-Text model above and only change DATASETS.TEST.

Download the checkpoints from HB16888/MonkeyOCRv2_det_dptext (HuggingFace) or WangXinhan/MonkeyOCRv2_det_dptext (ModelScope):

The reproduced environment uses Python 3.11, PyTorch 2.9.0, CUDA 12.8, torchvision 0.24.0, detectron2 0.6, NumPy 2.4.4, Transformers 4.57.1 and safetensors 0.7.0. The oCLIP baseline additionally needs MMOCR 1.0.1 (MMEngine 0.10.7, MMCV 2.0.1, MMDet 3.1.0). All models were trained on 8 GPUs (NVIDIA GeForce RTX 3090) with SOLVER.IMS_PER_BATCH: 8 for 200k iterations.

The code for these checkpoints is the detection/DPText-DETR add-on of MonkeyOCRv2, which sits on top of the official DPText-DETR release. Run it from that add-on directory (MonkeyOCRv2/detection/DPText-DETR):

install.sh creates a nested checkout, ./DPText-DETR, which is the DPText-DETR root referred to throughout this README:

Download Total-Text (including rotated images), CTW1500 (including rotated images), ICDAR19-ArT (including rotated images), Inverse-Text, the polygon json files and the evaluation ground-truths from the official DPText-DETR data preparation links, and organize them under the DPText-DETR root as:

All training and evaluation commands are run from the DPText-DETR root (detection/DPText-DETR/DPText-DETR), on 8 GPUs.

Each config already carries the MODEL.TRANSFORMER.INFERENCE_TH_TEST value that reproduces the corresponding row of the tables above, so evaluating on the dataset a model was trained on needs no extra flags:

Evaluation prints precision / recall / hmean on the copypaste: line, matching the tables above.

These reuse the Total-Text checkpoints and override the test set and the threshold:

The full set of thresholds used for the tables:

tools/search_th.py sweeps INFERENCE_TH_TEST for a trained model and reports the best F-score:

ArT has no public test ground-truth. Evaluating an ArT config writes <OUTPUT_DIR>/inference/art_submit.json, which has to be uploaded to the ICDAR19-ArT evaluation server to obtain the P / R / F numbers reported above:

This project builds on DPText-DETR, AdelaiDet, detectron2, MMOCR, oCLIP, and MonkeyOCRv2.

The DPText-DETR / AdelaiDet sources this add-on patches are released for non-commercial use only (see the DPText-DETR license); the same restriction applies to the add-on and to the released checkpoints.