Thinking with Anchors Project | ADOPD 2026 Paper: Thinking with Anchors: Grounded and Efficient Document Reasoning | ADOPD 2024 Paper | Dataset | Code

Use Restrictions

Non-commercial research use only. The ADOPD fine-tuned checkpoint weights in this repository are provided solely for non-commercial research. Commercial use of these checkpoint weights is not permitted. Users must also comply with every applicable upstream license and acceptable-use term; see USE_RESTRICTIONS.md.

Model Overview

Model developer: Thinking with Anchors project contributors
Base model: nvidia/LocateAnything-3B
Model type: vision-language localization model
Task: document entity box and polygon localization (Doc2Mask)
Fine-tuning dataset: adopd/adopd2026
Input: one document image and an entity-localization prompt
Output: generated box and polygon coordinate tokens

Description

LocateAnything-3B Polygon ADOPD is fine-tuned to locate visual entities in document images and represent each prediction with a bounding box and polygon. It uses LocateAnything's structured generation interface and supports high-resolution document pages.

Training Data

This checkpoint is fine-tuned on the ADOPD Doc2Mask task. Public supervision is stored in:

human_annotated_masks[].polygons

The companion exporter derives an entity box from all valid components and emits a simplified polygon for the largest component.

Input And Output

The default prompt is:

Locate all the instances that matches the following description: entity.

The response contains normalized LocateAnything box and polygon tokens. Use the companion parser to obtain pixel-space geometry.

Quick Start

bash

git clone https://github.com/SichenZhu/ADOPD2026.git
cd ADOPD2026/release_code

python -m pip install -e model_zoo/common
python -m pip install -e model_zoo/locateanything_3b

hf download adopd/LocateAnything-3B-polygon-ADOPD \
  --local-dir checkpoints/locany-polygon

adopd-locany-infer \
  --checkpoint checkpoints/locany-polygon \
  --image document.jpg \
  --output prediction.json

prediction.json contains the raw generated response, parsed boxes, and parsed polygons.

Inference requires a CUDA-capable NVIDIA GPU. Eagle source and its compatibility overlay are only required for fine-tuning.

Fine-Tuning And Evaluation

Prepare ADOPD2026 with adopd-locany-prepare --task entity-polygon, then use the training and evaluation commands in locateanything_3b.

Limitations

The generated representation contains one simplified polygon per target and does not preserve every disconnected mask component. Results can be sensitive to image resolution, prompt wording, and decoding settings.

License

The ADOPD fine-tuned checkpoint weights are subject to the non-commercial, research-only restriction above. The included NVIDIA License and the terms for the LocateAnything base model and bundled remote-code files also apply. Use is permitted only when all applicable terms are satisfied.

Citation

Please cite the ADOPD 2026 and ADOPD 2024 papers.

bibtex

@misc{zhu2026thinkingwithanchors,
  title={Thinking with Anchors: Grounded and Efficient Document Reasoning},
  author={Sichen Zhu and Yuchen Zhu and Wenzhuo Xu and Jason Kuen and Wanrong Zhu and Jing Shi and Xuan Shen and Quanyi Wang and Yiwei Wang and Yujun Cai and Bing Shuai and Qin Zhang and Yongxin Chen and Shilong Liu and Molei Tao and Jiuxiang Gu},
  year={2026}
}

bibtex

@inproceedings{gu2024adopd,
  title={{ADOPD}: A Large-Scale Document Page Decomposition Dataset},
  author={Jiuxiang Gu and Xiangxi Shi and Jason Kuen and Lu Qi and Ruiyi Zhang and Anqi Liu and Ani Nenkova and Tong Sun},
  booktitle={The Twelfth International Conference on Learning Representations},
  year={2024},
  url={https://openreview.net/forum?id=x1ptaXpOYa}
}