FFDetr — filled-form fine-tune
Fine-tune of
jbarrow/FFDetr (RF-DETR Medium) that detects
form fields
even when they already contain values — typed text, handwriting, checkbox marks,
signatures — on both born-digital and scanned pages. The base model only detects blank fields
reliably; on filled fields its recall collapses to near zero.
Classes
Same head layout and index order as the base model:
| id | class |
|---|
| 0 | Text (TextBox) |
| 1 | CheckBox (ChoiceButton) |
| 2 | Signature |
Results
Evaluated on a held-out 300-image synthetic validation set (including 60 field-less negative
pages) built from the CommonForms valid split — document-disjoint from training, with disjoint
fill-content generator profiles. IoU 0.5, confidence 0.5:
| Slice | Base P / R / F1 | This model P / R / F1 |
|---|
| filled_flat | 0.34 / 0.05 / 0.09 | 0.77 / 0.71 / 0.74 |
| filled_scan | 0.62 / 0.02 / 0.04 | 0.96 / 0.86 / 0.91 |
| hybrid_flat | 0.70 / 0.42 / 0.53 | 0.80 / 0.75 / 0.78 |
| blank_flat | 0.90 / 0.78 / 0.83 | 0.89 / 0.76 / 0.82 |
| blank_scan | 0.87 / 0.82 / 0.84 | 0.92 / 0.71 / 0.80 |
| overall | 0.84 / 0.53 / 0.65 | 0.87 / 0.77 / 0.82 |
- Overall AP@0.5: 0.635 (base) → 0.815
- False positives on field-less pages: 1.47 → 1.32 per negative page
- On pages that do contain fields, absolute FP rises 438 → 502 (+15%) because the model now
makes far more detections there — true positives rise 2669 → 3833 (+44%) — so the share of
false boxes among all detections falls from 16.5% to 13.2%, and precision improves on
every slice except blank_flat (flat: 0.90 → 0.89). The base model's low absolute FP on
filled pages is an artifact of it barely detecting anything there.
- Inference latency unchanged (~57 ms/img on A100 at resolution 1024; same architecture,
weights-only checkpoint)
On real filled documents (scanned bank forms with handwriting, typewriter-filled government
forms), the fine-tune matches the base model on strongly-outlined blank fields and additionally
detects value-bearing and graphics-obstructed fields the base model misses, without producing
detections on field-less pages.
Training data
8,000 images derived from 4,000 pages of
jbarrow/CommonForms
(3,200 pages with fields + 800 negatives, revision-pinned). Every page is paired with a synthetic
augmented counterpart: values rendered into field boxes (multilingual text incl. CJK/Arabic/Cyrillic,
choice marks, signature strokes; partial-fill variants) and/or scan degradation
(rotation, blur, contrast shift, noise, JPEG artifacts).
Recipe
Initialized from the base FFDetr checkpoint; detection head reinitialized for 3 classes.
8 epochs, lr 1e-5, batch 4 with grad-accum 4 (effective 16), resolution 1024, on 1× A100 40GB.
Usage
Pin a revision in production so a re-upload can never silently change your model:
1from huggingface_hub import hf_hub_download
2from commonforms.inference import FFDetrDetector
3
4path = hf_hub_download(
5 "pdf-net/ffdetr-filled",
6 "FFDetr.pth",
7 revision="e4fc1a8654915fe2f9bae63327062bc15711ecd6",
8)
9detector = FFDetrDetector(path)
The checkpoint is a drop-in replacement for the base model: same class indices, same input
resolution, same inference code path.
Known limitations
- Detected boxes follow the CommonForms field-rect convention (the fillable area). Ink written
outside that area — on or below an underline, spilling past the box — may not be fully covered
by the box. A future revision trains with overflowing fills and ink-union target boxes.
blank_scan recall is ~0.11 below the base model at conf 0.5 (precision is higher); expected
to recover with the larger final training run.
Status
Interim checkpoint from an 8k-image pilot. A final fine-tune on a larger dataset is planned and
will be published here as a new revision — pin a specific commit revision in production.
License: Apache-2.0, same as the base model (jbarrow/FFDetr), the CommonForms dataset, and the
rfdetr training library.
