ONNX export of mayocream/koharu-yolo26s,
a comic-page instance-segmentation model. Four classes, boxes and masks:
ID
Class
0
frame
1
dialogue_text
2
balloon
3
onomatopoeia_text
This repository contains only the exported graph. Weights, training details and
evaluation live in the source repository.
Graph
IN images float32 [batch, 3, height, width]
OUT output0 float32 [batch, instances, 38]
OUT output1 float32 [batch, 32, height/4, width/4]
output0 rows are x1 y1 x2 y2 confidence class_id followed by 32 mask
coefficients, in input-image pixels, sorted by confidence. output1 holds the
mask prototypes.
The head is end-to-end (end2end: True), so no NMS is applied and none is
needed — but see the deduplication note below.
Shapes are dynamic. Height and width must be multiples of the stride (32).
Crop each mask to its own box before use; the prototype basis is shared across
instances, so an uncropped mask can carry probability mass from elsewhere on the
page.
Notes for consumers
Deduplicate per class. The end-to-end head emits the same instance more than
once. Measured over 12 pages: 3 duplicates in 144 instances at confidence 0.25,
12 in 166 at 0.10. Pairs of same-class boxes above IoU 0.7 are duplicates, not
separate regions. This is a property of the head, not of the export — the
PyTorch checkpoint produces the identical duplicates.
Padding changes results. Square 1280x1280 padding and stride-multiple
rectangular padding are not interchangeable. Same graph, same pages, square
versus rectangular: 109 matched instances, 2 only in square, 5 only in
rectangular, box IoU as low as 0.900, and confidence differences averaging 0.064
with a maximum of 0.634. Rectangular padding matches the Ultralytics predict
convention and is ~24% faster (445 ms versus 582 ms per page, ONNX Runtime CPU,
M1). Pick one and keep it.
Execution provider changes results near the threshold. CoreML claims 406 of
435 nodes and agrees on geometry (box IoU minimum 0.995), but confidences shift
by up to 0.141 and two extra instances appeared across 130. Where reproducibility
matters more than latency, pin the CPU provider.
The export was checked against the PyTorch checkpoint on byte-identical input,
comparing decoded instances rather than raw tensors — output0 rows are
confidence-sorted, so a single flipped score permutes every row below it and
makes element-wise comparison meaningless.
Confidence
Instances (torch / ONNX)
Matched
Box IoU (min)
Mask IoU (min)
Max abs confidence delta
0.25
144 / 144
144
1.000
1.000
6e-05
0.10
166 / 166
166
1.000
1.000
6e-05
Mask prototypes agree to 9e-06. A static-shape export of the same checkpoint
agrees with this dynamic one to 2e-05 on square input.
Terms and attribution
This checkpoint is a derived research artifact. Use and redistribution must
comply with the terms of Manga109, MangaSeg, COO, the upstream checkpoints, and
Ultralytics. The Ultralytics exporter stamps AGPL-3.0 into the graph metadata.
Review those terms before commercial use or redistribution.
