This repository contains an early CoreML runtime bundle derived from datalab-to/surya-ocr-2 at source commit 3b3d4cdf88d6928b0acdc75181b13206ea67c4a3.

It is intended for native Apple OCR experiments and future iOS/macOS demo-app work. This revision includes a SwiftPM runtime package that runs the fixed-shape OCR canary path without Python.

Included packages

File	Purpose	Precision / quantization	Shape contract
`surya_vision_fp16.mlpackage`	Vision tower	CoreML FP16 compute	`pixel_values [1024,1536] -> image_embeds [256,1024]`
`surya_vision_int8.mlpackage`	Quantized vision tower	CoreML linear INT8 weight compression	`pixel_values [1024,1536] -> image_embeds [256,1024]`
`surya_prefill_fp16_seq300_cache512.mlpackage`	Language prefill, logits, and initial cache	CoreML FP16 compute	fixed prefill length `300`, cache length `512`
`surya_decode_step_fp16_cache512.mlpackage`	One-token cached decode step	CoreML FP16 compute	one token at a time, cache length `512`

Processor/tokenizer assets are included under processor/. Native constants and embedding assets are under native_assets/. Validation JSONs are under validation/.

Native Swift package

The SwiftPM package lives in native/SuryaCoreMLRuntime.

swift

import SuryaCoreMLRuntime

let runtime = try SuryaCoreMLRuntime(modelDirectory: modelDirectoryURL)
let result = try runtime.generate(image: cgImage, maxNewTokens: 128, useInt8Vision: true)
print(result.text ?? "")

The package currently handles fixed 512x512 Qwen image preprocessing, INT8/FP16 vision selection, fixed OCR prompt embeddings, image placeholder insertion, precomputed RoPE tables, generated-token embedding lookup, KV cache insertion, CoreML prefill/decode, and basic token decoding.

Current validation

All validation below was run on a Mac Studio with CoreML package prediction, using the canary prompt/image:

OCR this image to HTML.

The canary image contains:

text

Invoice 123
Total $42.00

Gate	Result
Prefill parity before CoreML export	native/custom first token `1039`; logits max diff `2.6702880859375e-05`
Prefill CoreML smoke	Torch/CoreML first token `1039`; logits max diff `0.3057253360748291`; mean diff `0.03853870555758476`
Decode CoreML iterative smoke	9/9 tokens match native; text `<p>Invoice`
CoreML prefill -> CoreML decode	9/9 tokens match native; text `<p>Invoice`
CoreML FP16 vision -> CoreML prefill -> CoreML decode	9/9 tokens match native; text `<p>Invoice`
CoreML INT8 vision -> CoreML prefill -> CoreML decode	9/9 tokens match native; text `<p>Invoice`
Native Swift image -> CoreML INT8 vision -> CoreML prefill -> CoreML decode	9/9 tokens match native; text `<p>Invoice`

The INT8 vision package has mean absolute diff 0.021211756393313408 vs the PyTorch vision tower on the canary.

What still lives in host code

The included Swift package is a fixed-shape native runtime for the current OCR path. The host app still owns:

providing a CGImage or already-preprocessed pixel_values
stopping criteria and max-token policy
UI and post-processing of generated OCR HTML
expanding beyond the current fixed 512x512 image / 300-token prefill shape

The included scripts/export_surya_coreml_runtime.py shows the Python reference glue used for validation. The included native/SuryaCoreMLRuntime package is the native implementation.

Example: run the current validation harness

bash

pip install coremltools torch transformers pillow qwen-vl-utils huggingface_hub
python scripts/export_surya_coreml_runtime.py vision-combined-runtime-smoke \
  --model-id datalab-to/surya-ocr-2 \
  --vision-package surya_vision_int8.mlpackage \
  --prefill-package surya_prefill_fp16_seq300_cache512.mlpackage \
  --decode-package surya_decode_step_fp16_cache512.mlpackage \
  --output validation/local_vision_int8_prefill_decode_smoke.json \
  --max-cache-length 512 \
  --steps 8

Expected canary output today:

json

{
  "all_tokens_match": true,
  "coreml_text": "<p>Invoice ",
  "native_text": "<p>Invoice "
}

Example: run the native Swift smoke

bash

cd native/SuryaCoreMLRuntime
swift run surya-coreml-smoke \
  --model-dir ../.. \
  --image /path/to/512x512-document.png \
  --max-tokens 8 \
  --vision int8

Validated canary output:

text

1039 2009 2046 2054 2047 2041 2035 2037 1979
<p>Invoice 

Important limitations

Fixed-shape canary export: the current packages are specialized to the traced 512x512 sample preprocessing path, with pixel_values [1024,1536], prefill length 300, and full-attention cache length 512.
The language prefill/decode packages are FP16 CoreML, not INT8/4-bit yet.
Only the vision tower has an INT8 package in this release.
This has not yet passed full allenai/olmOCR-bench.
The Swift decoder is intentionally small and should be hardened against the full tokenizer before production use.

Provenance

Generated non-destructively from datalab-to/surya-ocr-2. No fine-tuning was performed.

Reza2kn/surya-ocr-2-coreml-runtime

Surya OCR 2 CoreML Runtime