Source code and browser application: GitHub
IBEM im2typst — Phase 10
An experimental printed-formula image-to-Typst recognizer trained from project
seeded random initialization using only real IBEM formula crops. This repository
contains the current best validation-selected Phase 10 step-2,000 weights.
Status
This model is not production-qualified.
- Parameters: 14,157,152
- FP32 validation exact match: 92.6268% over 16,899 document-disjoint images
- At-most-32-token exact match: 97.8519%
- 33–64-token exact match: 88.4282% (below the 90% production gate)
- Displayed-formula exact match: 70.1197%
- Test split and confidence calibration: unopened
- INT8 built-in export probes: exact token parity
- Full-domain INT8 evaluation: not completed
The displayed token score is uncalibrated and is not the probability that a
formula is correct. Inspect and compile every generated Typst expression.
Files
| File | Purpose |
|---|
checkpoint-step-002000-transfer.pt | 56.7 MB optimizer-free FP32 PyTorch transfer checkpoint |
checkpoint-metadata.json | Checkpoint identity, lineage, hashes, and removed-state declaration |
encoder.int8.onnx | Dynamic-INT8 browser encoder |
decoder-step.int8.onnx | Cached autoregressive INT8 decoder step |
model-config.json | Recognizer architecture |
preprocess-config.json | Locked input preprocessing |
vocabulary.json | 528-token vocabulary with UTF-8 byte fallback |
output-policy.json | Locked package-free Typst output surface |
browser-deployment.json | Original browser export identity and artifact evidence |
evaluation.json | Complete FP32 validation report |
onnx-parity.json | FP32, FP16, and INT8 export-probe parity evidence |
The PyTorch file retains model_state and the lineage metadata required for
exact-model transfer. Optimizer state, RNG state, epoch/batch cursor, and cached
metrics were deliberately removed. It cannot resume the original optimizer.
browser-deployment.json preserves the identity of the original
multi-precision export and therefore names FP16/FP32 ONNX variants that are not
copied here. This model repository contains the FP32 PyTorch state and the two
small INT8 ONNX graphs; the unused FP16 and reference ONNX duplicates are
intentionally omitted.
Using the weights
The model uses a project-specific cnn-transformer-v3 architecture rather
than a Transformers architecture, so the hosted Hugging Face inference widget
is disabled. Use the companion GitHub source repository for the browser app,
Python model definition, preprocessing, training, evaluation, and export code.
After installing that Python package, the FP32 checkpoint can be loaded with:
1import torch
2
3from synthetic_im2typst.model import ImageToTypstRecognizer, ModelConfig
4
5checkpoint = torch.load(
6 "checkpoint-step-002000-transfer.pt",
7 map_location="cpu",
8 weights_only=True,
9)
10config = ModelConfig.from_data(checkpoint["model_config"])
11model = ImageToTypstRecognizer(config)
12model.load_state_dict(checkpoint["model_state"], strict=True)
13model.eval()
The two ONNX graphs are a cached encoder/decoder pair, not independent
single-call models. Use them together with vocabulary.json,
preprocess-config.json, and the browser inference runtime from the companion
source repository.
Architecture and input
The recognizer uses a compact convolutional encoder, depthwise axial 2-D
context, a height-4 learned horizontal-preserving pool, and a three-layer
autoregressive Transformer decoder. Input is foreground-cropped,
polarity-normalized, aspect-preserving resized, and padded to a 256-pixel
height with a maximum width of 2,048 pixels.
It expects one tightly cropped machine-typeset formula. It is not a page
segmenter, general OCR system, handwriting model, or photograph recognizer.
Training data
- 129,303 real training crops from 480 IBEM documents.
- 16,899 validation crops from 60 different documents.
- Images are grayscale crops from original IBEM page JPEGs, not formulas
rerendered in one standardized font.
- Targets are deterministic package-free Typst conversions of IBEM
latex_norm annotations.
- No Fusion dataset, synthetic formula corpus, teacher-generated labels,
external pretrained model, or fused samples were used.
IBEM is available from its
official Zenodo record under CC BY 4.0.
No IBEM image or annotation is included. The image under
example/ is an
original project-authored Typst render included only as a smoke-test sample.
Known limitations
- Small dot accents, top attachments, matrices, and alignment structures are
common compact-formula errors.
- Long and multiline displayed formulas are substantially weaker than short
inline formulas.
- Inputs with handwriting, perspective distortion, surrounding prose, or
arbitrary screenshot styling are not qualified.
- Exact-match evaluation is strict: one wrong token makes an output non-exact.
Identity and license
- Original full checkpoint SHA-256:
29d4a51adc71526f395750bb12a1783fbfd9bfa5a4c68fc9bfb665b68c633d52
- Optimizer-free checkpoint SHA-256:
8b7972f440556f328828f9e90247f266d9fbc59266f5ff8b23f7bce21f717924
- Parameter-state hash:
dcbf1a037fee205c82b7c52c9d0c0c2ee3d68d4a514f4593e15711f02bad31ec
- Training run ID:
64e092c2013ed43e35d3573e3e2a161c9948f12279ebb92513ab6b15c8811814
Project code, the identified weights, and the project-authored example are MIT
licensed. IBEM is not redistributed and retains its source license. See
LICENSE, LICENSES.md, MODEL_LICENSE.md, and DATA_PROVENANCE.md.
