It returns logits shaped [batch_size, sequence_length, 2]. Pulpie reads predictions at the <|sep|> token positions; class 1 is main, class 0 is other.

Files

File	Description
`model.onnx`	Validated FP32 ONNX Runtime model, dynamic batch and sequence axes
`tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`	Source tokenizer files
`config.json`, `configuration_eurobert.py`, `modeling_eurobert.py`	Source config/code for reference and tokenizer/model metadata
`export_report.json`	Export settings and ONNX graph summary
`verification_report.json`	Checker, parity, extraction, benchmark, and memory results
`quantized_report.json`	Int8 dynamic-quantization experiment results

Install

pip install onnxruntime transformers "pulpie[markdown]" huggingface_hub numpy

Raw ONNX Runtime Inference

python

import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

repo_id = "Mike0021/pulpie-orange-small-onnx"
model_path = hf_hub_download(repo_id, "model.onnx")

tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
sep_id = tokenizer.convert_tokens_to_ids("<|sep|>")

block = '<p _item_id="0">This is the main article text.</p>'
input_ids = [tokenizer.bos_token_id]
input_ids += tokenizer.encode(block, add_special_tokens=False)
input_ids += [sep_id, tokenizer.eos_token_id]

inputs = {
    "input_ids": np.asarray([input_ids], dtype=np.int64),
    "attention_mask": np.ones((1, len(input_ids)), dtype=np.int64),
}

session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
logits = session.run(["logits"], inputs)[0]
label = int(logits[0, input_ids.index(sep_id)].argmax())
print("main" if label == 1 else "other")

Pulpie-Style Extraction

python

import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

from pulpie.chunker import SEP_TOKEN, extract_blocks, pack_chunks, tokenize_blocks
from pulpie.markdown import to_markdown
from pulpie.model_utils import extract_item_ids, predictions_to_labels
from pulpie.reconstruct import extract_main_html
from pulpie.simplify import simplify

repo_id = "Mike0021/pulpie-orange-small-onnx"
model_path = hf_hub_download(repo_id, "model.onnx")
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])

def extract_markdown(html: str) -> str:
    simplified, map_html = simplify(html)
    blocks = extract_blocks(simplified)
    item_ids = extract_item_ids(blocks)
    sep_id = tokenizer.convert_tokens_to_ids(SEP_TOKEN)
    chunks = pack_chunks(
        tokenize_blocks(blocks, tokenizer),
        max_tokens=8192,
        sep_token_id=sep_id,
        bos_token_id=tokenizer.bos_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

    predictions = [0] * len(blocks)
    for chunk_ids, block_indices in chunks:
        input_ids = np.asarray([chunk_ids], dtype=np.int64)
        attention_mask = np.ones_like(input_ids, dtype=np.int64)
        logits = session.run(["logits"], {"input_ids": input_ids, "attention_mask": attention_mask})[0][0]
        sep_positions = np.nonzero(input_ids[0] == sep_id)[0]
        preds = logits[sep_positions].argmax(axis=-1).tolist()
        for i, block_idx in enumerate(block_indices):
            if i < len(preds):
                predictions[block_idx] = int(preds[i])

    labels = predictions_to_labels(item_ids, predictions)
    return to_markdown(extract_main_html(map_html, labels))

Verification

Environment: CPU-only, PyTorch 2.12.1+cu130, Transformers 4.57.6, ONNX 1.22.0, ONNX Runtime 1.27.0.

Check	Result
`onnx.checker.check_model(model.onnx)`	Pass
`onnxruntime.InferenceSession`	Pass
Nonstandard ONNX node domains	None
Main ONNX opset	18
Dynamic axes	`batch_size`, `sequence_length`
Graph nodes	671
End-to-end ONNX extraction	Non-empty Markdown, 149 chars on `article_basic`

Numerical parity against PyTorch FP32/eager:

Input	Shape	Max abs diff	Mean abs diff
`article_basic`	`[1, 86, 2]`	`6.50e-6`	`1.73e-6`
`docs_page`	`[1, 131, 2]`	`8.58e-6`	`1.53e-6`
`news_article`	`[1, 140, 2]`	`1.00e-5`	`1.55e-6`

All three inputs passed np.allclose(..., atol=1e-4, rtol=1e-3).

Benchmarks

CPU forward benchmark on a 131-token pulpie chunk, 5 measured reps:

Runtime	Mean latency	Min	Max
PyTorch FP32 eager	`181.29 ms`	`143.41 ms`	`233.41 ms`
ONNX Runtime FP32	`110.98 ms`	`107.31 ms`	`119.91 ms`

ONNX Runtime was 1.63x faster on this CPU benchmark. No CUDA device was available in the export environment, so GPU latency is not reported.

Approximate process RSS after model load and one inference:

Runtime	RSS	Delta after tokenizer/input prep
PyTorch FP32 eager	`1723 MB`	`751 MB`
ONNX Runtime FP32	`1916 MB`	`945 MB`

File sizes:

Artifact	Size
Source `model.safetensors` bf16	`404 MB`
Validated `model.onnx` FP32	`808 MB`

Quantization

Dynamic int8 quantization was evaluated locally. It reduced the ONNX file from 808 MB to 203 MB and loaded in ONNX Runtime, but it did not preserve extraction behavior: on the three verification examples it predicted no main blocks and produced empty Markdown. Because this did not maintain accuracy, the int8 model is not the validated artifact here. See quantized_report.json for the measured failure details.

Reproduction

The accepted model was exported with torch.onnx.export, opset 17 requested and opset 18 emitted by the current PyTorch exporter. ONNX Runtime extended optimization was tested but introduced com.microsoft fused nodes, so the uploaded model.onnx uses ORT basic optimization to keep the graph standard-domain and checker-clean.

bash

python scripts/export_onnx.py --source-model artifacts/source_model --output-dir artifacts/onnx_model
python scripts/verify_onnx.py --source-model artifacts/source_model --onnx-model artifacts/onnx_model/model.onnx

Model weights retain the source model's CC BY-NC 4.0 license.