Norwegian UPOS tagging, morphological features, and lemmatization with calibrated confidences, built for on-device, fully offline use. One compact model (17.6 M parameters) covers both written standards — Bokmål (nb) and Nynorsk (nn) — in a single set of weights; mixed input is fine. The artifact manifest also declares the BCP 47 macrolanguage no (since 0.2.3) and, since 0.2.4, the segmentation inventory the language-independent Prism runtimes (library 0.6.0+) read from the artifact. 0.2.5 re-lowers the int8 -fast programs with a grouped XNNPACK partitioner — ≈ 2× fp32 CPU speed at byte-identical weights and identical output (0.2.2–0.2.4 were manifest-only, byte-identical to 0.2.2; every published quality number still applies unchanged). Use 0.2.5 with Prism 0.6.0 or later; older runtimes keep working with any of these artifacts.

It beats UDPipe 2.17 on UPOS and lemmas on the official UD test splits — at about one twentieth of UDPipe's model size (fast artifact; one tenth for fp32), running fully offline on a laptop CPU.

This repository mirrors the versioned release artifacts of the Prism project. Prism ships native runtimes for Swift, C++, C, and Java/Kotlin that read these artifacts directly.

An application bundles exactly one folder. The folder is everything a Prism runtime needs; point the tagger API at its local path:

Quick starts for every binding: the project README. The artifact contract (programs, model.ptd weights, tokenizer and label schemas, checksums) is documented in docs/INTEGRATION.md.

Note: these are ExecuTorch programs with the decoding policy and calibration baked in — not transformers-loadable checkpoints. vocabulary.json is a standard Hugging Face tokenizer.json and loads with the tokenizers library.

Evaluated exactly once on the untouched official UD test splits against UDPipe 2.17 (gold tokenization, official CoNLL definitions):

Prism wins UPOS and lemmas on both written standards and stays behind only on exact morphology bundles — from a model a twentieth of UDPipe's size.

The frozen test evaluation above is fp32; the fast artifact is quality-gated on the development split (67,619 tokens across both standards — the test splits are evaluated exactly once and stay reserved for the fp32 benchmark). Accuracy with the identical production decoding policy:

Every delta is at most 0.014 percentage points — an order of magnitude below seed-to-seed training variance. int8 costs no measurable quality, and 0.2.5 changes only the delegate structure, not the weights, so these numbers are unchanged.

Measured with the reproducible benchmark suite on the checked-in CC0 example texts that ship inside the Prism repository (data/examples/) — anyone can rerun the identical protocol on their own hardware. Apple M4 Max, CPU only, release builds, median of 3 repetitions. The benchmark document is the Bokmål text repeated seven times: 385 sentences / 6,335 tokens, roughly a book chapter:

So the fast artifact tags a full chapter in under two seconds and a single ~900-token text in about half a second. The grouped int8 lowering in 0.2.5 lifts the fast artifact from ≈ 1.81× to ≈ 1.96× fp32 versus the previous per-op lowering, at identical output. The model-independent stages are effectively free next to the forward pass: runtime segmentation ≈ 8 M tokens/s, byte-level BPE ≈ 2.2–2.6 M tokens/s.

Reproduce it:

Recorded run and protocol: docs/benchmarks/prism-no-0.2.5.md; earlier records: docs/benchmarks/prism-no-0.2.2.md.

A 17.6 M-parameter encoder student (16-layer NorBERT4-xsmall backbone, hidden 192) distilled from a NorBERT4-large teacher, with a character CNN feeding morphology and lemma heads, a structured morphology decoder, and per-head temperature calibration (UPOS ECE 0.0017). Trained on the official UD gold treebanks plus teacher-labeled silver text. Full technical reference: docs/ARCHITECTURE.md.

This model exists thanks to openly licensed Norwegian resources:

Pinned revisions and checksums travel inside each artifact (manifest.json, LICENSES/).

Model weights: CC BY-SA 4.0. Using or bundling the unmodified artifact — including commercially, in closed-source applications — is fine (keep the attribution); redistributed modified weights must stay open. Prism source code is Apache 2.0.