LiteRT is Google's on-device runtime, the new name for TensorFlow Lite (Android: com.google.ai.edge.litert:litert), and litert-torch, the renamed ai-edge-torch, is its PyTorch converter: a PyTorch model converted unmodified with litert_torch.convert matched the original to 4e-7 on a Galaxy S26 (measured, LiteRT 2.2.0, Android 16, 2026-09-05).

allenai/OLMo-2-0425-1B-Instruct converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime.

OLMo-2 is AllenAI's fully-open model family — open weights, open data, open training code, Apache-2.0. This 1B variant is small enough to run on a phone, and is converted with the official upstream litert-torch with no fork patches.

The bundle carries the tokenizer and OLMo-2's native Tülu prompt template (<|user|> / <|assistant|>, stop token <|endoftext|>), so no separate tokenizer files are needed.

2. Run it in one command — this downloads the model and answers the prompt:

3. Keep it for repeated use — import once, then refer to it by name:

The same bundle runs on macOS, Linux and Windows.

Google AI Edge Gallery 1.0.16+ imports .litertlm models directly from Hugging Face — tap + (bottom-right) inside the app and pick this repository; no computer or adb needed. To sideload a local file instead:

then tap + and select the file. At 0.93 GB it fits comfortably on an 8 GB phone. To embed the model in your own Android app, use the LiteRT-LM Kotlin API (com.google.ai.edge.litertlm:litertlm-android).

litert-lm benchmark (litert-lm 0.15.0) on an Apple M4 Max, -p 256 -d 256 --runs 3 (the tool averages three iterations), max-num-tokens 4096, warm-up run discarded, otherwise idle machine:

Reproducibility: the GPU figures repeat to within about 1% across invocations; the CPU figures are noisier — six invocations of this model spread its CPU decode over 29.0–33.3 tok/s, so treat the CPU row as accurate to roughly ±7%.

On device — iPhone 17 Pro (iOS 27.0), GPU/Metal backend, single cold run with no warm-up turn, prompt "Explain on-device AI in one short sentence.", max 512 tokens:

The iPhone prefill figure is measured on a short chat prompt, so it is dominated by fixed per-turn overhead rather than prefill throughput — it is not comparable to the 256-token desktop prefill column. The model also fits and runs on a Pixel 8a (8 GB) via Gallery; no timing was taken there, so none is quoted.

GSM8K, n=100, greedy, 0-shot chain-of-thought, identical prompt and answer extraction for every row. The bf16 reference runs in PyTorch on CPU; the quantized rows run as .litertlm through the LiteRT-LM runtime.

63% is a coherent, non-degenerate score for a 1B: answers terminate cleanly at <|endoftext|> and do not degenerate. At this size 4-bit quantization costs about 9 points against bf16 — a 1B has less redundancy to absorb int4 rounding than a 3B+, where the same recipe lands at parity. int8 recovers only about 2 points for 60% more size (and the 2-point gap is within n=100 noise), so int4 is shipped as the size/quality trade-off for on-device use.

The published bundle runs on the Android GPU backend: LiteRT takes the whole graph and the model generates.

Measured on a Samsung Galaxy S26 (SM-S942Q / SM8850, Android 16) with litert_lm_advanced_main from litert-lm 0.16.0, --backend=gpu --sampler_backend=cpu, prompt What is the capital of France?. Peak is the process high-water mark (VmHWM) sampled during that same run. Gated 2026-08-24.

No speed rows, on purpose. On this handset the GPU backend wins prefill and does not win decode, so a GPU throughput figure only means something beside a CPU row from the same handset, and no S26 CPU row exists for this model yet.

GPU wiring, including the Gallery import toggle: GPU guide.

Converted with the official upstream litert-torch export_hf (clean git worktree at upstream/main, dev-fork patches excluded). Olmo2ForCausalLM rides the stock converter with no custom code: QK-norm and OLMo-2's reordered post-norm lower to generic ops. The int4 recipe is blockwise (block 32) + OCTAV with the embedding at int8. At 0.93 GB the whole bundle is a single sub-2 GiB section, so it memory-maps on iOS without needing an externalized embedder.

This is a weights-exact format conversion of allenai/OLMo-2-0425-1B-Instruct; no new training was performed. OLMo-2 is fully open: pretrained on AllenAI's public Dolma-family corpus (web text, code, academic papers, encyclopedic and Q&A data) and post-trained with the public Tülu 3 SFT + preference mix. Because the pretraining corpus is web-derived it may incidentally contain names and other PII; none was deliberately collected and this format conversion adds none. Apply your own content/PII filtering before deployment. See the base model card for full dataset and safety details.

The tokenizer.json embedded in OLMo-2-1B-Instruct_q4_block32_ekv4096.litertlm carried the GPT-2 default pre-tokenizer instead of the model's own regex, so digit groups and punctuation followed by a newline were split differently from the upstream tokenizer on every turn (the role markers <|user|>\n / <|assistant|>\n alone differed by 3 tokens per turn). OLMo-2-1B-Instruct_q4_block32_ekv4096.litertlm now embeds the upstream tokenizer.json byte for byte.

Tokenizer-only change: every section of the bundle except the tokenizer is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the chat template are unchanged and the speed and memory numbers on this card still describe this file per token — only the file's own sha256 differs. Verified on the LiteRT-LM runtime: the default turn, 7 probe strings, the 223 standalone characters U+00A1–U+017F and every special token now tokenize identically to the upstream tokenizer, and the four ASCII-only test questions answer byte-identically to the previous file (same ids in, same tokens out). Prompts containing accented letters, symbols or emoji reach the model differently from before, so individual answers to such prompts can change. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file, and any on-device rows were measured on the previous file too — the on-device gate has not been re-run on this one (the runtime's tokenizer code is the same on macOS and on device; the weights and graph are byte-identical). If you downloaded before 2026-08-30, re-download.

Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with litert-lm benchmark 0.16.1: CPU backend, 4 threads, 256 prefill + 256 decode tokens, --cache memory (the compile cache lives and dies with the process, so every invocation compiles the model from scratch; nothing is reused between runs), one warm-up plus one timed iteration per invocation, 3 invocations per file with cooldown in between. Values are the median across invocations (min–max in parentheses). No thermal throttling occurred during these runs (vcgencmd get_throttled stayed 0x0). Every file listed produced coherent text in a real generation on this backend before its numbers were recorded.

Apache-2.0, inherited from the base model allenai/OLMo-2-0425-1B-Instruct. Changes from the original work: weights converted from safetensors bf16 to LiteRT flatbuffers and quantized as described above; tokenizer and prompt template repackaged unmodified into the .litertlm bundle. This repository is a community conversion and is not affiliated with AllenAI.