microsoft/Mage-VL converted to the LiteRT-LM (.litertlm) format for on-device image+text inference with Google's LiteRT-LM runtime — the first Mage-VL in this format.
Mage-VL is Microsoft's 4.7B vision-language model: a 24-layer ViT with 3-D rotary position embeddings feeds a Qwen3-4B language decoder. It is a strong general describer and a very good document reader for its size. This bundle runs the image path through LiteRT-LM's fast_vlm runtime — give it an image and a question, get a grounded answer, fully on-device.
File
Mage-VL.litertlm (~2.81 GB)
Vision
mage_vl_vision ViT (24L, 1024-dim, full attention, 3-D rope) made static 448×448 → 784 patches → 2×2 merge → 196 image tokens, int8 weights
Adapter
PatchMerger (LN → 2×2 group → MLP), int8, output at the 2560 text hidden size
resized to 448×448 (OpenAI-CLIP normalization baked into the encoder)
Base model
microsoft/Mage-VL (Apache-2.0)
Performance (measured)
Platform
Decode
Time-to-first-token (image turn)
Engine init
Peak footprint
iPhone 17 Pro (CPU, release build)
~10 tok/s
~7.5–9 s (includes vision encode + prefill)
0.8 s warm / ~3–4 s first run
~1.5 GiB
macOS (Apple Silicon, LiteRT-LM CPU)
~66 tok/s
~2 s
~1 s
—
Text-only follow-up turns in the same conversation have ~2 s time-to-first-token on the phone.
How to use
1. Install the runtime
pip install litert-lm
2. Run it in one command — this downloads the bundle, encodes your image and answers:
bash
1litert-lm run --from-huggingface-repo litert-community/Mage-VL Mage-VL.litertlm \2 --attachment photo.jpg \3 --prompt "Describe this image in one sentence."
On the COCO sample image huggingface/documentation-images/coco_sample.png (two tabby cats on a pink blanket, remote controls beside them) this prints:
Two cats are sleeping on a pink couch next to remote controls.
Drop --prompt for an interactive chat, and pass --attachment more than once for several images. litert-lm serve exposes the same bundle as a local OpenAI-compatible API. The same file runs on macOS, Linux and Windows.
Performance
litert-lm benchmark (litert-lm 0.15.0) on an Apple M4 Max, -p 256 -d 256 --runs 3 (the tool averages three iterations), max-num-tokens 4096, warm-up run discarded, otherwise idle machine. These figures cover the text path; the vision encoder runs once per image and is not included.
Device
Backend
Prefill (256)
Decode
TTFT
Apple M4 Max (macOS)
CPU
65 tok/s
23.0 tok/s
4.11 s
Apple M4 Max (macOS)
GPU (Metal)
740 tok/s
80.0 tok/s
0.37 s
iPhone 17 Pro
CPU
—
~10 tok/s
7.5–9 s
Every desktop backend listed above was checked by actually generating a caption on it, not just by reading the benchmark tool's output.
The iPhone row is the 2026-08-03 on-device gate for the 2048-context build: engine init 2.8 s, 1.47 GiB peak while describing a photo, and a TTFT that includes vision encoding plus prefill.
Accuracy note
Device-verified on an iPhone 17 Pro and on the desktop LiteRT-LM runtime (macOS CPU):
General description / VQA (photo, on-device): accurate, detailed, and identical to the desktop runtime output token-for-token — an Ansel-Adams-style landscape → "a black and white photograph of a mountainous landscape … a winding river cutting through a dense forested valley … jagged, snow-capped mountains … heavy, brooding clouds".
Document OCR (Extract all the text from this image., on-device): perfect transcription of a full synthetic report page — every figure, the e-mail address and the phone number, at 448×448.
Vision tower: static-rewrite vs the reference implementation corr 1.0 (fp32), 0.994 at int8, zero FLEX/CUSTOM ops; patch pipeline verified bit-identical to the model's own image processor (max diff 2.4e-7).
Decoder: the Qwen3-4B text model is re-hosted as a standalone Qwen3ForCausalLM (state-dict strictly 1:1, untied lm_head) and quantized with the blockwise-128 + OCTAV int4 recipe that the shipped Qwen3-4B-class LLMs use; desktop 8-question sanity gate 7/8 with no degeneration.
No positional compromise. Unlike Qwen2-VL-family bundles, Mage-VL's language decoder natively uses plain sequential 1-D positions (no M-RoPE), which is exactly what the fast_vlm runtime supplies — the deployed decoder contract is mathematically identical to the original model. The 2-D-table-ranking caveat of the Qwen2-VL bundle does not apply here.
Image-only bundle. The base model's video pipeline (neural-codec frame compression) is not included — this bundle handles single images. Send each image in its own message; multiple images in one conversation are untested.
Galaxy S26 — GPU backend
The published bundle runs on the Android GPU backend and generates.
file
GPU backend
delegation
peak
Mage-VL.litertlm
runs
6618 / 6618 ops across 4 subgraphs on LiteRT GPU
820 MB
Measured on a Samsung Galaxy S26 (SM-S942Q / SM8850, Android 16) with litert_lm_advanced_main from litert-lm 0.16.0, --backend=gpu --sampler_backend=cpu, prompt What is the capital of France?. Peak is the process high-water mark (VmHWM) sampled during that same run. Gated 2026-08-25.
The op counts above are the LiteRT GPU partitions. XNNPACK additionally takes 1 of the 4 nodes in main; the runtime accepts that split.
The gate prompt carries no image, so this covers engine creation and the text path. The vision path on the GPU is not measured here.
No speed rows, on purpose. On this handset the GPU backend wins prefill and does not win decode, so a GPU throughput figure only means something beside a CPU row from the same handset, and no S26 CPU row exists for this model yet.
GPU wiring, including the Gallery import toggle: GPU guide.
Run on iPhone / macOS
Use the LiteRT-LM Swift runtime (swift-litert-lm). Load Mage-VL.litertlm with the vision tower enabled (Modality.textImage), attach a photo, and ask a question.
Vision-only bundle (no audio tower): bring the engine up with the vision modality only — requesting .all fails at session creation on bundles without an audio section.
Run on Android — Google AI Edge Gallery
Install a recent Google AI Edge Gallery, download Mage-VL.litertlm, import it (tap +, enable "Support image"), attach an image and ask. (Verified platforms above are iPhone, macOS, and — for engine bring-up and the text path on the GPU backend — a Galaxy S26 via the LiteRT-LM CLI rather than the Gallery app; Gallery import follows the same bundle contract as the other fast_vlm models here.)
Static rewrite of the dynamic-res vision tower. Mage-VL's ViT is native-resolution (packed patches, grid_thw, cu_seqlens varlen attention) and does not torch.export. The static graph fixes 448×448; a single image is one attention chunk, so the varlen machinery reduces to plain full attention.
Direct Conv2d patchify.temporal_patch_size=1, so the patch-embed is a stride-16 Conv2d — applied to the whole image in raster order (no per-patch reshuffle, no Conv3d fold needed).
3-D rope as a constant. The tower's rotary embedding splits head_dim 4:6:6 over (t,h,w) with interleaved rotation; for a single image t=0, and the (h,w) frequencies are precomputed from raster patch positions and baked into the graph.
No GATHER_ND (mobile-GPU-safe graph). Patches stay in raster order through the (permutation-equivariant) encoder; the 2×2 merge happens in the adapter with 4 strided slices + concat, all ops ≤4D. Static-rewrite corr vs the reference stays 1.0.
Context is 2048 (vs the trained 262k): the right on-device trade for a 4B decoder — the fp32 KV cache stays ~0.6 GB, keeping the whole session under ~1.5 GiB on the phone.
2026-08-28 — start_token fix (weights unchanged)
The bundle's LlmMetadata start_token held the literal string "None". This tokenizer has no BOS, and the LiteRT-LM engine resolved that string to a real vocabulary token — so every prompt began with the word None, which the model was never trained on. The start token has been removed.
Metadata-only change: every section of the bundle except the LlmMetadata block is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the tokenizer are unchanged and the speed and memory numbers on this card still describe exactly this file — only the file's own sha256 differs. What changed is the input: the token stream the model reads for a given conversation can differ from the previous file's, and it now matches this model's own reference chat stream. Greedy decoding can turn on a single token, so an individual answer can differ from the previous file in either direction. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file and have not been re-measured on this one. If you downloaded before 2026-08-28, re-download.
2026-08-29 — default system prompt restored (weights unchanged)
The upstream chat template emits a default system turn whenever the caller sends no system message — for this model: You are a helpful assistant.. The converter's template probe renders the template with a system message already present, so that block was never seen and never reached the bundle: with no system message the model was running without the default system turn it was tuned with. The chat template in Mage-VL.litertlm now emits the block exactly once when no system message is given. In Mage-VL.litertlm, the block is not emitted when you pass a system message. The restored block adds 11 prefill tokens to a conversation that sends no system message, so time-to-first-token grows by that much; per-token speed is unchanged.
Metadata-only change: every section of the bundle except the LlmMetadata block is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the tokenizer are unchanged and the speed and memory numbers on this card still describe exactly this file per token — only the file's own sha256 differs. What changed is the input: with no system message, the prompt now renders byte-identical to the upstream chat template's output, verified on the LiteRT-LM runtime. A system message you pass yourself renders as before. Greedy decoding can turn on a single token, so an individual answer can differ from the previous file. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file and have not been re-measured on this one. If you downloaded before 2026-08-29, re-download.
The tokenizer in Mage-VL.litertlm was a SentencePiece conversion of the model's BPE tokenizer, and the conversion lost the byte-level semantics: a standalone accented letter or symbol (é, ñ, ü, °, ·, …) was encoded to the id of a single-byte token instead of the token the upstream tokenizer uses, and any character without a whole-character vocabulary entry (emoji, most of Latin Extended-A) became the token the conversion had reused as UNK — the end-of-text token for this vocabulary. Mage-VL.litertlm now embeds the upstream tokenizer.json (the same HF tokenizer path most bundles in this collection use).
Tokenizer-only change: every section of the bundle except the tokenizer is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the chat template are unchanged and the speed and memory numbers on this card still describe this file per token — only the file's own sha256 differs. Verified on the LiteRT-LM runtime: the default turn, 7 probe strings, the 223 standalone characters U+00A1–U+017F and every special token now tokenize identically to the upstream tokenizer, and the four ASCII-only test questions answer byte-identically to the previous file (same ids in, same tokens out). Prompts containing accented letters, symbols or emoji reach the model differently from before, so individual answers to such prompts can change. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file, and any on-device rows were measured on the previous file too — the on-device gate has not been re-run on this one (the runtime's tokenizer code is the same on macOS and on device; the weights and graph are byte-identical). If you downloaded before 2026-08-30, re-download.
2026-08-31 — chat template fix: history turns with role 'assistant' (metadata-only, weights unchanged)
The chat template in Mage-VL.litertlm matched history turns only on role 'model', so a conversation created through the Conversation API with the standard role 'assistant' (for example create_conversation(messages=[...]) restoring a chat) silently dropped every assistant turn from the rendered prompt — the model saw consecutive user turns with its own replies gone. The template's role conditions now also accept 'assistant'. The engine's own send_message flow was never affected.
Template-only change: every section of the bundle except the metadata is byte-identical to the previous file (verified by per-section sha256), and single-turn renders, token ids, prefill counts and greedy answers are unchanged (verified on the LiteRT-LM runtime), so the numbers on this card still describe this file — only the file's own sha256 differs. Re-download only matters if you inject conversation history with role 'assistant'. Details: google-ai-edge/LiteRT-LM#3417 (https://github.com/google-ai-edge/LiteRT-LM/issues/3417).
