phi_4_mini_xnnpack_8da4w_e8.pte (2597.9 MB)

llm_params/gen_static.py, token-by-token prefill then greedy decode, a fresh process per prompt so no answer is read through the previous one's cache:

Stopping is the part worth looking at. Chat template is <|user|>...<|end|><|assistant|>, bos 199999, eos [200020, 199999] — the model's own generation_config.json. The upstream example config (examples/models/phi_4_mini/config/phi_4_mini_xnnpack.yaml) carries 151643 for both, which is Qwen's, and a model that never sees its end token does not stop.

Not measured on a phone. Worth noting against the rest of the shelf: this decodes at 49 tok/s where Qwen3.5-4B manages 8.6 at a comparable 2.8 GB. The Qwen3.5 exports run with use_sdpa_with_kv_cache: False and this one with it on.

Not export_llm directly, because its download path cannot reach this model: load_checkpoint_from_pytorch_model reads pytorch_model.bin and this repository ships safetensors only, so the code falls back to torchtune's checkpointer — a dependency with its own torch pin, for a key rename and two splits. The script reads the safetensors, calls upstream's own phi_4_hf_to_meta, and ties the output embedding back on: input and output embeddings are tied here, so there is no lm_head.weight to map to output.weight, and the model refuses a checkpoint without it. The torchtune converter beside it ties them; the safetensors one does not.

The 8-bit embedding needs from executorch.kernels import quantized before the program is loaded. Without it the method will not even load — kernel 'quantized_decomposed::embedding_byte.dtype_out' not found — which reads like a broken export rather than a runtime missing its kernels.

(conversion scripts: executorch-models · iOS sample: executorch-samples)