Views
No views yet
lfm2_700m_xnnpack_8da4w_e8.pte (486.5 MB)embedding_quantize: "8,0")export_llm, static shape (seq_len=1), max_seq_length 2048,
XNNPACK extended_opsllm_params/lfm2_700m_xnnpack_8da4w_e8.yamlllm_params/gen_static.py, token-by-token prefill then greedy decode, a fresh process per
prompt so no answer is read through the previous one's cache:| prompt | answer |
|---|---|
| capital of France? | "The capital city of France is Paris." |
| 17 times 4? | "17 times 4 is 68." |
use_sdpa_with_kv_cache is on. Upstream's qwen3_5 config leaves it off with no
reason given while the equally hybrid lfm2 config has it on; measured on Qwen3.5-2B in
one run, that is 8.20 tok/s against 16.64.dim and hidden_dim both divide by the quantizer's group size. 8da4w only touches a
linear whose in_features divide by it, and skips the rest silently — SmolLM2-135M, which
is 576 wide, came out at 475 MB against fp32's 540 with no warning at all.convert/check_params_used.py. SmolLM3 sets no_rope_layer_interval, which ModelArgs
declares and only the MLX and Qualcomm backends read, and it exports fine and then repeats
a single word forever.1python llm_params/gen_static.py \
2 --pte lfm2_700m_xnnpack_8da4w_e8.pte \
3 --tokenizer tokenizer.json \
4 --prompt $'<|im_start|>user\nWhat is the capital of France?<|im_end|>\n<|im_start|>assistant\n' \
5 --eos_ids "[7]"from executorch.kernels import quantized before the program is
loaded, and portable_lib._load_for_executorch rather than executorch.runtime. Without
that the method will not load at all — kernel 'quantized_decomposed::embedding_byte.dtype_out' not found — which reads like a broken
export rather than a runtime missing its kernels.