Unofficial model derivative: This model distribution is not an official, endorsed, affiliated, or sponsored distribution of Liquid AI, Qualcomm, Radxa, Microsoft, OpenAI, or Anthropic.

AI-assisted development disclosure: OpenAI Codex and Anthropic Claude Code were used for research, code generation and editing, debugging, experiment organization, and documentation. A human reviewed the outputs and made the Q6A hardware-validation, adoption, and publication decisions.

Overview

A QCS6490/Q6A A16W8 QDQ ONNX derivative of Liquid AI's LFM2.5-350M. The model distribution contains raw QDQ and host-side constants only. Users generate EPContext on their own device.

Files

qdq/chunk16_a16w8_qdq.onnx
qdq/decode_a16w8_qdq.onnx
host/embedding_int8_rowwise/*
host/rope_cache.npz
tokenizer/*
MODEL_LICENSE
asset-manifest.json

Change Notice

QDQ ONNX files reconstruct the source model into a QNN-oriented graph and apply A16W8 QDQ. Embedding is quantized to rowwise symmetric int8. RoPE cache is mechanically transformed from official ONNX initializers into NPZ.

Intended Use

Local experiments, an OpenAI-compatible API, and OpenWebUI on QCS6490/Q6A. General long-form quality, other HTP generations, and ctx4096 are not guaranteed.

Installation

Use the tested one-command installer described in the GitHub Quick Start. It downloads all 11 assets from pinned revision 773ff42cc383cb61ecf32eb13d1f828634fbd0e1, verifies SHA-256, generates QNN EPContexts on the target Q6A, and requires finite QNN-only generation/reload. Its deterministic API canary requires normal Tokyo, JSON {"answer":"Tokyo"}, and first token ID 40550.

Runtime Compatibility

The tested stack is QCS6490 HTP v68, ONNX Runtime 1.27.0, and onnxruntime-qnn 2.3.0. EPContext is device/runtime-specific and is not distributed. The installer fingerprints QNN EP/HTP/Stub/Skel hashes and provider/session options; any mismatch regenerates the context.

Performance

The released runner measures 17.00-17.60 tok/s API decode and 0.31-1.03 s TTFT. See the validated configuration and measurement conditions on GitHub.

License

LFM Open License v1.0. Free commercial use is limited to annual revenue below $10,000,000. Redistribution requires including the license, marking changes, and retaining attribution.

PrismPhi/lfm2.5-350m-q6a-qcs6490-qnn-npu

LFM2.5-350M on Radxa Dragon Q6A (QCS6490 QNN NPU)