Qwen3-4B — MNN / Hexagon cDSP (4-bit)

模型信息 / Model Information

⚠️ 中文：这是 MNN 专用格式，不是 Hugging Face transformers 的 safetensors/bin 格式，无法用 AutoModel.from_pretrained 直接加载。需使用 MNN-LLM App 或 MNN 推理引擎，并选择 hexagon 后端。

⚠️ English: This is an MNN-specific format, not the Hugging Face transformers safetensors/bin format. It cannot be loaded directly with AutoModel.from_pretrained. Use the MNN-LLM app or MNN inference engine with the hexagon backend.

文件结构 / File Structure

qwen3_4b_hexagon/
├── llm.mnn          # MNN 计算图 / MNN compute graph
├── llm.mnn.weight   # 4-bit 量化权重 (2.0 GB) / 4-bit quantized weights
├── llm.mnn.json     # 导出配置 / export config
├── config.json      # MNN 配置（含 chat template）/ MNN config (with chat template)
├── llm_config.json  # MNN-LLM 推理配置（采样参数等）/ MNN-LLM inference config (sampling params)
├── tokenizer.mtok   # MNN tokenizer
└── export_args.json # 导出参数记录 / export args record

使用方法 / Usage

• 计算图 / 权重：llm.mnn + llm.mnn.weight
• Tokenizer：tokenizer.mtok
• 推理参数（temperature、top_p、repetition_penalty 等）见 llm_config.json

⚠️ 本模型针对 MNN 自研 Hexagon cDSP 后端（source/backend/hexagon/，FastRPC + 自研 HVX/HMX kernel）优化，不是 QNN / 高通 AI Engine Direct 路线。若需 QNN HTP 离线模型，另见 qwen3-0.6b-mnn-qnn-npu。

• Compute graph / weights: llm.mnn + llm.mnn.weight
• Tokenizer: tokenizer.mtok
• Inference parameters (temperature, top_p, repetition_penalty, etc.) are in llm_config.json

⚠️ This model is optimized for the MNN in-house Hexagon cDSP backend (source/backend/hexagon/, FastRPC + custom HVX/HMX kernels). It is not the QNN / Qualcomm AI Engine Direct path. For QNN HTP offline models, see qwen3-0.6b-mnn-qnn-npu.

量化说明 / Quantization

• 权重 / Weights: quant_bit=4, quant_block=64, sym=true
• 中文：Embedding 采用 tie_word_embeddings，与 lm_head 共享权重（export_args.json 中 tie_word_embeddings: true）。
• English: The embedding uses tie_word_embeddings, sharing weights with lm_head (tie_word_embeddings: true in export_args.json).

许可 / License