.litertlm) conversion of
WeiboAI/VibeThinker-3B (a Qwen2.5-3B–architecture
reasoning model) for on-device / edge inference via Google AI Edge LiteRT-LM.| file | size | notes |
|---|---|---|
vibethinker3b_q8_ekv8192_lora16.litertlm | ~3.4 GB | prefill+decode, int8 weights, 8192 ctx, runtime-swappable LoRA (rank 16) |
WeiboAI/VibeThinker-3B (Qwen2.5-3B: 36 layers, hidden 2048, 16 heads / 2 KV groups, vocab 151936)litert-torch 0.9.0 generative converter (examples.qwen.convert_to_tflite)dynamic_int8 (int8 weights / fp32 activations)prefill_256, decode, plus LoRA-enabled prefill_256_lora_r16, decode_lora_r16qwen2p5, HF tokenizer, Qwen2.5 chat template embedded, stop tokens <|im_end|> (151645) / <|endoftext|> (151643)LoRA note: the rank-16 LoRA signatures target the q/k/v/o projections and let the LiteRT-LM runtime load/swap a fine-tuned adapter at init (EngineSettings::SetScopedLoraFile). Exporting these required fixing a grouped-query-attention out-dim bug inlitert-torch'slora.py(reported upstream: litert-torch#1066).
litert_lm_main / engine API):litert_lm_main --backend=cpu --model_path=vibethinker3b_q8_ekv8192_lora16.litertlmlitert_torch's
LoRA.from_safetensors(...).to_tflite() and load the resulting file via the runtime's scoped-LoRA API.WeiboAI/VibeThinker-3B.