Views
No views yet
.litertlm
bundle for the LiteRT-LM runtime. All credit for
the model itself goes to its author; this repo only packages it for phones and desktops.| file | quant | size |
|---|---|---|
Mordant-3B-Think_int8.litertlm | dynamic int8 (linears + embedding) | 3.76 GB |
python scripts/convert.py Kezmark/Mordant-3B-Think, 2026-08-25):<think> — is embedded verbatim (byte-equal to the checkpoint's
chat_template.jinja, 1474/1474).bos == eos == <|end_of_text|> and its template never renders a leading BOS, so an engine-prepended
start token reads as "this document already ended" — measured on this checkpoint, it flips
HF bf16 greedy output into a code-fence loop.-p 256 -d 256 --runs 3 --cache no)| backend | prefill tok/s | decode tok/s | TTFT |
|---|---|---|---|
| CPU | 97.3 | 20.2 | 2.68 s |
| GPU | 1129 | 71.9 | 0.24 s |
1pip install litert-lm
2litert-lm run Mordant-3B-Think_int8.litertlm \
3 --prompt "A cat sitting on a windowsill at sunset" --max-num-tokens 4096<think>…</think> block followed by the composed image prompt —
budget generation length accordingly. On Android, load the bundle in an app embedding the
LiteRT-LM engine (e.g. Google AI Edge Gallery-style hosts).