Neodragon — LiteRT (.tflite) conversions for on-device text-to-video
LiteRT conversions of
Qualcomm-AI-Research/Neodragon,
prepared for
Box — an open-source, fully offline
on-device AI app for Android.
These are
converted artifacts only. The models, research and original weights
are Qualcomm AI Research's work; see the
upstream repo and
paper.
Generates a 320×512 clip entirely on-device — no network, no server, no account.
Licence — please read before use
The weights are released by Qualcomm under two licences, and both carry over
to these conversions:
- BSD 3-Clause Clear —
permits redistribution and derivative works (a conversion is a derivative)
with notices retained. Note the "Clear" variant grants no patent licence.
- Qualcomm Responsible AI License —
adds use restrictions. It is not a grant licence; it sits on top of the BSD
grant and flows down to you. Prohibited uses include military use,
criminal/predictive-policing applications, social scoring, biometric
categorisation, emotion recognition in workplaces or education, and
distributing generated content without disclosing that it is machine
generated. Read the full text before using these files.
Box surfaces these restrictions in a consent screen before download and marks
generated output as AI-generated.
What's here
Fixed-shape LiteRT graphs. Filenames are deliberately flat — Box's downloader
flattens remote paths into a single directory.
Video pipeline (picture → video)
| file | size | shape |
|---|
dt5_text_encoder.tflite | 495 MB | (1,128) ids + mask → (1,128,4096) |
context_adapter_dynamic_int8.tflite | 124 MB | (1,128,4096) → (1,128,1536) |
clip_l_text_encoder.tflite | 472 MB | (1,77) → pooled 768 |
clip_g_text_encoder_int8.tflite | 670 MB | (1,77) → pooled 1280 |
dit_stage0_10x16_int4.tflite | 816 MB | pyramid stage 0 |
dit_stage1_20x32_int4.tflite | 817 MB | pyramid stage 1 |
dit_stage2_40x64_int4.tflite | 822 MB | pyramid stage 2 |
causal_video_vae_encoder_320x512.tflite | 419 MB | (1,3,1,320,512) → (1,16,1,40,64) |
causal_video_vae_decoder.tflite | 38 MB | (1,16,2,40,64) → (1,3,9,320,512) |
The three DiT graphs are the same 1.5B weights at three input resolutions; the
pipeline calls one per pyramid stage. Each accepts exactly two conditioning
latents, which is what limits this set to 9-frame clips.
First-frame pipeline (text → picture, SSD1B at 640×1024)
| file | size | shape |
|---|
ssd_1b_clip_l.tflite | 443 MB | (1,77) → (1,77,768) hidden_states[-2] |
ssd_1b_clip_g.tflite | 2.65 GB | (1,77) → (1,1280) + (1,77,1280) |
ssd_1b_unet_640x1024.tflite | 5.07 GB | (1,4,80,128) + t + ehs + text_embeds + time_ids |
ssd_1b_vae_decoder_640x1024.tflite | 189 MB | (1,4,80,128) → (1,3,640,1024) |
Tokenizer assets
| file | note |
|---|
clip_vocab.json, clip_merges.txt | CLIP BPE, shared by all four CLIP encoders |
t5_spiece.model | DT5 unigram SentencePiece |
clip_merges.txt retains its #version: 0.2 header line — skip it when
loading, or every merge rank shifts by one and tokenization silently corrupts.
Notes for anyone reusing these
- CPU only. The LiteRT GPU delegate supports at most rank-4 tensors, and
Neodragon's latents are 5D
(b,c,t,h,w). On a Tensor G5, --gpu places 256 of
3402 ops on the GPU and then segfaults (RESHAPE: Tensor dimensions must be less than 5). This is structural, not a configuration problem.
- Two different CLIP pairs.
clip_* are Neodragon's own encoders (for the
DiT's pooled vector); ssd_1b_clip_* are SSD1B's (for SDXL conditioning).
Their weights differ. The tokenizer vocabularies are identical, so one
tokenizer serves both.
- SDXL uses
hidden_states[-2], not last_hidden_state, from both CLIP
encoders. Using the last layer does not crash — it quietly degrades output.
- The SSD1B UNet is fp32 because int8 quantisation fails in
ai_edge_quantizer 0.8.0 on one layer
(CrossAttnDownBlock2D_2/ResnetBlock2D_0/conv2: "input operand has more
dimensions than allowed by the axis remapping").
- Measured on a Pixel 10 Pro Fold (Tensor G5), CPU: DiT stages ≈ 1.1 / 3.9 /
17.3 s per call.
Credit
All credit for the model and research to Qualcomm AI Research — Animesh
Karnewar and the Neodragon authors. These conversions exist only to make that
work runnable on more devices.
