SigLIP 2 (ViT-B/16, 224) — LiteRT (TFLite) GPU
On-device
LiteRT (
.tflite) conversion of
SigLIP 2 (Google 2025), a state-of-the-art CLIP-style image tower, converted
from
timm/vit_base_patch16_siglip_224.v2_webli
(ViT-B/16, 93M params; the image tower of
ViT-B-16-SigLIP2 /
google/siglip2).
A single forward pass turns one RGB image into a 768-d L2-normalized image
embedding for zero-shot classification, retrieval, and similarity — running
fully on the LiteRT CompiledModel GPU accelerator (ML Drift): all ops are
GPU-native (Replacing 809 out of 809 node(s) … LITERT_CL), no CPU fallback, no
Flex ops, and the GPU output matches PyTorch (corr ≈ 1.0).
Files
| File | Size | Description |
|---|
siglip2_base_224_fp16.tflite | 185 MB | FP16 single-graph model, GPU full-residency |
convert_siglip2.py | — | Reproducible conversion script (timm → tflite) |
I/O
- Input:
[1, 3, 224, 224] float32, NCHW, RGB normalized to [-1, 1]
((pixel/255 - 0.5) / 0.5). Normalization is applied by the caller.
- Output:
[1, 768] float32, L2-normalized image embedding.
For zero-shot classification, precompute text-label embeddings with the SigLIP 2
text tower (open_clip ViT-B-16-SigLIP2, prompt "This is a photo of {label}.")
and take the dot product on device.
Usage (Android, LiteRT CompiledModel)
1val model = CompiledModel.create(
2 context.assets, "siglip2_base_224_fp16.tflite",
3 CompiledModel.Options(Accelerator.GPU), null
4)
5val inputs = model.createInputBuffers()
6val outputs = model.createOutputBuffers()
7inputs[0].writeFloat(nchwFloatArray) // [1,3,224,224], RGB scaled to [-1,1]
8model.run(inputs, outputs)
9val embedding = outputs[0].readFloat() // [768], already L2-normalized
Performance
- ~60 ms / image steady-state on a Pixel 8a (Mali-G615) GPU (best ~9 ms),
full GPU residency, FP16.
Conversion notes
Converted with
litert-torch / ai-edge-torch.
Making the ViT image tower run fully on the GPU delegate
and produce correct
output on device required three verbatim (weights-exact, corr ≈ 1.0) model-side
rewrites (full GPU residency does
not imply a correct result):
- Fused-qkv → 4-D manual attention — the fused
qkv reshape emits a 5-D
head-split the delegate rejects; decompose into separate q/k/v. Self-attention
uses scaled_dot_product_attention, whose lowering keeps the batch-matmul 3-D
with a materialized transpose (both required for residency).
- Attention-pool single-query attention → broadcast-multiply + reduce-sum —
the pooling query is a constant latent, so a batch-matmul there is
const @ non-const (rejected / mis-computed); express as (q·k).sum + softmax
- Overflow-safe LayerNorm — the delegate computes the LayerNorm variance
reduction in fp16 even for an fp32 graph; deep-ViT massive activations make
sum((x-mean)²) exceed the fp16 max (65504), corrupting normalization (output
correlation collapses with depth while still reporting full residency). Scaling
by 1/32 before squaring keeps the sum in range.
Verified on a Pixel 8a GPU: zero banned ops, zero >4D tensors, full residency,
and GPU-vs-PyTorch output correlation ≈ 1.0 (the on-device GPU result, not just the
host CPU result).
Training data & PII
SigLIP 2 was pretrained by Google on the
WebLI dataset (billions of
web-crawled image–text pairs, multilingual, sigmoid contrastive objective). No new
training was performed for this conversion — it is a weights-exact format change of
the public
timm checkpoint. Because the source data is web-scraped, it may
incidentally contain people, faces, text, and other PII; no PII was deliberately
collected, and this conversion adds none. Apply your own content/PII filtering as
appropriate. See the original
SigLIP 2 model card
and
paper for full dataset details.
Performance
Measured on a
Pixel 8a (Tensor G3, Android 16) with the standard TFLite
benchmark_model tool — 10 warm-up runs then 50 timed runs, reported as the tool's mean.
| Runtime | Backend | Graph on GPU | Latency |
|---|
TFLite benchmark_model (TfLiteGpuDelegateV2) | GPU (OpenCL) | 44 / 809 | 963.6 ms |
TFLite benchmark_model | CPU (XNNPACK, 4 threads) | — | 312.9 ms |
Any on-device figure recorded when this model shipped came from a different runtime. It was taken through LiteRT's own CompiledModel accelerator (logcat reports it as LITERT_CL), which is the path the Kotlin sample app and the LiteRT API use, and it appears elsewhere on this card. The rows above are the classic TFLite OpenCL delegate, measured with a tool anyone can download and re-run. The two are not comparable, so read the rows above as a reproducible floor rather than as this model's speed on LiteRT.
On this delegate the CPU is the faster choice for siglip2_base_224_fp16.tflite (312.9 ms on CPU against 963.6 ms on GPU) — worth knowing before you reach for the GPU on a mid-range phone.
Note that the GPU does not take the whole graph here (44 / 809); the remainder runs on the CPU and the split costs a per-partition round trip.
Snapdragon NPU (Hexagon)
The NPU is 1.97x faster than the GPU (6.94 ms against 13.70 ms) and loads 10.18x faster (184 ms against 1872 ms).
| backend | compiled | inference (median / min) | load |
|---|
| NPU (Hexagon v81) | on-device JIT | 6.94 ms / 6.88 ms | 184 ms |
| GPU (Adreno) | — | 13.70 ms / 13.43 ms | 1872 ms |
Measured on a Samsung Galaxy S26 (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16) with LiteRT CompiledModel 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status NONE throughout. Headroom 0.59–0.61, where 1.0 is the throttling threshold.
The NPU rows ran the published file unchanged. LiteRT compiled it for the Hexagon on the device at first load. That first compile took 6.5 s here. The
load column above is the cached load every later run pays. Recipe and the runtime libraries it needs:
NPU guide.
License & attribution
- Apache-2.0 (original SigLIP 2 / timm checkpoint).
- This is a format conversion; all credit to the original authors (Google).
