LiteRT is Google's on-device runtime, the new name for TensorFlow Lite (Android: com.google.ai.edge.litert:litert), and litert-torch, the renamed ai-edge-torch, is its PyTorch converter: a PyTorch model converted unmodified with litert_torch.convert matched the original to 4e-7 on a Galaxy S26 (measured, LiteRT 2.2.0, Android 16, 2026-09-05).
Megvii YOLOX-M (COCO, Apache-2.0) re-authored to a GPU-native LiteRT .tflite via the
official litert_torch path (no onnx2tf). FP16, 51.0 MB, input 640×640.
Verified on a Pixel 8a: the whole graph runs on the GPU delegate (full LITERT_CL residency,
zero CPU fallback) and the GPU output matches the CPU/PyTorch reference (corr ≥ 0.999).
Why this is GPU-clean
YOLOX is a pure CNN, but its Focus stem (stride-2 space-to-depth slicing) lowers to
GATHER_ND, which the GPU delegate rejects. Here the Focus + its following 3×3 conv are folded
into a single, numerically-exact 6×6 stride-2 conv, so the graph has zero GATHER/GATHER_ND/
TopK/Cast ops and no >4D tensors. Activations (SiLU) lower to LOGISTIC+MUL.
I/O
Inputimages[1, 640, 640, 3] NHWC, BGR, 0–255, no normalization (YOLOX letterbox:
uniform-scale to fit, pad bottom/right with gray 114).
80 class`. obj/class are already sigmoid'd; boxes are not decoded.
Host-side decode (kept out of the graph for GPU-cleanliness)
For anchor i at grid (gx,gy) with stride ∈ {8,16,32}:
cx=(raw_cx+gx)*stride, cy=(raw_cy+gy)*stride, w=exp(raw_w)*stride, h=exp(raw_h)*stride;
score = obj * max_class; then per-class NMS. Divide boxes by the letterbox ratio to map back.
Reference Kotlin + Python decode in the sample below.
Minimal usage
Android (Kotlin, CompiledModel GPU)
kotlin
1val model = CompiledModel.create(context.assets,"yolox_m.tflite",2 CompiledModel.Options(Accelerator.GPU),null)3val inputs = model.createInputBuffers()4val outputs = model.createOutputBuffers()5inputs[0].writeFloat(nhwc)// [1,640,640,3] BGR 0-255, letterbox pad 1146model.run(inputs, outputs)7val raw = outputs[0].readFloat()// [1,8400,85] -> decode + NMS on host (see Python)
Python (desktop verification)
python
1import numpy as np
2from PIL import Image
3from ai_edge_litert.interpreter import Interpreter
45SIZE =6406img = Image.open("photo.jpg").convert("RGB")7r =min(SIZE / img.width, SIZE / img.height)8w, h =round(img.width * r),round(img.height * r)9canvas = np.full((SIZE, SIZE,3),114, np.float32)# letterbox, gray 11410canvas[:h,:w]= np.asarray(img.resize((w, h)), np.float32)11x = np.ascontiguousarray(canvas[...,::-1])[None]# RGB -> BGR, 0-255, NHWC1213it = Interpreter(model_path="yolox_m.tflite"); it.allocate_tensors()14it.set_tensor(it.get_input_details()[0]["index"], x); it.invoke()15out = it.get_tensor(it.get_output_details()[0]["index"])[0]# [8400,85]1617grids, strides =[],[]# anchors = grid cells, s 8/16/3218for s in(8,16,32):19 n = SIZE // s
20 gy, gx = np.mgrid[:n,:n]21 grids.append(np.stack([gx, gy],-1).reshape(-1,2)); strides.append(np.full((n * n,1), s))22g = np.concatenate(grids).astype(np.float32); sv = np.concatenate(strides).astype(np.float32)23xy =(out[:,:2]+ g)* sv; wh = np.exp(out[:,2:4])* sv # boxes in 640-space24score = out[:,4:5]* out[:,5:]# obj x class (already sigmoid)25cls, conf = score.argmax(1), score.max(1)26for i in np.where(conf >0.35)[0]:# + per-class NMS in practice27 x1, y1 =(xy[i]- wh[i]/2)/ r; x2, y2 =(xy[i]+ wh[i]/2)/ r
28print(f"coco class {cls[i]}{conf[i]:.2f} [{x1:.0f},{y1:.0f},{x2:.0f},{y2:.0f}]")
Performance
COCO val2017 AP 46.9 (FP32 reference). Real-time on Pixel 8a GPU.
Training data & PII
Trained by Megvii on COCO 2017 (train2017), a public academic object-detection dataset
(Creative Commons). COCO images contain people as one of the 80 object categories; no names,
identities, or other personal attributes are modeled or output — the model emits only class id +
box. No additional or private data was used. Weights are the official Megvii release; only the op
graph was re-authored for GPU (weights unchanged).
Measured on a Pixel 8a (Tensor G3, Android 16) with the standard TFLite benchmark_model tool — 10 warm-up runs then 50 timed runs, reported as the tool's mean.
Runtime
Backend
Graph on GPU
Latency
TFLite benchmark_model (TfLiteGpuDelegateV2)
GPU (OpenCL)
465 / 465
81.4 ms
TFLite benchmark_model
CPU (XNNPACK, 4 threads)
—
836.1 ms
Any on-device figure recorded when this model shipped came from a different runtime. It was taken through LiteRT's own CompiledModel accelerator (logcat reports it as LITERT_CL), which is the path the Kotlin sample app and the LiteRT API use, and it appears elsewhere on this card. The rows above are the classic TFLite OpenCL delegate, measured with a tool anyone can download and re-run. The two are not comparable, so read the rows above as a reproducible floor rather than as this model's speed on LiteRT.
Snapdragon NPU (Hexagon)
The NPU is 3.88x faster than the GPU (5.25 ms against 20.37 ms) and loads 8.03x faster (122 ms against 982 ms).
backend
compiled
inference (median / min)
load
NPU (Hexagon v81)
on-device JIT
5.25 ms / 5.20 ms
122 ms
GPU (Adreno)
—
20.37 ms / 20.05 ms
982 ms
Measured on a Samsung Galaxy S26 (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16) with LiteRT CompiledModel 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status NONE throughout. Headroom 0.67, where 1.0 is the throttling threshold.
The NPU rows ran the published file unchanged. LiteRT compiled it for the Hexagon on the device at first load. That first compile took 2.5 s here. The load column above is the cached load every later run pays. Recipe and the runtime libraries it needs: NPU guide.
Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with the LiteRT benchmark_model tool from litert-cli-nightly 0.2.0.dev20260805: CPU inference (XNNPACK, 4 threads), 3 invocations per file of 10 warm-up plus 50 timed runs (the tool caps a phase at 150 s, so very slow graphs run fewer — the Runs column is the actual timed total). The latency is the median across invocations; the spread is the min–max over all timed runs. No thermal throttling occurred during these runs (vcgencmd get_throttled stayed 0x0).
