Views
No views yet
CompiledModel GPU delegate
(no CPU fallback). DM-Count (NeurIPS 2020)
regresses a person density map whose sum is the crowd size — it counts hundreds of
people where detector-based counting saturates.
[1, 3, 512, 512] NCHW, RGB, ImageNet-normalized (mean [0.485,0.456,0.406], std [0.229,0.224,0.225]).[1, 1, 64, 64] non-negative density map — sum(map) = estimated person count; normalize per frame for a heatmap overlay.F.upsample_bilinear (align_corners=True RESIZE_BILINEAR, banned on the delegate) is a
linear operator, re-authored as two constant-matrix multiplies — with the constant on the
RHS (lowers to FULLY_CONNECTED; the delegate rejects BATCH_MATMUL with a constant
LHS). Desktop corr vs PyTorch is 1.000000 with an identical count.1val options = CompiledModel.Options(Accelerator.GPU)
2val model = CompiledModel.create(context.assets, "dmcount.tflite", options, null)
3val inBufs = model.createInputBuffers()
4val outBufs = model.createOutputBuffers()
5
6inBufs[0].writeFloat(inputNCHW) // [1,3,512,512] RGB, ImageNet-norm
7model.run(inBufs, outBufs)
8val density = outBufs[0].readFloat() // [64*64] density map
9val count = density.sum() // estimated number of people1import numpy as np
2from ai_edge_litert.compiled_model import CompiledModel
3
4model = CompiledModel.from_file("dmcount.tflite")
5inputs = model.create_input_buffers(0)
6outputs = model.create_output_buffers(0)
7inputs[0].write(np.ascontiguousarray(x, np.float32)) # [1,3,512,512] RGB, ImageNet-norm
8model.run_by_index(0, inputs, outputs)
9n = model.get_output_buffer_requirements(0, 0)["buffer_size"] // 4
10density = outputs[0].read(n, np.float32).reshape(64, 64)
11count = float(density.sum())build_dmcount.py): loads the MIT DM-Count (UCF-QNRF)
weights and exports the raw density map. The UCF-QNRF checkpoint generalizes best across
scenes; the upstream repo also bundles an NWPU-Crowd variant.benchmark_model tool — 10 warm-up runs then 50 timed runs, reported as the tool's mean.| Runtime | Backend | Graph on GPU | Latency |
|---|---|---|---|
LiteRT CompiledModel (LITERT_CL) | GPU | 30 / 30 | ~79 ms |
TFLite benchmark_model (TfLiteGpuDelegateV2) | GPU (OpenCL) | 30 / 30 | 98.2 ms |
TFLite benchmark_model | CPU (XNNPACK, 4 threads) | — | 3185.7 ms |
LITERT_CL figure is the one recorded when this model shipped, taken through LiteRT's own CompiledModel accelerator — the path the Kotlin sample app and the LiteRT API use. The TfLiteGpuDelegateV2 figure is the classic TFLite OpenCL delegate, measured with a tool anyone can download and re-run. They agree on how much of the graph the GPU takes; they disagree on speed, and the classic delegate is the slower of the two here. Read the TfLiteGpuDelegateV2 row as a reproducible floor, not as this model's speed on LiteRT.| backend | inference (median / min) | load |
|---|---|---|
| NPU (Hexagon v81) | 9.54 ms / 9.16 ms | 139 ms |
| GPU (Adreno) | 27.64 ms / 27.38 ms | 943 ms |
CompiledModel 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status NONE throughout. Headroom 0.70-0.71, where 1.0 is the throttling threshold.