You can deploy this model and start earning money today!

qualcomm/
Track-Anything

pytorch

android

video-object-tracking

2304.11968

apache-2.0

Views

No views yet

Track-Anything: Optimized for Qualcomm Devices

Track-Anything is a video based machine learning model to track an object in a video.

This is based on the implementation of Track-Anything found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime	Precision	Chipset	SDK Versions	Download
ONNX	float	Universal	QAIRT 2.45, ONNX Runtime 1.27.1	Download
TFLITE	float	Universal	QAIRT 2.45	Download

For more device-specific assets and performance metrics, visit Track-Anything on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

Custom weights (e.g., fine-tuned checkpoints)
Custom input shapes
Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for Track-Anything on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.video_object_tracking

Model Stats:

Input resolution: 320x568
Model checkpoint: xmem
Model size (encode_key_with_shrinkage) (float): 37.1 MB
Model size (encode_key_without_shrinkage) (float): 37.1 MB
Model size (encode_value) (float): 88.8 MB
Model size (segment) (float): 115 MB
Number of parameters (encode_key_with_shrinkage): 9.72M
Number of parameters (encode_key_without_shrinkage): 9.71M
Number of parameters (encode_value): 23.3M
Number of parameters (segment): 30.1M

Performance Summary

Model	Runtime	Precision	Chipset	Inference Time (ms)	Peak Memory Range (MB)	Primary Compute Unit
encode_key_with_shrinkage	ONNX	float	Snapdragon® X2 Elite	2.773 ms	10 - 10 MB	NPU
encode_key_with_shrinkage	ONNX	float	Snapdragon® X Elite	5.331 ms	18 - 18 MB	NPU
encode_key_with_shrinkage	ONNX	float	Snapdragon® 8 Gen 3 Mobile	3.696 ms	9 - 206 MB	NPU
encode_key_with_shrinkage	ONNX	float	Snapdragon® 8 Gen 1 Mobile	10.689 ms	9 - 191 MB	NPU
encode_key_with_shrinkage	ONNX	float	Qualcomm® Dragonwing™ IQ-8275	9.544 ms	8 - 11 MB	NPU
encode_key_with_shrinkage	ONNX	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	5.242 ms	0 - 20 MB	NPU
encode_key_with_shrinkage	ONNX	float	Qualcomm® QCS8450	10.689 ms	9 - 191 MB	NPU
encode_key_with_shrinkage	ONNX	float	Qualcomm® Dragonwing™ IQ-9075	8.841 ms	8 - 11 MB	NPU
encode_key_with_shrinkage	ONNX	float	Qualcomm® Dragonwing™ IQ-X7181	5.331 ms	18 - 18 MB	NPU
encode_key_with_shrinkage	ONNX	float	Qualcomm® Dragonwing™ Q-8750	3.017 ms	1 - 170 MB	NPU
encode_key_with_shrinkage	ONNX	float	Snapdragon® 8 Elite Mobile	3.017 ms	1 - 170 MB	NPU
encode_key_with_shrinkage	ONNX	float	Snapdragon® 8 Elite Gen 5 Mobile	2.401 ms	2 - 175 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Snapdragon® 8 Gen 3 Mobile	4.068 ms	0 - 212 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Snapdragon® 8 Gen 1 Mobile	11.854 ms	3 - 200 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Qualcomm® Dragonwing™ IQ-8275	9.534 ms	3 - 31 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Qualcomm® Dragonwing™ IQ-8275	31.25 ms	3 - 179 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	5.962 ms	3 - 5 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Qualcomm® SA8775P	9.457 ms	3 - 181 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Qualcomm® SA8650P	9.457 ms	3 - 181 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Qualcomm® SA8255P	9.457 ms	3 - 181 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Qualcomm® QCS8450	11.854 ms	3 - 200 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Qualcomm® Dragonwing™ IQ-9075	9.208 ms	2 - 29 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Qualcomm® Dragonwing™ Q-8750	3.339 ms	0 - 184 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Qualcomm® SA7255P	31.25 ms	3 - 179 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Qualcomm® SA8295P	10.315 ms	3 - 163 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Snapdragon® 8 Elite Mobile	3.339 ms	0 - 184 MB	NPU
encode_key_with_shrinkage	TFLITE	float	Snapdragon® 8 Elite Gen 5 Mobile	2.464 ms	0 - 183 MB	NPU
encode_key_without_shrinkage	ONNX	float	Snapdragon® X2 Elite	3.886 ms	80 - 80 MB	NPU
encode_key_without_shrinkage	ONNX	float	Snapdragon® X Elite	6.566 ms	80 - 80 MB	NPU
encode_key_without_shrinkage	ONNX	float	Snapdragon® 8 Gen 3 Mobile	5.064 ms	55 - 257 MB	NPU
encode_key_without_shrinkage	ONNX	float	Snapdragon® 8 Gen 1 Mobile	13.497 ms	48 - 251 MB	NPU
encode_key_without_shrinkage	ONNX	float	Qualcomm® Dragonwing™ IQ-8275	11.554 ms	49 - 52 MB	NPU
encode_key_without_shrinkage	ONNX	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	6.92 ms	42 - 210 MB	NPU
encode_key_without_shrinkage	ONNX	float	Qualcomm® QCS8450	13.497 ms	48 - 251 MB	NPU
encode_key_without_shrinkage	ONNX	float	Qualcomm® Dragonwing™ IQ-9075	10.665 ms	58 - 60 MB	NPU
encode_key_without_shrinkage	ONNX	float	Qualcomm® Dragonwing™ IQ-X7181	6.566 ms	80 - 80 MB	NPU
encode_key_without_shrinkage	ONNX	float	Qualcomm® Dragonwing™ Q-8750	4.371 ms	52 - 249 MB	NPU
encode_key_without_shrinkage	ONNX	float	Snapdragon® 8 Elite Mobile	4.371 ms	52 - 249 MB	NPU
encode_key_without_shrinkage	ONNX	float	Snapdragon® 8 Elite Gen 5 Mobile	3.475 ms	2 - 181 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Snapdragon® 8 Gen 3 Mobile	5.511 ms	20 - 236 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Snapdragon® 8 Gen 1 Mobile	14.187 ms	20 - 223 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Qualcomm® Dragonwing™ IQ-8275	11.819 ms	20 - 64 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Qualcomm® Dragonwing™ IQ-8275	33.593 ms	20 - 203 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	7.817 ms	20 - 22 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Qualcomm® SA8775P	11.646 ms	20 - 208 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Qualcomm® SA8650P	11.646 ms	20 - 208 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Qualcomm® SA8255P	11.646 ms	20 - 208 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Qualcomm® QCS8450	14.187 ms	20 - 223 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Qualcomm® Dragonwing™ IQ-9075	11.374 ms	20 - 63 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Qualcomm® Dragonwing™ Q-8750	4.395 ms	19 - 204 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Qualcomm® SA7255P	33.593 ms	20 - 203 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Qualcomm® SA8295P	12.69 ms	20 - 184 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Snapdragon® 8 Elite Mobile	4.395 ms	19 - 204 MB	NPU
encode_key_without_shrinkage	TFLITE	float	Snapdragon® 8 Elite Gen 5 Mobile	3.577 ms	0 - 221 MB	NPU
encode_value	ONNX	float	Snapdragon® X2 Elite	8.095 ms	20 - 20 MB	NPU
encode_value	ONNX	float	Snapdragon® X Elite	13.213 ms	39 - 39 MB	NPU
encode_value	ONNX	float	Snapdragon® 8 Gen 3 Mobile	11.353 ms	0 - 209 MB	NPU
encode_value	ONNX	float	Snapdragon® 8 Gen 1 Mobile	31.431 ms	12 - 208 MB	NPU
encode_value	ONNX	float	Qualcomm® Dragonwing™ IQ-8275	18.746 ms	6 - 15 MB	NPU
encode_value	ONNX	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	17.103 ms	0 - 43 MB	NPU
encode_value	ONNX	float	Qualcomm® QCS8450	31.431 ms	12 - 208 MB	NPU
encode_value	ONNX	float	Qualcomm® Dragonwing™ IQ-9075	18.515 ms	6 - 14 MB	NPU
encode_value	ONNX	float	Qualcomm® Dragonwing™ IQ-X7181	13.213 ms	39 - 39 MB	NPU
encode_value	ONNX	float	Qualcomm® Dragonwing™ Q-8750	9.586 ms	8 - 192 MB	NPU
encode_value	ONNX	float	Snapdragon® 8 Elite Mobile	9.586 ms	8 - 192 MB	NPU
encode_value	ONNX	float	Snapdragon® 8 Elite Gen 5 Mobile	8.243 ms	8 - 183 MB	NPU
encode_value	TFLITE	float	Snapdragon® 8 Gen 3 Mobile	8.266 ms	2 - 288 MB	NPU
encode_value	TFLITE	float	Snapdragon® 8 Gen 1 Mobile	21.443 ms	3 - 291 MB	NPU
encode_value	TFLITE	float	Qualcomm® Dragonwing™ IQ-8275	18.152 ms	0 - 55 MB	NPU
encode_value	TFLITE	float	Qualcomm® Dragonwing™ IQ-8275	51.337 ms	3 - 217 MB	NPU
encode_value	TFLITE	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	11.881 ms	3 - 6 MB	NPU
encode_value	TFLITE	float	Qualcomm® SA8775P	16.984 ms	3 - 215 MB	NPU
encode_value	TFLITE	float	Qualcomm® SA8650P	16.984 ms	3 - 215 MB	NPU
encode_value	TFLITE	float	Qualcomm® SA8255P	16.984 ms	3 - 215 MB	NPU
encode_value	TFLITE	float	Qualcomm® QCS8450	21.443 ms	3 - 291 MB	NPU
encode_value	TFLITE	float	Qualcomm® Dragonwing™ IQ-9075	16.922 ms	0 - 55 MB	NPU
encode_value	TFLITE	float	Qualcomm® Dragonwing™ Q-8750	6.846 ms	3 - 216 MB	NPU
encode_value	TFLITE	float	Qualcomm® SA7255P	51.337 ms	3 - 217 MB	NPU
encode_value	TFLITE	float	Qualcomm® SA8295P	18.833 ms	3 - 220 MB	NPU
encode_value	TFLITE	float	Snapdragon® 8 Elite Mobile	6.846 ms	3 - 216 MB	NPU
encode_value	TFLITE	float	Snapdragon® 8 Elite Gen 5 Mobile	5.388 ms	2 - 202 MB	NPU
segment	ONNX	float	Snapdragon® X2 Elite	11.702 ms	21 - 21 MB	NPU
segment	ONNX	float	Snapdragon® X Elite	23.37 ms	38 - 38 MB	NPU
segment	ONNX	float	Snapdragon® 8 Gen 3 Mobile	17.348 ms	0 - 244 MB	NPU
segment	ONNX	float	Snapdragon® 8 Gen 1 Mobile	43.455 ms	1 - 230 MB	NPU
segment	ONNX	float	Qualcomm® Dragonwing™ IQ-8275	35.782 ms	21 - 46 MB	NPU
segment	ONNX	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	23.105 ms	0 - 288 MB	NPU
segment	ONNX	float	Qualcomm® QCS8450	43.455 ms	1 - 230 MB	NPU
segment	ONNX	float	Qualcomm® Dragonwing™ IQ-9075	34.88 ms	21 - 45 MB	NPU
segment	ONNX	float	Qualcomm® Dragonwing™ IQ-X7181	23.37 ms	38 - 38 MB	NPU
segment	ONNX	float	Qualcomm® Dragonwing™ Q-8750	13.286 ms	5 - 213 MB	NPU
segment	ONNX	float	Snapdragon® 8 Elite Mobile	13.286 ms	5 - 213 MB	NPU
segment	ONNX	float	Snapdragon® 8 Elite Gen 5 Mobile	10.827 ms	3 - 215 MB	NPU
segment	TFLITE	float	Snapdragon® 8 Gen 3 Mobile	17.447 ms	0 - 291 MB	NPU
segment	TFLITE	float	Snapdragon® 8 Gen 1 Mobile	43.075 ms	2 - 294 MB	NPU
segment	TFLITE	float	Qualcomm® Dragonwing™ IQ-8275	35.963 ms	0 - 83 MB	NPU
segment	TFLITE	float	Qualcomm® Dragonwing™ IQ-8275	103.376 ms	2 - 195 MB	NPU
segment	TFLITE	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	23.199 ms	2 - 4 MB	NPU
segment	TFLITE	float	Qualcomm® SA8775P	33.281 ms	2 - 193 MB	NPU
segment	TFLITE	float	Qualcomm® SA8650P	33.281 ms	2 - 193 MB	NPU
segment	TFLITE	float	Qualcomm® SA8255P	33.281 ms	2 - 193 MB	NPU
segment	TFLITE	float	Qualcomm® QCS8450	43.075 ms	2 - 294 MB	NPU
segment	TFLITE	float	Qualcomm® Dragonwing™ IQ-9075	34.307 ms	0 - 83 MB	NPU
segment	TFLITE	float	Qualcomm® Dragonwing™ Q-8750	13.221 ms	0 - 199 MB	NPU
segment	TFLITE	float	Qualcomm® SA7255P	103.376 ms	2 - 195 MB	NPU
segment	TFLITE	float	Qualcomm® SA8295P	37.568 ms	2 - 202 MB	NPU
segment	TFLITE	float	Snapdragon® 8 Elite Mobile	13.221 ms	0 - 199 MB	NPU
segment	TFLITE	float	Snapdragon® 8 Elite Gen 5 Mobile	11.594 ms	2 - 199 MB	NPU

License

The license for the original implementation of Track-Anything can be found here.

References

Community

Join our AI Hub Slack community to collaborate, post questions and learn more about on-device AI.
For questions or feedback please reach out to us.

qualcomm/Track-Anything