This repository hosts MiniVLA – a modular and deployment-friendly Vision-Language-Action (VLA) model designed for edge hardware (e.g., Jetson Orin Nano).
It contains model checkpoints, Hugging Face–compatible Qwen-0.5B LLM, and ONNX/TensorRT exports for accelerated inference.
🔎 Introduction
To enable low-latency, high-security desktop robot tasks on local devices, this project focuses on addressing the deployment and performance challenges of lightweight multimodal models on edge hardware. Using OpenVLA-Mini as a case study, we propose a hybrid acceleration pipeline designed to alleviate deployment bottlenecks on resource-constrained platforms.
We reproduced a lightweight VLA model and then significantly reduced its end-to-end latency and GPU memory usage by exporting the vision encoder into ONNX and TensorRT engines. While we observed a moderate drop in the task success rate (around 5-10% in LIBERO desktop operation tasks), our results still demonstrate the feasibility of achieving efficient, real-time VLA inference on the edge side.
🏗️ System Architecture
The MiniVLA deployment is designed with modular microservices:
Inputs: image + language instruction
Vision Encoder: DinoV2 / SigLIP → ONNX/TensorRT
LLM: Qwen 2.5 0.5B (Hugging Face / TensorRT-LLM)
Router & Fallback: balances between local inference and accelerated microservices
Robot Action: decoded from predicted action tokens
LLM Acceleration: Hugging Face → TensorRT-LLM engine, deployed as microservice (/llm/generate)
Main Process: Orchestrates requests, ensures fallback, and outputs robot actions
📦 Contents
models/
Contains the original MiniVLA model checkpoints, based on Stanford-ILIAD/minivla-vq-libero90-prismatic.
Special thanks to the Stanford ILIAD team for their open-source contribution.
qwen25-0_5b-trtllm/
Qwen-0.5B language model converted to TensorRT-LLM format.
qwen25-0_5b-with-extra-tokenizer/
Hugging Face–compatible Qwen-0.5B model with extended tokenizer.
12import requests
34url ="http://llm.svc:8810/llm/generate"5payload ={"prompt":"Close the top drawer of the cabinet."}6response = requests.post(url, json=payload)7generated_actions = response.json()
🔑 Key Contributions
Built an end-to-end online inference framework with a FastAPI service (/act), transforming offline benchmark code into a real-time deployable system.
Reproduced a lightweight OpenVLA-Mini and proposed a hybrid acceleration pipeline.
Exported the vision encoder to TensorRT, reducing perception latency and GPU memory usage.
Improved GPU memory efficiency: reduced average utilization from ~67% to ~43%, and peak usage from ~85% to ~65%, making deployment feasible under 8 GB memory constraints (similar to Jetson-class devices).
Integrated Qwen 2.5 0.5B in Hugging Face and TensorRT-LLM formats.
Designed a modular system architecture with router & fallback for robustness.
Demonstrated efficient edge-side VLA inference on Jetson Orin Nano in LIBERO tasks, with only a moderate performance drop (5–10%).
⚠️ Note: Although the experiments were run on RTX 4060, the GPU memory (8 GB) is comparable to entry-level Jetson devices, making it a suitable proxy for evaluating edge deployment feasibility.
GPU Memory Utilization (Long-Sequence Tasks)
Model Variant
Avg. GPU Utilization
Peak GPU Utilization
Original MiniVLA (PyTorch, no TRT)
~67%
~85%
MiniVLA w/ TensorRT Vision Acceleration
~43%
~65%
Observation:
The hybrid acceleration pipeline (TensorRT vision + VLA main process) reduced average GPU utilization by ~24% and peak usage by ~20%.
This indicates better GPU memory efficiency, allowing longer sequence tasks to run stably under resource-constrained devices.
Specify the license here (e.g., Apache 2.0, MIT, or same as MiniVLA / Qwen license).
📚 Citation
If you use MiniVLA in your research or deployment, please cite:
@misc{MiniVLA2025,
title = {MiniVLA: A Modular Vision-Language-Action Model for Edge Deployment},
author = {Xintao Zhen},
year = {2025},
url = {https://huggingface.co/xintaozhen/MiniVLA}
}
