The models follow the BEVFusion [1] architecture (MIT Han Lab) and run with TensorRT inside Autoware. They are exported as ONNX so they can be deployed across hardware; Autoware builds the TensorRT engines from the ONNX files on first launch. The sparse convolution backend corresponds to spconv, executed at inference time through the spconv_cpp TensorRT plugins that Autoware installs automatically in its setup script.

Model overview


Task	3D object detection (oriented bounding boxes) from a LiDAR point cloud, optionally fused with multi-camera images
Architecture	BEVFusion (sparse-convolution LiDAR encoder, optional camera-to-BEV branch, transformer detection head)
Variant family	`t4base_120m`
Detected classes	`CAR`, `TRUCK`, `BUS`, `BICYCLE`, `PEDESTRIAN`
Runtime	TensorRT (FP16 by default) via the `autoware_bevfusion` ROS 2 node, with `autoware_tensorrt_plugins` for sparse convolution
Format	ONNX (Autoware builds the TensorRT engines locally on first launch)
License	Apache-2.0

Variants in this repository

Variant	Modality	ONNX files used	Cameras
`bevfusion_lidar`	LiDAR only	`bevfusion_lidar.onnx`	0
`bevfusion_camera_lidar`	Camera-LiDAR fusion	`bevfusion_camera_lidar.onnx` + `bevfusion_image_backbone.onnx`	6 (raw 1440x1080, ROI 576x384)

Common model parameters for both variants (from the ml_package_*.param.yaml files): point cloud range [-122.4, -122.4, -3.0, 122.4, 122.4, 5.0] m (roughly 120 m detection radius, matching the t4base_120m family name), voxel size [0.17, 0.17, 0.2] m, max_points_per_voxel: 10, num_proposals: 500, out_size_factor: 8, use_intensity: false.

Pre-processing (voxelization, multi-frame densification, optional image undistortion) and post-processing (circle NMS, IoU NMS, yaw normalization, distance-based score thresholding, area-based class remapping) run in the node, not in the ONNX graphs.

Files

File	Description
`bevfusion_lidar.onnx`	Main network, `bevfusion_lidar` variant
`bevfusion_camera_lidar.onnx`	Main network, `bevfusion_camera_lidar` variant
`bevfusion_image_backbone.onnx`	Image backbone, used by `bevfusion_camera_lidar`
`ml_package_bevfusion_lidar.param.yaml`	Model parameters for `bevfusion_lidar`
`ml_package_bevfusion_camera_lidar.param.yaml`	Model parameters for `bevfusion_camera_lidar`
`detection_class_remapper.param.yaml`	Area-based class remapping (e.g. large car to truck/trailer)
`deploy_metadata.yaml`	Deployment metadata recording the artifact version of this repository

TensorRT engines are not distributed here. TensorRT engines are specific to the GPU architecture and TensorRT version they are built on and are not portable, so Autoware builds them locally from the ONNX files on first launch (or via build_only:=true).

Inputs and outputs (as used by the node)

Inputs

Topic	Type	Description
`~/input/pointcloud`	`sensor_msgs/msg/PointCloud2`	Input point cloud, `PointXYZIRC` layout as defined in `autoware_point_types`
`~/input/image*`	`sensor_msgs/msg/Image`	Input images (RGB8), camera-lidar variant only
`~/input/camera_info*`	`sensor_msgs/msg/CameraInfo`	Camera intrinsics, camera-lidar variant only

Output is ~/output/objects (autoware_perception_msgs/msg/DetectedObjects): oriented 3D boxes with class and score. The node also publishes per-stage processing-time debug topics.

Usage in Autoware

The node expects these artifacts in $HOME/autoware_data/ml_models/bevfusion/ and launches with, e.g.:

bash

ros2 launch autoware_bevfusion bevfusion.launch.xml \
  model_name:=bevfusion_lidar \
  model_path:=$HOME/autoware_data/ml_models/bevfusion

model_name selects the variant (bevfusion_lidar, the default, or bevfusion_camera_lidar). Add build_only:=true to build the TensorRT engines from the ONNX files as a one-off pre-task. See the package README for the full parameter reference.

Training

The models were trained on TIER IV's internal database; the training data is not publicly available. The consuming package README documents training on roughly 35k LiDAR frames for 30 epochs. Version-specific training details for this t4base_120m/v2 release are not publicly documented.

Related training and inference resources:

Sparse convolution: https://github.com/traveller59/spconv
Sparse convolution TensorRT plugins used by Autoware: https://github.com/autowarefoundation/spconv_cpp

Limitations

Trained on TIER IV's internal sensor configurations; accuracy on a different LiDAR or camera setup (mounting positions, beam count, camera count and resolution) can drop without fine-tuning.
Only the five classes above are detected. Other road users fall outside the label set.
The input point cloud must follow the PointXYZIRC layout defined in autoware_point_types.
The consuming package notes that full integration of the camera-LiDAR fusion mode into the Autoware pipeline is still future work; the model can be employed without changes as a LiDAR-only detector.

Provenance


Original source	`https://awf.ml.dev.web.auto/perception/models/bevfusion/t4base_120m/v2/`
Hugging Face tag	`v2.0`

Citation

bibtex

@inproceedings{liu2023bevfusion,
  title     = {BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation},
  author    = {Liu, Zhijian and Tang, Haotian and Amini, Alexander and Yang, Xinyu and Mao, Huizi and Rus, Daniela and Han, Song},
  booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
  year      = {2023}
}

References

[1] Liu et al., "BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation", ICRA 2023, arXiv:2205.13542.
[2] spconv, sparse convolution library: https://github.com/traveller59/spconv
[3] spconv_cpp, Autoware's sparse convolution TensorRT plugin implementation: https://github.com/autowarefoundation/spconv_cpp

AutowareFoundation/bevfusion

BEVFusion for Autoware (bevfusion)