Asset Harvester is an image-to-3D model and end-to-end system that converts sparse, in-the-wild object observations from real driving logs into complete, simulation-ready assets. The model generates 3D assets from a single image or multiple images of vehicles, VRUs or other road objects extracted from autonomous driving sessions. To run Asset Harvester, please check our codebase.
🐛 Asset Harvester code bugs / documentation issues / feature requests: file a GitHub issue using the appropriate template (Bug report, Documentation request, or Feature request) at https://github.com/NVIDIA/asset-harvester/issues/new/choose . The relevant NVIDIA responder is auto-assigned.
(Note: the Hugging Face Community tab for the Asset Harvester model will be disabled on 08/04/2026. Please use the channels above.)
Asset Harvester teaser
Asset Harvester turns real-world driving logs into complete, simulation-ready 3D assets — from just one or a few in-the-wild object views. It handles vehicles, pedestrians, riders, and other road objects, even under heavy occlusion, noisy calibration, and extreme viewpoint bias. A multiview diffusion model generates consistent novel viewpoints, and a feed-forward Gaussian reconstructor lifts them to full 3D in seconds. The result: high-fidelity 3D Gaussian splat assets ready for insertion into simulation environments. The pipeline plugs directly into NVIDIA NCore and NuRec for scalable data ingestion and closed-loop simulation.
Here's how the model checkpoints in this repo are used in the end-to-end system following the order in the pipeline: The AV object Mask2former instance segmentation model is used for image processing when parsing input views from NCore data sessions.
The input images are encoded by C-Radio,
and the multiview diffusion model, SparseViewDiT, is then used to generate 16 multiview images of the input objects.
In cases where camera parameters are not provided, the multiview diffusion model includes a camera pose estimation submodule that predicts camera parameters for the input images.
Lastly, an Object TokenGS lifts the images to a 3D asset.
This system is ready for commercial/non-commercial use
🚗 Example Results 🚗
Each row contains the input image, object mask, and a rendering of the harvested 3DGS asset.
This system is exposed as a collection of models on HuggingFace and inference scripts on Github.
Automation Level:
Partial Automation
Use Case:
Physical AI developers who are looking to create 3D assets of vehicles or VRUs for either closed-loop simulation or Synthetic Data Generation (SDG).
Known Technical Limitations:
The system is not guaranteed to perform well with occluded objects or objects that are outside of the common distribution. For example, a heavily occluded vehicle can generate a poor or hallucinated 3D asset.
Known Risk(s):
AV and robotics developers should be aware that this model cannot guarantee a 100% success rate. In cases of unsuccessful generation, the output may not possess an accurate real-world representation of the asset and should not be relied upon in safety-critical simulations.
System architecture details described in white paper above.
System Input:
Input Type(s): 1 or more images (up until 4) Input Format: Red, Green, Blue (RGB) Input Parameters: Two-Dimensional (2D) Other Properties Related to Input:
We currently accept up to 4 input images for each object. The resolution of the images are 512x512. The input images are extracted from NVIDIA’s NCore data along w/ other metadata needed for downstream processing:
Camera orientation of each image
Camera distance of each image
Camera field of view of each image
Bounding box dimensions of each object
System Output:
Output Type(s): Corresponding 3D Gaussian asset to the object in input images Output Format: Polygon File Format (PLY) Output Parameters: Three-Dimensional (3D) Other Properties Related to Output:
A PLY file (3D Gaussian Splatting, 3DGS) contains 3D object data with the following specific components:
Header: Defines the file structure, including format (ASCII or binary), Gaussian elements, their properties (e.g., position, appearance coefficients, opacity, scale, rotation), and data types (e.g., float, int).
Gaussian Data: Stores the parameters of each 3D Gaussian as vertex elements: center position (x, y, z), spherical harmonics DC coefficients (f_dc_0, f_dc_1, f_dc_2), opacity, anisotropic scale (scale_0, scale_1, scale_2), and rotation quaternion (rot_0, rot_1, rot_2, rot_3).
The systems can run on a single GPU with an Nvidia GPU with CUDA Compute Capability greater than or equal to 8.0. The following is required:
GPU performance >= 300 Tflops
GPU memory size >= 30GB
GPU memory bandwidth >= 768 GB/s
System RAM >= 32 GB
System disk storage >= 100GB
CPU >= 16 threads x 3GHz
System Version:
Asset_Harvester_GA
Inference:
Engine: Pytorch Test Hardware: A100, H100
Ethical Considerations:
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards.
Please report security vulnerabilities or NVIDIA AI Concerns here.
Model Card++
Bias
Field
Response
Participation considerations from adversely impacted groups protected classes in model design and testing:
None
Measures taken to mitigate against unwanted bias:
None
Explainability
Field
Response
Intended Domain
Autonomous Driving Simulation
Model Type:
Image-to-3D Asset
Intended Users:
Autonomous Vehicles developers enhancing and improving Neural Reconstruction pipelines.
Output
3D Asset
Describe how the model works
The system takes as an input one or few images, and outputs a corresponding 3D asset
Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of
None
Technical Limitations
The system is not guaranteed to perform well with occluded objects or objects that are outside of the common distribution. For example, a heavily occluded vehicle image can generate a poor or hallucinated 3D asset
Verified to have met prescribed NVIDIA quality standards
Yes
Performance Metrics
PSNR (Peak Signal-to-Noise Ratio)
Potential Known Risks
AV and robotics developers should be aware that this model cannot guarantee a 100% success rate. In cases of unsuccessful generation, the output may not possess an accurate real-world representation of the asset and should not be relied upon in safety-critical simulations.
The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development. Restrictions enforce dataset access during training
