Model Card for BrickGPT
These are the model weights for BrickGPT, the first approach for generating physically stable toy brick models from text prompts, as described in
Generating Physically Stable and Buildable Brick Structures from Text.
This model was fine-tuned from
meta-llama/Llama-3.2-1B-Instruct.
Model Details
Model Description
- Developed by: Carnegie Mellon University Generative Intelligence Lab
- Funded by: This work is partly supported by the Packard Foundation, Cisco Research Grant, and Amazon Faculty Award. This work is also in part supported by the Manufacturing Futures Institute, Carnegie Mellon University, through a grant from the Richard King Mellon Foundation. KD is supported by the Microsoft Research PhD Fellowship.
- Model type: Autoregressive
- Language(s): English
- License: MIT
- Finetuned from model: meta-llama/Llama-3.2-1B-Instruct
- Project page: https://avalovelace1.github.io/BrickGPT/
Model Sources
- Repository: AvaLovelace1/BrickGPT
- Paper: Generating Physically Stable and Buildable Brick Structures from Text
- Demo: https://avalovelace1.github.io/BrickGPT/demo/
Limitations
The model is restricted to creating structures made of 1-unit-tall cuboid bricks on a 20x20x20 grid. It was trained on a dataset of 21 object categories: basket, bed, bench, birdhouse, bookshelf, bottle, bowl, bus, camera, car, chair, guitar, jar, mug, piano, pot, sofa, table, tower, train, vessel. Performance on prompts from outside these categories may be limited.
How to Get Started with the Model
See the
GitHub repo for usage examples and an interactive CLI demo.
Training Details
Training Data
BrickGPT was trained using
StableText2Brick, a dataset of 47k toy brick structures.
Training Procedure
The model was fine-tuned using LoRA applied to the q_proj and v_proj matrices. We used AdamW optimization. The learning rate followed a cosine decay with warmup.
Training Hyperparameters
- Training regime: bf16 mixed precision
- Epochs: 3
- Global batch size: 64
- Max learning rate: 0.002
- Learning rate warmup steps: 100
- LoRA rank: 32
- LoRA alpha: 16
- LoRA dropout: 0.05
Evaluation
See the paper for detailed evaluations.
Environmental Impact
- Hardware Type: 8x NVIDIA RTX A6000 (48 GB)
- Hours used: 0.5
Citation
If you find this model useful for your research, please cite the following work.
1@inproceedings{pun2025brickgpt,
2 title = {Generating Physically Stable and Buildable Brick Structures from Text},
3 author = {Pun, Ava and Deng, Kangle and Liu, Ruixuan and Ramanan, Deva and Liu, Changliu and Zhu, Jun-Yan},
4 booktitle = {ICCV},
5 year = {2025}
6}
Model Card Contact
Framework versions
