You can deploy this model and start earning money today!

oyiya/
SuperMat

Name: SuperMat
Availability: InStock

material-decomposition

diffusion

image-to-image

2411.17515

sd2-community/stable-diffusion-2-1

finetune

mit

Views

No views yet

Model Overview

This repository hosts the pretrained parameters for the SuperMat project, as described in "SuperMat: Physically Consistent PBR Material Estimation at Interactive Rates" (ICCV 2025)

Model	Description
supermat.pth	Base SuperMat model for material decomposition
supermat_mv.pth	Multi-view version of SuperMat processing six orthogonal views
uv_refine_bc.pth	UV refinement network for albedo materials
uv_refine_rm.pth	UV refinement network for roughness & metallic materials

All models are built upon the base model stabilityai/stable-diffusion-2-1. Note: The official stabilityai/stable-diffusion-2-1 model has been removed. You may need to obtain the base model parameters through alternative sources, such as sd2-community/stable-diffusion-2-1.

Model Details

SuperMat (supermat.pth)

The core model for material decomposition. It takes RGBA images as input and decomposes materials from the target object.

SuperMat Multi-View (supermat_mv.pth)

An extended version that processes six orthogonal views simultaneously. This model leverages multi-view consistency for improved material estimation. For each view, the camera-to-world (c2w) matrix is provided as camera embeddings.

UV Refinement Networks

Two specialized networks for refining UV maps:

uv_refine_bc.pth: Refines the UV map for albedo materials
uv_refine_rm.pth: Refines the UV map for roughness & metallic materials

Download & Usage

Download the desired model(s) from this repository and place them in the checkpoints folder:

checkpoints/
├── supermat.pth
├── supermat_mv.pth
├── uv_refine_bc.pth
└── uv_refine_rm.pth

The models are independent of each other, so you only need to download those required for your specific inference task.

Input Requirements

Image Format

SuperMat models expect RGBA images where only the target object appears as foreground, with alpha values set to 0 for all other regions
During inference, the input image is alpha-composited with a gray background (0.5, 0.5, 0.5)

Resolution Preferences

SuperMat models: 512×512 resolution (recommended)
UV refinement networks: 1024×1024 resolution (recommended)

Multi-View Specific Requirements

For the multi-view model:

All inputs for a single case should be organized in one folder
Input images must follow the naming convention as shown in examples/bag_rendered_6views
Camera information is stored in meta.json (refer to the example for the required format with c2w matrices)

Quick Inference Examples

SuperMat Single-Image

python inference_supermat.py \
  --input examples/ring_rendered_2views \
  --output-dir outputs \
  --checkpoint checkpoints/supermat.pth \
  --base-model sd2-community/stable-diffusion-2-1 \
  --device cuda:0 \
  --image-size 512

SuperMat Multi-View

python inference_supermat_mv.py \
  --input examples/bag_rendered_6views \
  --output-dir outputs_mv \
  --checkpoint checkpoints/supermat_mv.pth \
  --base-model sd2-community/stable-diffusion-2-1 \
  --device cuda:0 \
  --image-size 512 \
  --num_views 6 \
  --use-camera-embeds

UV Refinement (Albedo)

python inference_uv_refine.py \
  --input-uv examples/axe_uv/uv_bc.png \
  --input-uv-position examples/axe_uv/uv_position.png \
  --input-uv-mask examples/axe_uv/uv_mask.png \
  --output-dir outputs_uv_bc \
  --checkpoint checkpoints/uv_refine_bc.pth \
  --base-model sd2-community/stable-diffusion-2-1 \
  --device cuda:0 \
  --image-size 1024

For complete usage instructions, please refer to the main repository.

Citation

If you find these models useful in your research, please cite:

@inproceedings{hong2025supermat,
  title={Supermat: Physically consistent pbr material estimation at interactive rates},
  author={Hong, Yijia and Guo, Yuan-Chen and Yi, Ran and Chen, Yulong and Cao, Yan-Pei and Ma, Lizhuang},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={25083--25093},
  year={2025}
}

oyiya/SuperMat