You can deploy this model and start earning money today!

heig-vd-geo/
DeepChoice

Name: DeepChoice
Availability: InStock

heig-vd-geo/DeepChoice

heig-vd-geo/ImagesAndPointCloudsCulturalHeritageDataset

mit

Views

No views yet

DeepChoice

DeepChoice is a lightweight multi-view fusion framework for image-guided 3D semantic segmentation.
For each 3D point, the preprocessing pipeline gathers the visible images, computes geometric and radiometric visibility criteria, attaches per-view 2D semantic scores, and the model learns one weight per view before fusing the evidence into a final 3D prediction.

This repository contains:

the DeepChoice training, evaluation, and inference code
preprocessing pipelines for GridNet-HD and Cultural Heritage
two DeepChoice backbones:
- DeepChoiceTransformer
- DeepChoiceMLP
a 2D image baseline used to export per-image logits / softmax scores
public config files and released checkpoints

Citation

If you use this repository, please cite:

bibtex

@article{Carreaud2026DeepChoice,
  title={DeepChoice: Learning View Weighting for Image-Guided 3D Semantic Segmentation},
  author={Antoine Carreaud and Digre Frinde and Shanci Li and Jan Skaloud and Adrien Gressin},
  year={2026},
  url={https://cspaper.org/openprint/20260331.0001v1},
  journal={OpenPrint:20260331.0001v1}
}

Repository Layout

text

.
├── main.py
├── configs/
├── dataset/
├── dataset_generation/
├── experiments/
├── image_baseline/
├── model/
├── scripts/
├── train/
├── utils/
└── weights/

Main directories:

dataset_generation/: preprocessing and batch generation
dataset/: PyTorch datasets for training and evaluation
model/: DeepChoice backbones
train/: training, testing, and inference logic
image_baseline/: 2D segmentation baseline used to export image scores
experiments/: reusable experiment runners
scripts/: small utilities for figure generation or format conversion
weights/: released checkpoints

Public Config Files

DeepChoice configs:

configs/config_deepchoice_base.yaml
configs/config_gridnet_hd_transformer.yaml
configs/config_gridnet_hd_mlp.yaml
configs/config_cultural_heritage_transformer.yaml
configs/config_cultural_heritage_mlp.yaml

Dataset generation configs:

configs/config_gridnet_hd_dataset_generation.yaml
configs/config_cultural_heritage_dataset_generation.yaml

Released Checkpoints

DeepChoice:

weights/deepchoice/gridnet_hd_transformer_best_model.pt
weights/deepchoice/gridnet_hd_mlp_best_model.pt
weights/deepchoice/cultural_heritage_transformer_best_model.pt
weights/deepchoice/cultural_heritage_mlp_best_model.pt

Image baseline:

weights/image_baseline/gridnet_hd_best_model.pth
weights/image_baseline/cultural_heritage_best_model.pth

Main Results

The tables below summarize the main quantitative results reported in the paper.

GridNet-HD test split

Method	mIoU (%)	Params
Hard vote	66.78	60 M
Mean-probability fusion	69.37	60 M
DeepChoice-MLP	70.52	60 M + 0.09 M
DeepChoice-Transformer	70.63	60 M + 0.02 M
AnyView oracle	84.33	-

On the full GridNet-HD benchmark, DeepChoice-Transformer improves over hard voting by +3.85 mIoU points and over mean-probability fusion by +1.26 mIoU points.

Cultural Heritage (`3_SS`)

Method	mIoU (%)	mF1 (%)
Hard vote	65.66	73.87
Mean-probability fusion	66.73	74.81
DeepChoice-MLP	66.35	74.54
DeepChoice-Transformer	67.21	75.49
AnyView oracle	86.92	92.83

On Images&PointClouds Cultural Heritage, DeepChoice-Transformer improves over hard voting by +1.55 mIoU points and over mean-probability fusion by +0.48 mIoU points.

Environment

Reference environment used during development:

Component	Version
OS	Ubuntu 24.04
Python	3.12
PyTorch	2.7
CUDA	12.x
JAX	0.6
laspy	2.x

Installation:

bash

git clone https://huggingface.co/heig-vd-geo/DeepChoice
cd DeepChoice

conda create -n deepchoice python=3.12
conda activate deepchoice

pip install --upgrade pip
pip install -r requirements.txt

The repository includes a .gitattributes file configured for Git LFS so released .pt and .pth checkpoints can be pushed cleanly.

GridNet-HD Preprocessing

The GridNet-HD preprocessing pipeline expects, for each tile:

a LAS point cloud
oriented images
camera poses and intrinsics / extrinsics
per-image logits or softmax scores exported by the 2D baseline

Configuration:

configs/config_gridnet_hd_dataset_generation.yaml

The GridNet-HD preprocessing config expects the official split file distributed with the dataset at ./data/gridnet_hd/public/split.json.

Compute normals:

python scripts/compute_normals.py --config configs/config_gridnet_hd_dataset_generation.yaml

Generate DeepChoice batches:

python scripts/run_batch_generation.py --config configs/config_gridnet_hd_dataset_generation.yaml

For each visible (point, image) pair, the pipeline computes:

incidence angle
distance
local contrast
blur response
signal-to-noise ratio
saturation

Generated batches are written under:

artifacts/gridnet_hd/batches/train
artifacts/gridnet_hd/batches/val
artifacts/gridnet_hd/batches/test

Each .pt batch contains:

visibility: [N, max_views, num_features]
logits: [N, max_views, num_classes]
mask: [N, max_views]
target: [N]
coords_int
coords_scale
coords_offset
coords_tile_offset

Cultural Heritage Preprocessing

Configuration:

configs/config_cultural_heritage_dataset_generation.yaml

Compute normals:

python scripts/compute_normals_cultural_heritage.py --config configs/config_cultural_heritage_dataset_generation.yaml

Generate batches:

python scripts/run_batch_generation_cultural_heritage.py --config configs/config_cultural_heritage_dataset_generation.yaml

This setup is useful for internal experiments and custom split protocols. Exact reproduction of a paper-specific Cultural Heritage protocol may require additional split management.

DeepChoice Training

Main DeepChoice entry point:

main.py

Supported modes:

train
test
infer

Base config:

configs/config_deepchoice_base.yaml

Example single-process training:

python main.py --config configs/config_deepchoice_base.yaml --mode train

Example distributed training:

torchrun --standalone --nproc_per_node=4 main.py --config configs/config_deepchoice_base.yaml --mode train

Useful config sections:

data: batch roots and split-specific overrides
dataset: selected visibility features and normalization ranges
model: backbone choice and architecture hyperparameters
training: optimizer, scheduler, dataloading, and DDP settings
test: evaluation and inference split settings

Important training options:

training.file_batch_size
training.eval_file_batch_size
training.train_limit_files
training.randomize_train_limit_each_epoch
training.val_split

Evaluation

Example evaluation with baselines:

bash

python main.py \
  --config configs/config_deepchoice_base.yaml \
  --mode test \
  --weights_path /path/to/best_model.pt \
  --split test

The test pipeline reports:

model mIoU and mF1
hard_vote
mean_prob_vote
anyview

Inference and LAS Export

Standard inference:

bash

python main.py \
  --config configs/config_deepchoice_base.yaml \
  --mode infer \
  --weights_path /path/to/best_model.pt \
  --split test \
  --inference_output /path/to/output_dir

Inference with an additional comparison model:

bash

python main.py \
  --config /path/to/transformer_config.yaml \
  --mode infer \
  --weights_path /path/to/transformer_best_model.pt \
  --comparison_config /path/to/mlp_config.yaml \
  --comparison_weights_path /path/to/mlp_best_model.pt \
  --comparison_field_name best_mlp \
  --split test \
  --inference_output /path/to/output_dir

Generated outputs:

pred_batch_*.pt
one LAS file per tile

Each LAS may contain:

classification: primary model prediction
ground_truth
best_transformer
best_mlp
mean_prob_vote
hard_vote

Image Baseline

The repository also contains the 2D image segmentation baseline used to export per-image semantic scores:

image_baseline/main.py

Provided configs:

image_baseline/configs/gridnet_hd.yaml
image_baseline/configs/cultural_heritage.yaml

Train:

python image_baseline/main.py --config image_baseline/configs/gridnet_hd.yaml --mode train

Validate:

bash

python image_baseline/main.py \
  --config image_baseline/configs/gridnet_hd.yaml \
  --mode val \
  --weights_path /path/to/best_model.pth

Export logits / softmax scores:

bash

python image_baseline/main.py \
  --config image_baseline/configs/gridnet_hd.yaml \
  --mode export_logits \
  --weights_path /path/to/best_model.pth

The exported arrays are stored as .npy files following the dataset-relative image structure.

Experiment Runners

Reusable experiment runners:

experiments/run_weekend_experiments.py
experiments/run_validation_experiments.py

Paper-related plans included in the repository:

experiments/weekend_vislogits_nobalanced_plan.yaml
experiments/weekend_vislogits_maxviews_plan.yaml
experiments/test_vislogits_best_plan.yaml
experiments/weekend_vislogits_cultural_heritage_plan.yaml
experiments/test_vislogits_cultural_heritage_best_plan.yaml

Typical launch:

bash

python experiments/run_weekend_experiments.py \
  --base-config configs/config_deepchoice_base.yaml \
  --plan /path/to/weekend_plan.yaml \
  --output-root /path/to/output_root \
  --continue-on-error

GridNet-HD input ablation:

bash

python experiments/run_weekend_experiments.py \
  --base-config configs/config_deepchoice_base.yaml \
  --plan experiments/weekend_vislogits_nobalanced_plan.yaml \
  --output-root /path/to/output_root \
  --continue-on-error

GridNet-HD max-views ablation:

bash

python experiments/run_weekend_experiments.py \
  --base-config configs/config_deepchoice_base.yaml \
  --plan experiments/weekend_vislogits_maxviews_plan.yaml \
  --output-root /path/to/output_root \
  --continue-on-error

GridNet-HD final test evaluation:

bash

python experiments/run_validation_experiments.py \
  --base-config configs/config_deepchoice_base.yaml \
  --plan experiments/test_vislogits_best_plan.yaml \
  --output-root /path/to/output_root

Cultural Heritage experiment:

bash

python experiments/run_weekend_experiments.py \
  --base-config configs/config_deepchoice_base.yaml \
  --plan experiments/weekend_vislogits_cultural_heritage_plan.yaml \
  --output-root /path/to/output_root \
  --continue-on-error

Cultural Heritage evaluation of the released best models:

bash

python experiments/run_validation_experiments.py \
  --base-config configs/config_deepchoice_base.yaml \
  --plan experiments/test_vislogits_cultural_heritage_best_plan.yaml \
  --output-root /path/to/output_root

Utility Scripts

Currently kept in scripts/:

scripts/compute_normals.py
scripts/compute_normals_cultural_heritage.py
scripts/run_batch_generation.py
scripts/run_batch_generation_cultural_heritage.py
scripts/make_visibility_paper_figure.py
scripts/convert_cultural_heritage_txt_to_las.py

heig-vd-geo/DeepChoice

DeepChoice

Citation

Repository Layout

Public Config Files

Released Checkpoints

Main Results

GridNet-HD test split

Cultural Heritage (3_SS)

Environment

GridNet-HD Preprocessing

Cultural Heritage Preprocessing

DeepChoice Training

Evaluation

Inference and LAS Export

Image Baseline

Experiment Runners

Utility Scripts

heig-vd-geo/
DeepChoice

Cultural Heritage (`3_SS`)