Dense semantic RGB-D SLAM using a Semantic Gaussian Field (SG-Field). Each Gaussian stores position, covariance, opacity, RGB color, and a 16-dimensional semantic feature vector. A lightweight 1x1 Conv2d decoder maps semantic features to per-pixel class logits.

Core components:

SG-Field: 3D Gaussian representation with semantic features
Differentiable Splatting: CUDA-accelerated rendering via gaussian-semantic-rasterization
Tracking: Frame-to-model pose optimization (rotation + translation)
Mapping: Joint optimization of Gaussians + semantic decoder
DSR: Depth-adaptive Scale Regularization
RSKM: Random Sampling-based Keyframe Mapping

Exported Formats

Format	File	Use Case
SafeTensors (field)	`pytorch/gs3lam_office2_v1_field.safetensors`	Gaussian field parameters
SafeTensors (decoder)	`pytorch/gs3lam_office2_v1_decoder.safetensors`	Semantic decoder weights
ONNX (decoder)	`onnx/gs3lam_office2_v1_decoder.onnx`	Cross-platform decoder inference
Poses (npy)	`pytorch/gs3lam_office2_v1_poses.npy`	Estimated camera trajectory [N,4,4]
Checkpoint	`checkpoints/best.pt`	Full checkpoint for resuming

Note: TensorRT engines must be generated on target hardware due to architecture-specific compilation.

Training Details

Scene: Replica office2 (2000 frames, 1200x680 rendered at 600x340)
Hardware: NVIDIA L4 23GB
Config: 20 tracking iterations, 15 mapping iterations (L4-tuned)
Gaussians: 21K (capped at 80K with periodic pruning)
Duration: ~5.5 hours
CUDA Extension: gaussian-semantic-rasterization (sm_89)

Current Metrics (L4 baseline, reduced resolution)

Metric	Value	Paper Target
PSNR	3.39 dB	>= 35.0 dB
ATE	178 cm	<= 0.50 cm

Note: These metrics reflect an L4-constrained run at half resolution with aggressive Gaussian capping (21K vs paper's ~775K). Paper-quality reproduction requires full resolution on A100/H100 hardware.

Usage

python

import torch
from anima_slam_gs3lam.export import load_checkpoint, reconstruct_field, reconstruct_decoder

ckpt = load_checkpoint("checkpoints/best.pt")
field = reconstruct_field(ckpt, device="cuda")
decoder = reconstruct_decoder(ckpt, device="cuda")

API

bash

# Start service
python -m anima_slam_gs3lam

# Health check
curl http://localhost:8080/health
curl http://localhost:8080/ready
curl http://localhost:8080/info

Docker

docker compose -f docker-compose.serve.yml --profile api up -d

CUDA Extension

The gaussian-semantic-rasterization CUDA kernel is shared at: /mnt/forge-data/shared_infra/cuda_extensions/gaussian_semantic_rasterization/

License

Apache 2.0 — Robot Flow Labs / AIFLOW LABS LIMITED

ilessio-aiflowlab/project_slam_gs3lam

GS3LAM — Gaussian Semantic Splatting SLAM

Paper

Architecture