You can deploy this model and start earning money today!

bdck/
nksr-wrapper

Name: nksr-wrapper
Availability: InStock

2305.19590

Views

No views yet

NKSR Wrapper — Neural Kernel Surface Reconstruction

License

A clean, high-level Python wrapper around Neural Kernel Surface Reconstruction (NKSR) by Huang et al. (CVPR 2023, Highlight).
Drop a .ply or .pcd point cloud in → get a watertight, high-quality triangle mesh out.

📦 What is NKSR?

Neural Kernel Surface Reconstruction is a deep-learning method that turns a raw, sparse, and potentially noisy point cloud into a smooth, watertight 3D mesh. Unlike classical methods (Poisson, Alpha shapes) it learns a continuous implicit surface from data, and unlike vanilla neural fields (NeRF, DeepSDF) it scales to millions of points and generalises across objects, rooms, and outdoor scenes without per-scene training.

The three key innovations

Innovation	Why it matters
Compactly-supported kernel functions	The implicit field is built from local kernel basis functions that have finite support. This makes the linear system sparse, so it can be solved with fast sparse PCG solvers instead of dense matrix inversion. Result: room-scale reconstruction in seconds.
Gradient fitting solve	Instead of only fitting point positions (SDF ≈ 0), NKSR also fits surface normals as gradients of the field. This makes the reconstruction dramatically more robust to noise and outliers.
Minimal training, maximum generalisation	The model is trained once on a mixture of synthetic and real data (the "kitchen-sink" config) and then works out-of-the-box on new scans without any fine-tuning.

🚀 Quick start

1. Install dependencies

NKSR itself contains custom CUDA kernels, so you need a working PyTorch + CUDA environment.

bash

# 1. Clone the original NKSR repo and install it
#    (see https://github.com/nv-tlabs/NKSR for the latest instructions)
git clone https://github.com/nv-tlabs/NKSR.git
cd NKSR
pip install -r requirements.txt
pip install --no-build-isolation package/

# 2. Install this wrapper
pip install -e .

2. One-liner reconstruction

python

from nksr_wrapper import NKSRMeshReconstructor, load_point_cloud, save_mesh

points, normals = load_point_cloud("scan.ply")
recon = NKSRMeshReconstructor(device="cuda:0")
mesh = recon.reconstruct(points, normals, detail_level=1.0)
save_mesh("mesh.ply", mesh.vertices, mesh.faces)

Or use the CLI:

python scripts/reconstruct.py scan.ply mesh.ply --detail 1.0 --mise-iter 1

🔬 How it works (the full pipeline)

If you want to understand what is happening under the hood, here is the step-by-step pipeline that NKSR executes every time you call reconstruct().

Step 0 — Input

You provide:

xyz — (N, 3) point positions
normal — (N, 3) oriented normals (optional but strongly recommended)
sensor — (N, 3) sensor/camera positions (optional; used for normal orientation when normals are missing)

Step 1 — Voxelisation (Sparse Feature Hierarchy)

The input points are splatted into a sparse voxel grid at multiple resolutions (a quad-/octree-like structure called the Sparse Feature Hierarchy, SVH). Instead of a dense 3D array, only occupied voxels are stored. This is what lets NKSR handle millions of points without exploding memory.

Key parameter: voxel_size (default ≈ 0.1 in the pretrained config). One voxel_size unit = one spatial unit in your point-cloud coordinate system.

Step 2 — Feature encoding (PointNet → Sparse 3D U-Net)

PointEncoder — A small PointNet-style ResNet processes the raw points inside each voxel and produces a 32-dim feature vector per voxel.
SparseStructureNet — A sparse 3D convolutional U-Net with skip connections processes these voxel features across multiple scales. It also predicts an adaptive structure: if a region is empty, the network stops subdividing early, saving computation.

Step 3 — Geometry field (Kernel Field)

This is the heart of NKSR.

The network outputs kernel basis parameters at each voxel. At any 3D query point x, the implicit function is evaluated as a weighted sum of compact kernel functions centred on nearby voxels:

f(x) = Σ_i w_i · φ_i(x)

where φ_i is a compact kernel (e.g. Wendland or similar) and w_i are learned weights. Because the kernels have finite support, the sum only involves neighbours within a small radius → sparse linear system.

Step 4 — Sparse linear solve (PCG)

NKSR now solves for the weights w by fitting two constraints:

Position constraint: f(x_j) ≈ 0 on every input point (the surface is the zero level-set).
Normal constraint: ∇f(x_j) ≈ n_j on voxel centres (the gradient of the field matches the surface normal).

These constraints are assembled into a large but sparse linear system and solved with a preconditioned conjugate-gradient (PCG) solver. The normal constraint is the secret sauce: it anchors the gradient of the field, making the reconstruction much less sensitive to noise than methods that only fit positions.

Step 5 — Mask / trimming field (optional)

A secondary field (either a learned UDF — Unsigned Distance Field — or a simple layer field) identifies regions that are outside the true surface. This trims away spurious floaters and fills small holes, producing a clean watertight boundary.

Step 6 — Mesh extraction (Dual Marching Cubes + MISE)

Finally, the zero level-set of the implicit field is turned into triangles:

Dual Marching Cubes (DMC) is run on the dual graph of the sparse voxel hierarchy. DMC produces nicer topology than standard Marching Cubes (fewer skinny triangles, better sharp-feature preservation).
MISE (Multi-resolution IsoSurface Extraction) adaptively subdivides cells that straddle the zero crossing. Each mise_iter doubles the effective resolution in those cells, giving you a crisp mesh without wasting polygons on empty space.

Result: mesh.v (V×3 vertices) and mesh.f (F×3 face indices).

🧰 API Reference

`NKSRMeshReconstructor`

python

class NKSRMeshReconstructor(
    device="cuda:0",
    config="ks",
    chunk_tmp_device="cpu",
)

device — PyTorch device (CUDA strongly recommended).
config — Pretrained model name:
- "ks" — Kitchen-sink (recommended default). Trained on a mixture of synthetic and real scans; generalises to objects, indoor rooms, and outdoor scenes.
- "snet" — ShapeNet objects with normals.
- "snet-wonormal" — ShapeNet objects without normals.
chunk_tmp_device — Where to stash finished chunks when reconstructing huge scenes. "cpu" offloads to system RAM.

`.reconstruct(...)`

python

mesh = recon.reconstruct(
    points,                # (N, 3)  required
    normals=None,          # (N, 3)  optional, strongly recommended
    sensor_positions=None, # (N, 3)  optional, helps orient normals
    colors=None,           # (N, 3)  optional, for colored mesh output

    # Quality / resolution
    detail_level=1.0,      # 0.0 = smooth, 1.0 = max detail
    voxel_size=None,       # override resolution explicitly
    mise_iter=1,           # 0 = base, 1 = 2× in subdivided cells, 2 = 4×

    # Large-scene settings
    chunk_size=-1.0,       # >0 enables out-of-core chunking
    overlap_ratio=0.05,

    # Solver tuning
    solver_max_iter=2000,
    solver_tol=1e-5,
    approx_kernel_grad=False,

    # Normal estimation fallback
    estimate_normals_if_missing=True,
    normal_knn=64,
    normal_drop_threshold_deg=85.0,
)

Returns a MeshResult dataclass with:

.vertices — (V, 3) float array
.faces — (F, 3) int array
.vertex_colors — (V, 3) float array, if colors was provided
.save(path) — convenience method to write PLY/OBJ/GLB via Trimesh

📂 Repository layout

nksr-wrapper/
├── nksr_wrapper/
│   ├── __init__.py          # public API
│   ├── reconstructor.py     # NKSRMeshReconstructor + MeshResult
│   └── io.py                # load_point_cloud, save_mesh
├── scripts/
│   └── reconstruct.py       # CLI entry point
├── examples/
│   ├── quickstart.py        # minimal script
│   └── chunked_reconstruction.py   # large-scene example
├── setup.py
├── requirements.txt
└── README.md

🖥️ CLI Usage

bash

# Basic reconstruction
python scripts/reconstruct.py scan.ply mesh.ply --detail 1.0

# Large scene (chunked)
python scripts/reconstruct.py huge_scan.ply mesh.ply --chunk-size 50.0

# No normals in file — estimate on-the-fly
python scripts/reconstruct.py scan.ply mesh.ply --estimate-normals

# With per-point colors → colored mesh
python scripts/reconstruct.py scan.ply mesh.ply --colors colors.npy --mise-iter 2

🎯 Tips & Troubleshooting

Problem	Solution
Mesh is too noisy / has spikes	Lower `detail_level` (try `0.3`) or increase `voxel_size`
Mesh is too smooth / missing fine detail	Raise `detail_level` (try `1.0`) or set `mise_iter=2`
Out-of-memory on large scans	Use `chunk_size=50.0` and `chunk_tmp_device="cpu"`
Mesh is inside-out	Normals are unoriented. Provide `sensor_positions` or pre-orient normals with Open3D
Reconstruction is very slow	You are probably on CPU. NKSR requires CUDA for the custom sparse kernels.
PLY file has no normals	Use `--estimate-normals` or pass `sensor_positions` to the reconstructor

📚 Citation

If you use NKSR in your research, please cite the original paper:

bibtex

@inproceedings{huang2023nksr,
  title={Neural Kernel Surface Reconstruction},
  author={Huang, Jiahui and Gojcic, Zan and Atzmon, Matan and 
          Litany, Or and Fidler, Sanja and Williams, Francis},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2023}
}

Original code: https://github.com/nv-tlabs/NKSR
Pretrained weights: https://huggingface.co/heiwang1997/nksr-checkpoints

📄 License

This wrapper is released under the MIT License. NKSR itself is under its own license (see the original repository).

Built with ❤️ on top of NVIDIA t-labs' NKSR.

bdck/nksr-wrapper

NKSR Wrapper — Neural Kernel Surface Reconstruction

📦 What is NKSR?

The three key innovations

🚀 Quick start

1. Install dependencies

2. One-liner reconstruction

🔬 How it works (the full pipeline)

Step 0 — Input

Step 1 — Voxelisation (Sparse Feature Hierarchy)

Step 2 — Feature encoding (PointNet → Sparse 3D U-Net)

Step 3 — Geometry field (Kernel Field)

Step 4 — Sparse linear solve (PCG)

Step 5 — Mask / trimming field (optional)

Step 6 — Mesh extraction (Dual Marching Cubes + MISE)

🧰 API Reference

NKSRMeshReconstructor

.reconstruct(...)

📂 Repository layout

🖥️ CLI Usage

🎯 Tips & Troubleshooting

📚 Citation

📄 License

bdck/
nksr-wrapper

`NKSRMeshReconstructor`

`.reconstruct(...)`