Neural / differentiable rendering toolkit for high-quality photorealistic 3D reconstruction and novel viewpoint synthesis.

A neural and differentiable rendering toolkit for high-quality photorealistic 3D reconstruction and novel viewpoint synthesis. Addresses computational efficiency and scalability for large-scale, high-resolution scenes. Enables efficient large-scale photorealistic 3D reconstruction, urban reconstruction, and 3D intrinsic decomposition and editing.

• ▸Efficient high-resolution depth rendering (4032×3024, <1 min, 1 GPU)
• ▸Multi-GPU training and rendering
• ▸Adaptive sampling: uniform, importance, random, selective (loss/density-guided)
• ▸Supports COLMAP, 7-Scenes, ScanNet, FalconViz UAV datasets

Winner code for the 7th KAUST-NVIDIA Workshop AI Competition.

Solution and project code for the 7th KAUST-NVIDIA Workshop AI competition, using XGBoost for geospatial data prediction. Won the competition leaderboard.

ECCV 2020 — Reflection Separation via Multi-bounce Polarization State Tracing.

Generalizes reflection removal to real-world complex light interactions. A polarization sensor captures 4 linearly polarized photos simultaneously; a polarization-guided image formation model plus supervised learning for the ray-tracing model yield unprecedented reconstruction quality on real and synthetic data.

SIGGRAPH Asia 2019 — Hierarchical and View-invariant Light Field Segmentation on 4D Ray Graphs.

A new light field segmentation method respecting texture appearance, depth consistency, and occlusion, producing well-shaped segments robust to viewpoint changes. Hierarchical — a single optimization yields a whole hierarchy of segmentations. Introduces a "disjoint tree" data structure for efficient submodular optimization on very large graphs.

Monocular Long-term Target Following on UAVs — FAST algorithm on quadrotors.

Long-term visual tracking on UAVs. Exploits correlation between a frequency tracker and a spatial detector; the novel FAST algorithm achieves robustness (temporal variance/invariance coverage) and efficiency (coarse-to-fine redetection without extra classifier or exhaustive search). Implemented on a quadrotor for indoor/outdoor real-time automatic smooth long-term target following.

Cluster Sensing Superpixel (CSS) — ~5× faster than SOTA with comparable performance.

Cluster Sensing Superpixel (CSS) method where cluster centers have representativeness (local max pixel density) and isolation. CSS identifies centers via pixel density, then integrates superpixel cues into a bipartite graph segmentation framework, applied to microscopy image segmentation. Roughly 5× faster than SOTA with comparable performance.