Towards Learning to Complete Anything in Lidar
We propose CAL (Complete Anything in Lidar) for Lidar-based shape-completion
in-the-wild. This is closely related to Lidar-based semantic/panoptic scene
completion. However, contemporary methods can only complete and recognize
objects from a closed vocabulary labeled in existing Lidar datasets. Different
to that, our zero-shot approach leverages the temporal context from multi-modal
sensor sequences to mine object shapes and semantic features of observed
objects. These are then distilled into a Lidar-only instance-level completion
and recognition model. Although we only mine partial shape completions, we find
that our distilled model learns to infer full object shapes from multiple such
partial observations across the dataset. We show that our model can be prompted
on standard benchmarks for Semantic and Panoptic Scene Completion, localize
objects as (amodal) 3D bounding boxes, and recognize objects beyond fixed class
vocabularies. Our project page is
https://research.nvidia.com/labs/dvl/projects/complete-anything-lidar
68010305e501b14512a3f5aa
Ayca Takmaz
claimed_verified
2025-04-18T09:39:50.743Z
6499631f906b89ee66f1fb81
/avatars/fe27062dfc20cf57169d596457d0ef92.svg
Ayca Takmaz
user
aycatakmaz
68010305e501b14512a3f5ab
Cristiano Saltori
68010305e501b14512a3f5ac
Neehar Peri
68010305e501b14512a3f5ad
Tim Meinhardt
68010305e501b14512a3f5ae
Riccardo de Lutio
68010305e501b14512a3f5af
Laura Leal-Taixé
68010305e501b14512a3f5b0
Aljoša Ošep
Towards Learning to Complete Anything in Lidar
We propose CAL (Complete Anything in Lidar) for Lidar-based shape-completion
in-the-wild. This is closely related to Lidar-based semantic/panoptic scene
completion. However, contemporary methods can only complete and recognize
objects from a closed vocabulary labeled in existing Lidar datasets. Different
to that, our zero-shot approach leverages the temporal context from multi-modal
sensor sequences to mine object shapes and semantic features of observed
objects. These are then distilled into a Lidar-only instance-level completion
and recognition model. Although we only mine partial shape completions, we find
that our distilled model learns to infer full object shapes from multiple such
partial observations across the dataset. We show that our model can be prompted
on standard benchmarks for Semantic and Panoptic Scene Completion, localize
objects as (amodal) 3D bounding boxes, and recognize objects beyond fixed class
vocabularies. Our project page is
https://research.nvidia.com/labs/dvl/projects/complete-anything-lidar
CAL uses multi-modal sensor sequences for zero-shot Lidar-based shape completion and recognition, extending beyond fixed class vocabularies.
Lidar zero-shot semantic scene completion panoptic scene completion multi-modal instance-level shape completion 3D bounding boxes