Deep Learning for 3D Data: Online Segmentation of LiDAR Sequences - Representation of Large Shape Collection

Abstract: I will present two deep learning based approaches for 3D data. I will start by presenting our paper Online Segmentation of LiDAR Sequences: Dataset and Algorithm (ECCV 2022) that introduces HelixNet, a 10 billion point dataset with fine-grained labels, timestamps, and sensor rotation information necessary to accurately assess the real-time readiness of segmentation algorithms. We build on this dataset to propose Helix4D, a compact and efficient spatio-temporal transformer architecture specifically designed for rotating LiDAR sequences. Helix4D operates on acquisition slices corresponding to a fraction of a full sensor rotation, significantly reducing the total latency. Helix4D reaches accuracy on par with the best segmentation algorithms on HelixNet and SemanticKITTI with a reduction of over 5x in terms of latency and 50x in model size. I will then present our method Representing Shape Collections with Alignment-Aware Linear Models (3DV 2021) in which we characterize 3D shapes as affine transformations of linear families learned without supervision, and showcase its advantages on the understanding of large shape collections.