Authors:
Alexandr Notchenko
;
Vladislav Ishimtsev
;
Alexey Artemov
;
Vadim Selyutin
;
Emil Bogomolov
and
Evgeny Burnaev
Affiliation:
Skolkovo Institute of Science and Technology, Moscow, Russian Federation
Keyword(s):
Semantic Segmentation, Scene Understanding, Volumetric Scenes, Part-level Segmentation.
Abstract:
We propose Scan2Part, a method to segment individual parts of objects in real-world, noisy indoor RGB-D scans. To this end, we vary the part hierarchies of objects in indoor scenes and explore their effect on scene understanding models. Specifically, we use a sparse U-Net-based architecture that captures the fine-scale detail of the underlying 3D scan geometry by leveraging a multi-scale feature hierarchy. In order to train our method, we introduce the Scan2Part dataset, which is the first large-scale collection providing detailed semantic labels at the part level in the real-world setting. In total, we provide 242,081 correspondences between 53,618 PartNet parts of 2,477 ShapeNet objects and 1,506 ScanNet scenes, at two spatial resolutions of 2 cm3 and 5 cm3. As output, we are able to predict fine-grained per-object part labels, even when the geometry is coarse or partially missing. Overall, we believe that both our method as well as newly introduced dataset is a stepping stone forw
ard towards structural understanding of real-world 3D environments.
(More)