Abstract
We address single-view 3D shape classification with partial Point Cloud Data (PCD) inputs. Conventional PCD classifiers achieve the best performance when trained and evaluated with complete 3D object scans. However, they all experience a performance drop when trained and evaluated on partial single-view PCD. We propose a Single-View PCD Classifier (SVP-Classifier), which first hallucinates the features of other viewpoints covering the unseen part of the object with a Conditional Variational Auto-Encoder (CVAE). It then aggregates the hallucinated multi-view features with a multi-level Graph Convolutional Network (GCN) to form a global shape representation that helps to improve the single-view PCD classification performance. With experiments on the single-view PCDs generated from ModelNet40 and ScanObjectNN, we prove that the proposed SVP-Classifier outperforms the best single-view PCD-based methods, after they have been retrained on single-view PCDs, thus reducing the gap between single-view methods and methods that employ complete PCDs. Code and datasets are available: https://github.com/IIT-PAVIS/SVP-Classifier.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
An, J., Cho, S.: Variational autoencoder based anomaly detection using reconstruction probability. Spec. Lect. IE 2(1), 1–18 (2015)
Angelina Uy, M., Pham, Q.H., Hua, B.S., Thanh Nguyen, D., Yeung, S.K.: Revisiting point cloud classification: a new benchmark dataset and classification model on real-world data. arXiv, arXiv-1908 (2019)
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
Dong, G., Liao, G., Liu, H., Kuang, G.: A review of the autoencoder and its variants: a comparative perspective from target recognition in synthetic-aperture radar images. IEEE Geosci. Remote Sens. Mag. 6(3), 44–68 (2018)
Ioannidou, A., Chatzilari, E., Nikolopoulos, S., Kompatsiaris, I.: Deep learning advances in computer vision with 3D data: a survey. ACM Comput. Surv. (CSUR) 50(2), 1–38 (2017)
Kingma, D., Welling, M.: Auto-encoding variational Bayes. In: ICLR 2014 (2014)
Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B.: PointCNN: convolution on x-transformed points. Adv. Neural Inf. Process. Syst. 31, 820–830 (2018)
Mohammadi, S.S., Wang, Y., Del Bue, A.: PointView-GCN: 3D shape classification with multi-view point clouds. In: 2021 IEEE International Conference on Image Processing (ICIP), pp. 3103–3107. IEEE (2021)
Qi, C.R., Liu, W., Wu, C., Su, H., Guibas, L.J.: Frustum PointNets for 3D object detection from RGB-D data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 918–927 (2018)
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017)
Qi, C.R., Su, H., Nießner, M., Dai, A., Yan, M., Guibas, L.J.: Volumetric and multi-view CNNs for object classification on 3D data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5648–5656 (2016)
Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, pp. 5099–5108 (2017)
Sohn, K., Lee, H., Yan, X.: Learning structured output representation using deep conditional generative models. Adv. Neural Inf. Process. Syst. 28, 3483–3491 (2015)
Vishwanath, K.V., Gupta, D., Vahdat, A., Yocum, K.: ModelNet: towards a datacenter emulation environment. In: Proceedings of the IEEE Ninth International Conference on Peer-to-Peer Computing, pp. 81–82. IEEE (2009)
Wang, Y., Carletti, M., Setti, F., Cristani, M., Bue, A.D.: Active 3D classification of multiple objects in cluttered scenes. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pp. 2602–2610 (2019)
Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. ACM Trans. Graph. (TOG) 38(5), 1–12 (2019)
Xu, Y., Fan, T., Xu, M., Zeng, L., Qiao, Yu.: SpiderCNN: deep learning on point sets with parameterized convolutional filters. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11212, pp. 90–105. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01237-3_6
Yuniarti, A., Suciati, N.: A review of deep learning techniques for 3D reconstruction of 2D images. In: 2019 12th International Conference on Information & Communication Technology and System (ICTS), pp. 327–331. IEEE (2019)
Zhao, Y., Birdal, T., Deng, H., Tombari, F.: 3D point capsule networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1009–1018 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mohammadi, S.S., Wang, Y., Taiana, M., Morerio, P., Del Bue, A. (2022). SVP-Classifier: Single-View Point Cloud Data Classifier with Multi-view Hallucination. In: Sclaroff, S., Distante, C., Leo, M., Farinella, G.M., Tombari, F. (eds) Image Analysis and Processing – ICIAP 2022. ICIAP 2022. Lecture Notes in Computer Science, vol 13232. Springer, Cham. https://doi.org/10.1007/978-3-031-06430-2_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-06430-2_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06429-6
Online ISBN: 978-3-031-06430-2
eBook Packages: Computer ScienceComputer Science (R0)