Xuehang Guo
ABSTRACT
In recent years, significant strides have been made in point cloud semantic segmentation, which, however, are unspectacular when the training is deprived of sufficient densely-annotated samples, especially with the face of new classes unseen during the training. Given limited data and unacquainted categories, learning efficiency becomes of great concern to the overall segmentation outcome. To obtain improved segmentation performance under this few-shot training condition, we introduce a bidirectional learning method that allows mutual prototype learning between support set and query set. Specifically, we manage to realize enhanced efficiency by exploiting the support and query sets to a larger extent, effectively extracting information and generating prototypes in two opposite learning orientations. Refined by our method, models are able to achieve better performance in few-shot 3D semantic segmentation tasks without the need of further introducing more parameters that may lead to higher model complexity. To validate our method, we respectively test different models for 1-shot and 5-shot settings on the S3DIS [23] dataset. The remarkably improved IoU scores on unseen classes in the evaluation tests show the effectiveness of our proposed method.
- Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. 2017. PointNet: Deep learning on point sets for 3D classification and segmentation. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, 77-85. https://doi.org/10.1109/CVPR.2017.16.Google Scholar
Cross Ref
- Charles R. Qi, Li Yi, Hao Su, and Leonidas J. Guibas. 2017. PointNet++: Deep hierarchical feature learning on point sets in a metric space. arXiv: 1706.02413. https://doi.org/10.48550/arXiv.1706.02413.Google ScholarCross Ref
- Jake Snell, Kevin Swersky, and Richard S. Zemel. 2017. Prototypical networks for few-shot learning. NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems, December 2017, 4080–4090. arXiv:1703.05175. https://doi.org/10.48550/arXiv.1703.05175.Google ScholarCross Ref
- Na Zhao, Tat-Seng Chua, and Gim Hee Lee. 2021. Few-shot 3D point cloud semantic segmentation. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021, 8869-8878. https://doi.org/10.1109/CVPR46437.2021.00876.Google ScholarCross Ref
- Xian Zhong, Cheng Gu, Wenxin Huang, Lin Li, Shuqin Chen, and Chia-Wen Lin. 2020. Complementing representation deficiency in few-shot image classification: A Meta-Learning approach. 25th International Conference on Pattern Recognition (ICPR), arXiv:2007.10778. https://doi.org/10.48550/arXiv.2007.10778.Google ScholarCross Ref
- Ardhendu Shekhar Tripathi, Martin Danelljan, Luc Van Gool, and Radu Timofte. 2021. Fast Few-Shot classification by Few-Iteration Meta-Learning. Internet Content Rating Association (ICRA). arXiv:2010.00511. https://doi.org/10.48550/arXiv.2010.00511.Google ScholarCross Ref
- Sepp Hochreiter, Arthur S. Younger, and Peter R. Conwell. 2001. Learning to learn using gradient descent. International Conference on Artificial Neural Networks, Springer, 87-94. https://doi.org/10.1007/3-540-44668-0_13.Google ScholarCross Ref
- Jane X. Wang, Zeb Kurth-Nelson, Dhruva Tirumala, Hubert Soyer, Joel Z. Leibo, Remi Munos, Charles Blundell, Dharshan Kumaran, and Matt Botvinick. 2016. Learning to reinforcement learn. arXiv:1611.05763. https://doi.org/10.48550/arXiv.1611.05763.Google ScholarCross Ref
- Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. 2016. Matching networks for one shot learning. Advances in Neural Information Processing Systems, June 2016, 3630-3638. arXiv:1606.04080. https://doi.org/10.48550/arXiv.1606.04080.Google ScholarCross Ref
- Chi Zhang, Guosheng Lin, Fayao Liu, Rui Yao, and Chunhua Shen. 2019. CANet: Class-Agnostic segmentation networks with iterative refinement and attentive Few-Shot learning. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, 5212-5221. https://doi.org/10.1109/CVPR.2019.00536.Google ScholarCross Ref
- Rinu Boney and Alexander Ilin. 2018. Semi-Supervised and active Few-Shot learning with prototypical networks. arXiv:1711.10856. https://doi.org/10.48550/arXiv.1711.10856.Google ScholarCross Ref
- Nanqing Dong and Eric P. Xing. 2018. Few-Shot semantic segmentation with prototype learning. British Machine Vision Conference, September 2018.Google Scholar
- Gen Li, Varun Jampani, Laura Sevilla-Lara, Deqing Sun, Jonghyun Kim, and Joongkyu Kim. 2021. Adaptive prototype learning and allocation for Few-Shot segmentation. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021, 8330-8339. https://doi.org/10.1109/CVPR46437.2021.00823.Google ScholarCross Ref
- Sachin Ravi and Hugo Larochelle. 2017. Optimization as a model for few-shot learning. International Conference on Learning Representations (ICLR).Google Scholar
- Kaixin Wang, Jun Hao Liew, Yingtian Zou, Daquan Zhou, and Jiashi Feng. 2019. PANet: Few-Shot image semantic segmentation with prototype alignment. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 2019, 9196-9205. https://doi.org/10.1109/ICCV.2019.00929.Google ScholarCross Ref
- Xudong Li, Li Feng, Lei Li, and Chen Wang. 2021. Few-shot Meta-learning on Point Cloud for Semantic Segmentation. arXiv:2104.02979. https://doi.org/10.48550/arXiv.2104.02979.Google ScholarCross Ref
- Kate Rakelly, Evan Shelhamer, Trevor Darrell, Alexei A. Efros, and Sergey Levine. 2018. Few-Shot segmentation propagation with guided networks. arXiv:1806.07373. https://doi.org/10.48550/arXiv.1806.07373.Google ScholarCross Ref
- Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, and Justin M. Solomon. 2018. Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics 38(5), January 2018. arXiv:1801.07829. https://doi.org/10.48550/arXiv.1801.07829.Google ScholarCross Ref
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Computation and Language (cs.CL); Machine Learning (cs.LG). arXiv:1706.03762. https://doi.org/10.48550/arXiv.1706.03762.Google ScholarCross Ref
- Mingtao Feng, Liang Zhang, Xuefei Lin, Syed Zulqarnain Gilani, Ajmal Mian. 2019. Point attention network for semantic segmentation of 3D point clouds. arXiv:1909.12663. https://doi.org/10.48550/arXiv.1909.12663.Google ScholarCross Ref
- Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An image is worth 16x16 words: Transformers for image recognition at scale. International Conference on Learning Representations (ICLR). arXiv:2010.11929. https://doi.org/10.48550/arXiv.2010.11929.Google ScholarCross Ref
- Jinlu Liu and Yongqiang Qin. 2020. Prototype refinement network for Few-Shot segmentation. arXiv:2002.03579. https://doi.org/10.48550/arXiv.2002.03579.Google ScholarCross Ref
- Iro Armeni, Ozan Sener, Amir R. Zamir, Helen Jiang and Ioannis Brilakis, Martin Fischer, and Silvio Savarese. 2016. 3D semantic parsing of Large-Scale indoor spaces. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, 1534-1543. https://doi.org/10.1109/CVPR.2016.170.Google ScholarCross Ref
Recommendations
Few-shot 3D Point Cloud Semantic Segmentation with Prototype Alignment
ICMLT '23: Proceedings of the 2023 8th International Conference on Machine Learning TechnologiesSemantic Segmentation for 3D point clouds has made great progress in recent years. Most existing approaches for 3D point cloud segmentation are fully supervised, and they require a large number of well-annotated data for training. The training data is ...
Crossmodal Few-shot 3D Point Cloud Semantic Segmentation
MM '22: Proceedings of the 30th ACM International Conference on MultimediaRecently, few-shot 3D point cloud semantic segmentation methods have been introduced to mitigate the limitations of existing fully supervised approaches, i.e., heavy dependence on labeled 3D data and poor capacity to generalize to new categories. ...
Number-Adaptive Prototype Learning for 3D Point Cloud Semantic Segmentation
Computer Vision – ECCV 2022 WorkshopsAbstract3D point cloud semantic segmentation is one of the fundamental tasks for 3D scene understanding and has been widely used in the metaverse applications. Many recent 3D semantic segmentation methods learn a single prototype (classifier weights) for ...
Comments