Chao Zhang
ABSTRACT
Human segmentation using point clouds requires clustering of points belonging to the same human body part. In the supervised learning scenario, previous studies can segment the human body parts to some extent. However, segmentation easily fails for complex postures, especially for the parts with a wide range of motion (e.g., parts from the hand to the upper arm). To alleviate this problem, first, the Random Vertex Displacement (RVD) filter is applied to an existing human body point clouds dataset to augment the training data. Specifically, the RVD filter creates a sphere with a given radius centered on each point that constitutes the human point cloud. The point is randomly shifted within the sphere for augmentation. The model trained with the RVD augmented data is treated as the teacher network. Second, we train a student network from scratch to generate the same intermediate representation to mimic the teacher network. In the experiment, the teacher network improves the average IoU by around 2%, and to our surprise, the student network further outperforms the teacher by another 2%, which well validates the effectiveness of the proposed two-stage scheme for human segmentation.
- Jertec Andrej, Bojani David, Bartol Kristijan, Pribani Tomislav, Petkovi Tomislav, and Petrak Slavenka. 2019. On using PointNet Architecture for Human Body Segmentation. In ISPA. 253–257.Google Scholar
- Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis. 2005. SCAPE : shape completion and animation of people. In SIGGRAPH. ACM, 408–416.Google Scholar
- Federica Bogo, Javier Romero, Matthew Loper, and Michael J. Black. 2014. FAUST: Dataset and evaluation for 3D mesh registration. In CVPR. IEEE, 3794–3801.Google Scholar
- Yunlu Chen, Vincent Tao Hu, Efstratios Gavves, Thomas Mensink, Pascal Mettes, Pengwan Yang, and Cees GM Snoek. 2020. Pointmixup: Augmentation for point clouds. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16. Springer, 330–345.Google Scholar
- Xiang Deng and Zhongfei Zhang. 2021. Can Students Outperform Teachers in Knowledge Distillation based Model Compression?https://openreview.net/forum?id=XZDeL25T12lGoogle Scholar
- Tommaso Furlanello, Zachary Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. 2018. Born again neural networks. In International Conference on Machine Learning. PMLR, 1607–1616.Google Scholar
- Daniela Giorgi, Silvia Biasotti, , and Laura Paraboschi. 2007. Shape retrieval contest200. In Watertight models track.SHREC competition8. NIST.Google Scholar
- Kan Guo, Dongqing Zou, and Xiaowu Chen. 2015. 3D mesh labeling via deep convolutional neural networks. ACM Transactions on Graphics (TOG) 35, 1 (2015), 1–12.Google ScholarDigital Library
- Niv Haim, Nimrod Segol, Heli Ben-Hamu, Haggai Maron, and Yaron Lipman. 2019. Surface networks via general covers. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 632–641.Google ScholarCross Ref
- Evangelos Kalogerakis, Aaron Hertzmann, and Karan Singh. 2010. Learning 3D mesh segmentation and labeling. In ACM SIGGRAPH 2010 papers. 1–12.Google ScholarDigital Library
- Damian Krawczyk and Robert Sitnik. 2023. Segmentation of 3D Point Cloud Data Representing Full Human Body Geometry: A Review. Pattern Recognition (2023), 109444.Google ScholarDigital Library
- Ruihui Li, Xianzhi Li, Pheng-Ann Heng, and Chi-Wing Fu. 2020. Pointaugment: an auto-augmentation framework for point cloud classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6378–6387.Google ScholarCross Ref
- Mixamo. 2008. Adobe Fuse3D Characters. Retrieved October 30, 2020 from https://www.mixamo.comGoogle Scholar
- Maoying Qiao, Jun Cheng, Wei Bian, and Dacheng Tao. 2014. Biview learning for human posture segmentation from 3D points cloud. PloS one 9, 1 (2014), e85811.Google ScholarCross Ref
- Qi Charles R, Su Hao, Mo Kaichun, and Guibas Leonidas J. 2017. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR. 652–660.Google Scholar
- Ayça Takmaz, Jonas Schult, Irem Kaftan, Mertcan Akçay, Robert Sumner, Bastian Leibe, Francis Engelmann, and Siyu Tang. 2022. 3D Segmentation of Humans in Point Clouds with Synthetic Data. arXiv preprint arXiv:2212.00786 (2022).Google Scholar
- Ueshima Takuma, Hotta Katsuya, Tokai Shogo, and Zhang Chao. 2021. Training PointNet for human point cloud segmentation with 3D meshes. In Fifteenth International Conference on Quality Control by Artificial Vision, Vol. 11794. SPIE, 72–77.Google Scholar
- Wenming Tang and Guoping Qiu. 2021. Dense graph convolutional neural networks on 3D meshes for 3D object segmentation and classification. Image and Vision Computing 114 (2021), 104265.Google ScholarDigital Library
- Nitika Verma, Edmond Boyer, and Jakob Verbeek. 2018. Feastnet: Feature-steered graph convolutions for 3d shape analysis. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2598–2606.Google ScholarCross Ref
- Daniel Vlasic, Ilya Baran, Wojciech Matusik, and Jovan Popović. 2008. Articulated meshanimation from multi-view silhouettes. In SIGGRAPH. ACM.Google Scholar
- Dongbo Zhang, Zheng Fang, Xuequan Lu, Hong Qin, Antonio Robles-Kelly, Chao Zhang, and Ying He. 2020. Deep Patch-Based Human Segmentation. In Neural Information Processing: 27th International Conference, ICONIP 2020, Bangkok, Thailand, November 23–27, 2020, Proceedings, Part I 27. Springer, 229–240.Google Scholar
Index Terms
- Augment with Teacher and Distill with Student: A Two-Stage Teacher-Student Network Training Scheme for 3D Human Segmentation
Recommendations
A self-training teacher-student model with an automatic label grader for abdominal skeletal muscle segmentation
AbstractDeep learning on a limited number of labels/annotations is a challenging task for medical imaging analysis. In this paper, we propose a novel self-training segmentation pipeline (Self-Seg in short) for segmenting skeletal muscle in CT ...
Highlights- We designed a semi-supervised learning pipeline for skeletal muscle segmentation.
Semi-supervised Learning via Improved Teacher-Student Network for Robust 3D Reconstruction of Stereo Endoscopic Image
MM '21: Proceedings of the 29th ACM International Conference on Multimedia3D reconstruction of stereo endoscope image, as an enabling technique for varied surgical systems, e.g., medical droids, navigations, etc., suffers from severe overfitting problems due to scarce labels. Semi-supervised learning based on Teacher-Student ...
Teacher-Student Semi-supervised Approach for Medical Image Segmentation
Fast and Low-Resource Semi-supervised Abdominal Organ SegmentationAbstractAccurate segmentation of anatomical structures is a critical step for medical image analysis. Deep learning architectures have become the state-of-the-art models for automatic medical image segmentation. However, these models require an extensive ...
Comments