skip to main content
10.1145/3591156.3591162acmotherconferencesArticle/Chapter ViewAbstractPublication PagesivspConference Proceedingsconference-collections
research-article

A Human Body Part Semantic Segmentation Enabled Parsing for Human Pose Estimation

Published: 16 June 2023 Publication History

Abstract

Human Body Part Semantic Segmentation and Human Pose estimation are considered to be essential for understanding human behaviours. Both of these tasks are correlated with each other. Employing them together in a unified framework to perform two distinct Human Centric Visual Analysis tasks simultaneously allows benefiting from each other. Taking advantage of the correlation between Human Body Part Semantic Segmentation and Human Pose Estimation, this paper proposes a unified framework that explores efficient context modelling. The framework simultaneously predicts the human body part semantic segmentation and pose estimation with high-quality results. The results extracted from the segmentation are used to predict the pose estimation task. An experimental analysis of the proposed framework is done on the benchmark LIP Dataset. The analysis of the results shows that the proposed framework outperforms the state-of-the-art by 7.3% when evaluated on mean IoU. Moreover, Mean Accuracy, Pixel Accuracy and PCKh are the other metrics used for the evaluation of the framework.

References

[1]
Israr Akhter, Ahmad Jalal, and Kibum Kim. 2021. Pose estimation and detection for event recognition using Sense-Aware features and Adaboost classifier. In 2021 International Bhurban Conference on Applied Sciences and Technologies (IBCAST). IEEE, 500–505.
[2]
Anam Arshad, Vivek Tiwari, Mayank Lovanshi, and Rahul Shrivastava. 2023. Role Identification from Human Activity Videos using Recurrent Neural Networks. In proceedings of the 8th IEEE International Women in Engineering (WIE) Conference on Electrical and Computer Engineering (WIECON-ECE).
[3]
Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks 5, 2 (1994), 157–166.
[4]
Kunal Bose, Kumar Shubham, Vivek Tiwari, and Kuldip Singh Patel. 2023. Insect Image Semantic Segmentation and Identification Using UNET and DeepLab V3+. In ICT Infrastructure and Computing. Springer, 703–711.
[5]
Markus Braun, Qing Rao, Yikang Wang, and Fabian Flohr. 2016. Pose-rcnn: Joint object detection and pose estimation using 3d object proposals. In 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 1546–1551.
[6]
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. 2017. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence 40, 4 (2017), 834–848.
[7]
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. 2017. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence 40, 4 (2017), 834–848.
[8]
Yilun Chen, Zhicheng Wang, Yuxiang Peng, Zhiqiang Zhang, Gang Yu, and Jian Sun. 2018. Cascaded pyramid network for multi-person pose estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7103–7112.
[9]
Xiao Chu, Wei Yang, Wanli Ouyang, Cheng Ma, Alan L Yuille, and Xiaogang Wang. 2017. Multi-context attention for human pose estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1831–1840.
[10]
Matthias Dantone, Juergen Gall, Christian Leistner, and Luc Van Gool. 2013. Human pose estimation using body parts dependent joint regressors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3041–3048.
[11]
Jian Dong, Qiang Chen, Xiaohui Shen, Jianchao Yang, and Shuicheng Yan. 2014. Towards unified human parsing and pose estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 843–850.
[12]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
[13]
Ruyi Ji, Dawei Du, Libo Zhang, Longyin Wen, Yanjun Wu, Chen Zhao, Feiyue Huang, and Siwei Lyu. 2020. Learning semantic neural tree for human parsing. In European Conference on Computer Vision. Springer, 205–221.
[14]
Leonid Karlinsky and Shimon Ullman. 2012. Using linking features in learning non-parametric part models. In European Conference on Computer Vision. Springer, 326–339.
[15]
Xiaodan Liang, Ke Gong, Xiaohui Shen, and Liang Lin. 2018. Look into person: Joint body parsing & pose estimation network and a new benchmark. IEEE transactions on pattern analysis and machine intelligence 41, 4 (2018), 871–885.
[16]
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In European conference on computer vision. Springer, 740–755.
[17]
Mayank Lovanshi and Vivek Tiwari. 2023. Human Pose Estimation: Benchmarking Deep Learning-based Methods. In proceedings of the IEEE Conference on Interdisciplinary Approaches in Technology and Management for Social Innovation.
[18]
Yawei Luo, Zhedong Zheng, Liang Zheng, Tao Guan, Junqing Yu, and Yi Yang. 2018. Macro-micro adversarial network for human parsing. In Proceedings of the European conference on computer vision (ECCV). 418–434.
[19]
Alejandro Newell, Kaiyu Yang, and Jia Deng. 2016. Stacked hourglass networks for human pose estimation. In European conference on computer vision. Springer, 483–499.
[20]
Xuecheng Nie, Jiashi Feng, and Shuicheng Yan. 2018. Mutual learning to adapt for joint human parsing and pose estimation. In Proceedings of the European Conference on Computer Vision (ECCV). 502–517.
[21]
Wanli Ouyang, Xiao Chu, and Xiaogang Wang. 2014. Multi-source deep learning for human pose estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2329–2336.
[22]
Varun Ramakrishna, Daniel Munoz, Martial Hebert, James Andrew Bagnell, and Yaser Sheikh. 2014. Pose machines: Articulated pose estimation via inference machines. In European Conference on Computer Vision. Springer, 33–47.
[23]
Mrigank Rochan 2018. Future semantic segmentation with convolutional lstm. arXiv preprint arXiv:1807.07946 (2018).
[24]
Tao Ruan, Ting Liu, Zilong Huang, Yunchao Wei, Shikui Wei, and Yao Zhao. 2019. Devil in the details: Towards accurate single and multiple human parsing. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 4814–4821.
[25]
Rahul Shrivastava, Vivek Tiwari, Swati Jain, Basant Tiwari, Alok Kumar Singh Kushwaha, and Vibhav Prakash Singh. 2022. A role-entity based human activity recognition using inter-body features and temporal sequence memory. IET Image Processing (2022).
[26]
Jonathan Tompson, Ross Goroshin, Arjun Jain, Yann LeCun, and Christoph Bregler. 2015. Efficient object localization using convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 648–656.
[27]
Chengjia Wang, Tom MacGillivray, Gillian Macnaught, Guang Yang, and David Newby. 2018. A two-stage 3D Unet framework for multi-class segmentation on full resolution image. arXiv preprint arXiv:1804.04341 (2018).
[28]
Wenguan Wang, Hailong Zhu, Jifeng Dai, Yanwei Pang, Jianbing Shen, and Ling Shao. 2020. Hierarchical human parsing with typed part-relation reasoning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8929–8939.
[29]
Shih-En Wei, Varun Ramakrishna, Takeo Kanade, and Yaser Sheikh. 2016. Convolutional pose machines. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 4724–4732.
[30]
Yongpeng Wu, Dehui Kong, Shaofan Wang, Jinghua Li, and Baocai Yin. 2022. HPGCN: Hierarchical poselet-guided graph convolutional network for 3D pose estimation. Neurocomputing 487 (2022), 243–256.
[31]
Fangting Xia, Peng Wang, Xianjie Chen, and Alan L Yuille. 2017. Joint multi-person pose estimation and semantic part segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 6769–6778.
[32]
Fangting Xia, Peng Wang, Xianjie Chen, and Alan L Yuille. 2017. Joint multi-person pose estimation and semantic part segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 6769–6778.
[33]
Yang Xu and Ting Ting Qiu. 2021. Human activity recognition and embedded application based on convolutional neural network. Journal of Artificial Intelligence and Technology 1, 1 (2021), 51–60.
[34]
Yuhui Yuan, Lang Huang, Jianyuan Guo, Chao Zhang, Xilin Chen, and Jingdong Wang. 2018. Ocnet: Object context network for scene parsing. arXiv preprint arXiv:1809.00916 (2018).
[35]
Dan Zeng, Yuhang Huang, Qian Bao, Junjie Zhang, Chi Su, and Wu Liu. 2021. Neural Architecture Search for Joint Human Parsing and Pose Estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 11385–11394.
[36]
Feng Zhang, Xiatian Zhu, and Mao Ye. 2019. Fast human pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3517–3526.
[37]
Xiaomei Zhang, Yingying Chen, Bingke Zhu, Jinqiao Wang, and Ming Tang. 2020. Blended grammar network for human parsing. In European Conference on Computer Vision. Springer, 189–205.
[38]
Xiaomei Zhang, Yingying Chen, Bingke Zhu, Jinqiao Wang, and Ming Tang. 2020. Part-aware context network for human parsing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8971–8980.
[39]
Ziwei Zhang, Chi Su, Liang Zheng, and Xiaodong Xie. 2020. Correlating edge, pose with parsing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8900–8909.

Cited By

View all
  • (2024)Multidimensional Impacts of Generative AI and an In-Depth Analysis of LLMs with Their Expanding Horizons in Technology and Society2024 OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 4.010.1109/OTCON60325.2024.10687980(1-6)Online publication date: 5-Jun-2024
  • (2024)Advanced Machine Learning Techniques for Data Prediction in WSNs2024 OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 4.010.1109/OTCON60325.2024.10687955(1-6)Online publication date: 5-Jun-2024
  • (2024)Unveiling Human Actions: Vision-Based Activity Recognition Using ConvLSTM and LRCN Models2024 OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 4.010.1109/OTCON60325.2024.10687817(1-6)Online publication date: 5-Jun-2024
  • Show More Cited By

Index Terms

  1. A Human Body Part Semantic Segmentation Enabled Parsing for Human Pose Estimation

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      IVSP '23: Proceedings of the 2023 5th International Conference on Image, Video and Signal Processing
      March 2023
      207 pages
      ISBN:9781450398381
      DOI:10.1145/3591156
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 16 June 2023

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Human parsing
      2. Human part semantic segmentation
      3. Human pose detection
      4. Look into person

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      IVSP 2023

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)91
      • Downloads (Last 6 weeks)18
      Reflects downloads up to 15 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Multidimensional Impacts of Generative AI and an In-Depth Analysis of LLMs with Their Expanding Horizons in Technology and Society2024 OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 4.010.1109/OTCON60325.2024.10687980(1-6)Online publication date: 5-Jun-2024
      • (2024)Advanced Machine Learning Techniques for Data Prediction in WSNs2024 OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 4.010.1109/OTCON60325.2024.10687955(1-6)Online publication date: 5-Jun-2024
      • (2024)Unveiling Human Actions: Vision-Based Activity Recognition Using ConvLSTM and LRCN Models2024 OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 4.010.1109/OTCON60325.2024.10687817(1-6)Online publication date: 5-Jun-2024
      • (2024)A Comprehensive Framework for Evaluating Cyber-Physical Threats in Energy Internet2024 International Conference on Intelligent Systems and Advanced Applications (ICISAA)10.1109/ICISAA62385.2024.10828794(1-7)Online publication date: 25-Oct-2024
      • (2024)Estimation of Human Fall Detectionusing Body Pose Rule based Algorithm for Video Surveillance2024 IEEE 9th International Conference for Convergence in Technology (I2CT)10.1109/I2CT61223.2024.10544121(1-5)Online publication date: 5-Apr-2024
      • (2024)Investigating the Potential of Quantum Communication in Secure Networking2024 IEEE 13th International Conference on Communication Systems and Network Technologies (CSNT)10.1109/CSNT60213.2024.10546208(313-321)Online publication date: 6-Apr-2024
      • (2024)Bringing Theoretical Concepts to Life in the Design and Implementation of Hardware for Edge Intelligence in Web3.02024 IEEE 13th International Conference on Communication Systems and Network Technologies (CSNT)10.1109/CSNT60213.2024.10546129(421-428)Online publication date: 6-Apr-2024
      • (2024)Automated Malware Classification Using Deep Learning Neural Networks2024 IEEE 13th International Conference on Communication Systems and Network Technologies (CSNT)10.1109/CSNT60213.2024.10546010(206-212)Online publication date: 6-Apr-2024
      • (2024)Enhancing the Efficiency of Computational Genetic Epidemiology using Advanced Method2024 IEEE 13th International Conference on Communication Systems and Network Technologies (CSNT)10.1109/CSNT60213.2024.10545992(655-662)Online publication date: 6-Apr-2024
      • (2024)Bio-Inspired Algorithms for Self-Healing and Adaptation in Sensor Networks2024 IEEE 13th International Conference on Communication Systems and Network Technologies (CSNT)10.1109/CSNT60213.2024.10545949(35-41)Online publication date: 6-Apr-2024
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media