Skip to main content
SpringerLink
Log in

Deep Mixture of MRFs for Human Pose Estimation

  • Conference paper
  • First Online:
Computer Vision – ACCV 2018 (ACCV 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11363))

Included in the following conference series:

  • 3251 Accesses

Abstract

In this paper, we propose a new geometric model based on mixture of Markov Random Fields (MRFs) for human pose estimation. We build on previous work that expresses the global constraints on the relative locations of the body joints using an auto-encoder ConvNet which performs dimensionality reduction on the heat maps, and recovers in this manner a low dimensional manifold on which the global pose of the human body lies. To address the shortcomings of this architecture, and obtain more meaningful vectors that span the low dimensional pose space, we propose to replace the auto-encoder network layer with a layer that implements a Gaussian mixture model (GMM) that provides a soft clustering of the human pose predictions in an online fashion. We show that: (a) a large number of meaningful global poses is feasible, this way preserving the underlying structure of informative body poses; (b) the clustering helps to properly initialize the MRF filters of each different global pose, (c) a body joint masking data augmentation procedure can be better exploited and that (d) the system stability is significantly improved. To the best of our knowledge, this is the first time that a clustering algorithm like GMM is used in an online fashion for the problem of 2D human pose estimation. The efficacy of our framework has been demonstrated through extensive experiments on widely used public benchmarks.

I. Marras and P. Palasek—This work was done while the authors were at Queen Mary University of London.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Andriluka, M., Pishchulin, L., Gehler, P., Bernt, S.: 2D human pose estimation: new benchmark and state of the art analysis. In: CVPR, pp. 3686–3693, June 2014

    Google Scholar 

  2. Arthur, D., Vassilvitskii, S.: k-means++: the advantages of careful seeding. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1027–1035. Society for Industrial and Applied Mathematics (2007)

    Google Scholar 

  3. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)

    MATH  Google Scholar 

  4. Bulat, A., Tzimiropoulos, G.: Human pose estimation via convolutional part heatmap regression. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 717–732. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_44

    Chapter  Google Scholar 

  5. Carreira, J., Agrawal, P., Fragkiadaki, K., Malik, J.: Human pose estimation with iterative error feedback. In: CVPR, pp. 4733–4742 (2016)

    Google Scholar 

  6. Chen, X., Yuille, A.L.: Articulated pose estimation by a graphical model with image dependent pairwise relations. In: NIPS, pp. 1736–1744 (2014)

    Google Scholar 

  7. Chen, Y., Shen, C., Wei, X., Liu, L., Yang, J.: Adversarial PoseNet: a structure-aware convolutional network for human pose estimation. In: ICCV, pp. 1212–1221 (2017)

    Google Scholar 

  8. Chu, X., Yang, W., Ouyang, W., Ma, C., Yuille, A.L., Wang, X.: Multi-context attention for human pose estimation. In: CVPR, pp. 1831–1840 (2017)

    Google Scholar 

  9. Dantone, M., Gall, J., Leistner, C., Van Gool, L.: Body parts dependent joint regressors for human pose estimation in still images. PAMI 36, 2131–2143 (2014)

    Article  Google Scholar 

  10. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B (Methodol.) 39, 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  11. Dilokthanakul, N., et al.: Deep unsupervised clustering with Gaussian mixture variational autoencoders. arXiv preprint arXiv:1611.02648 (2016)

  12. Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. JMLR 12, 2121–2159 (2011)

    MathSciNet  MATH  Google Scholar 

  13. Felzenszwalb, P., McAllester, D., Ramanan, D.: A discriminatively trained, multiscale, deformable part model. In: CVPR, pp. 1–8 (2008)

    Google Scholar 

  14. Gkioxari, G., Arbeláez, P., Bourdev, L., Malik, J.: Articulated pose estimation using discriminative armlet classifiers. In: CVPR, pp. 3342–3349 (2013)

    Google Scholar 

  15. Gkioxari, G., Toshev, A., Jaitly, N.: Chained predictions using convolutional neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 728–743. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_44

    Chapter  Google Scholar 

  16. Güler, R., Neverova, N., Kokkinos, I.: Densepose: dense human pose estimation in the wild. In: CVPR (2018)

    Google Scholar 

  17. Hu, P., Ramanan, D.: Bottom-up and top-down reasoning with convolutional latent-variable models. In: CVPR, pp. 5600–5609 (2016)

    Google Scholar 

  18. Insafutdinov, E., Pishchulin, L., Andres, B., Andriluka, M., Schiele, B.: DeeperCut: a deeper, stronger, and faster multi-person pose estimation model. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 34–50. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_3

    Chapter  Google Scholar 

  19. Jain, A., Tompson, J., LeCun, Y., Bregler, C.: MoDeep: a deep learning framework using motion features for human pose estimation. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9004, pp. 302–315. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16808-1_21

    Chapter  Google Scholar 

  20. Johnson, S., Everingham, M.: Clustered pose and nonlinear appearance models for human pose estimation. In: BMVC, pp. 12.1–12.11 (2010)

    Google Scholar 

  21. Johnson, S., Everingham, M.: Learning effective human pose estimation from inaccurate annotation. In: CVPR, pp. 1465–1472 (2011)

    Google Scholar 

  22. Ke, L., Chang, M.-C., Qi, H., Lyu, S.: Multi-scale structure-aware network for human pose estimation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11206, pp. 731–746. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01216-8_44

    Chapter  Google Scholar 

  23. Krapac, J., Verbeek, J., Jurie, F.: Modeling spatial layout with Fisher vectors for image categorization. In: ICCV, pp. 1487–1494. IEEE (2011)

    Google Scholar 

  24. Lifshitz, I., Fetaya, E., Ullman, S.: Human pose estimation using deep consensus voting. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 246–260. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_16

    Chapter  Google Scholar 

  25. Luvizon, D.C., Tabia, H., Picard, D.: Human pose regression by combining indirect part detection and contextual information. CoRR (2017)

    Google Scholar 

  26. Marras, I., Palasek, P., Patras, I.: Deep globally constrained MRFs for human pose estimation. In: ICCV, pp. 3466–3475 (2017)

    Google Scholar 

  27. Neal, R.M., Hinton, G.E.: A view of the EM algorithm that justifies incremental, sparse, and other variants. In: Jordan, M.I. (ed.) Learning in Graphical Models, pp. 355–368. Springer, Dordrecht (1998). https://doi.org/10.1007/978-94-011-5014-9_12

    Chapter  Google Scholar 

  28. Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29

    Chapter  Google Scholar 

  29. Nie, X., Feng, J., Zuo, Y., Yan, S.: Human pose estimation with parsing induced learner. In: CVPR, pp. 2100–2108 (2018)

    Google Scholar 

  30. Palasek, P., Patras, I.: Discriminative convolutional Fisher vector network for action recognition. arXiv preprint arXiv:1707.06119 (2017)

  31. Peng, X., Tang, Z., Yang, F., Feris, R.S., Metaxas, D.: Jointly optimize data augmentation and network training: adversarial data augmentation in human pose estimation. In: CVPR, pp. 2226–2234 (2018)

    Google Scholar 

  32. Pfister, T., Simonyan, K., Charles, J., Zisserman, A.: Deep convolutional neural networks for efficient pose estimation in gesture videos. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9003, pp. 538–552. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16865-4_35

    Chapter  Google Scholar 

  33. Pishchulin, L., et al.: Deepcut: joint subset partition and labeling for multi person pose estimation. In: CVPR, pp. 4929–4937 (2016)

    Google Scholar 

  34. Sapp, B., Taskar, B.: Modec: multimodal decomposable models for human pose estimation. In: CVPR, pp. 3674–3681 (2013)

    Google Scholar 

  35. Tieleman, T., Hinton, G.: Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw. Mach. Learn. 4(2), 26–31 (2012)

    Google Scholar 

  36. Tompson, J., Goroshin, R., Jain, A., LeCun, Y., Bregler, C.: Efficient object localization using convolutional networks. In: CVPR, pp. 648–656 (2015)

    Google Scholar 

  37. Tompson, J., Jain, A., LeCun, Y., Bregler, C.: Joint training of a convolutional network and a graphical model for human pose estimation. In: NIPS, pp. 1799–1807 (2014)

    Google Scholar 

  38. Toshev, A., Szegedy, C.: Deeppose: human pose estimation via deep neural networks. In: CVPR, pp. 1653–1660 (2014)

    Google Scholar 

  39. Tüske, Z., Tahir, M.A., Schlüter, R., Ney, H.: Integrating Gaussian mixtures into deep neural networks: softmax layer with hidden variables. In: ICASSP, pp. 4285–4289 (2015)

    Google Scholar 

  40. Variani, E., McDermott, E., Heigold, G.: A Gaussian mixture model layer jointly optimized with discriminative features within a deep neural network architecture. In: ICASSP, pp. 4270–4274 (2015)

    Google Scholar 

  41. Wei, S.E., Ramakrishna, V., Kanade, T., Sheikh, Y.: Convolutional pose machines. In: CVPR, pp. 4724–4732 (2016)

    Google Scholar 

  42. Yang, Y., Ramanan, D.: Articulated human detection with flexible mixtures of parts. PAMI 35(12), 2878–2890 (2013)

    Article  Google Scholar 

  43. Zong, B., et al.: Deep autoencoding Gaussian mixture model for unsupervised anomaly detection. In: ICLR (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronic Engineering and Computer Science, Queen Mary University of London, London, UK

    Ioannis Marras, Petar Palasek & Ioannis Patras

  2. Huawei Noah’s Ark Lab, London, UK

    Ioannis Marras

  3. MindVisionLabs Ltd, London, UK

    Petar Palasek

Authors
  1. Ioannis Marras
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Petar Palasek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ioannis Patras
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ioannis Marras .

Editor information

Editors and Affiliations

  1. IIIT Hyderabad, Hyderabad, India

    C. V. Jawahar

  2. ANU, Canberra, ACT, Australia

    Hongdong Li

  3. Simon Fraser University, Burnaby, BC, Canada

    Greg Mori

  4. ETH Zurich, Zurich, Zürich, Switzerland

    Konrad Schindler

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Marras, I., Palasek, P., Patras, I. (2019). Deep Mixture of MRFs for Human Pose Estimation. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds) Computer Vision – ACCV 2018. ACCV 2018. Lecture Notes in Computer Science(), vol 11363. Springer, Cham. https://doi.org/10.1007/978-3-030-20893-6_45

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-20893-6_45

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-20892-9

  • Online ISBN: 978-3-030-20893-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation