Skip to main content
SpringerLink
Log in

Parametric Reshaping of Humans in Videos Incorporating Motion Retargeting

  • Conference paper
  • First Online:
Computer Vision, Pattern Recognition, Image Processing, and Graphics (NCVPRIPG 2017)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 841))

  • 1446 Accesses

Abstract

We propose a system capable of changing the shape of humans in monocular video sequences. Initially, a 3D model is fit over each frame of the video sequence in a spatio-temporally coherent manner, using the feature points provided by the user in a semi-automatic interface and the silhouette correspondences obtained from background subtraction. The 3D morphable model learned from laser scans of different human subjects is used to generate a model having the shape parameters like height, weight, leg length, etc. specified by the user. The deformed model is then retargeted to transfer the semantics of the motion, like step size of the person. This retargeted model is used to perform a body-aware warping of the foreground of each frame. Finally, the warped foreground is composited over the inpainted background. Spatio-temporal consistency is achieved through the combination of automatic pose fitting and body-aware frame warping. Motion retargeting makes the system produce visually pleasing and natural results like the motion of a taller human is higher than that of the human before warping. We have demonstrated the results of shape changes on different subjects with a variety of actions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    SnapCut is implemented as RotoBrush tool in Adobe After Effects.

  2. 2.

    \(\eta (\varPhi , v_{i})\) and \(\rho (\varPhi , v_{i})\) are also written as \(\eta _{i}(\varPhi )\) and \(\rho _{i}(\varPhi )\) respectively for notational convenience.

  3. 3.

    Retargeting is performed using Maya Human IK interface.

  4. 4.

    reshaping_with_retargeting.mp4.

References

  1. Bai, X., Wang, J., Simons, D., Sapiro, G.: Video SnapCut: robust video cutout using localized classifiers. ACM Trans. Graph. (TOG) 28(3) (2009)

    Article  Google Scholar 

  2. Baran, I., Popovic, J.: Automatic rigging and animation of 3D characters. ACM Trans. Graph. (TOG) 29(3) (2007)

    Google Scholar 

  3. Criminisi, A., Perez, P., Toyama, K.: A region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)

    Article  Google Scholar 

  4. Gkalelis, N., Kim, H., Hilton, A., Nikolaidis, N., Pitas, I.: The i3DPost multi-view and 3D human action/interaction. In: CVMP (2009)

    Google Scholar 

  5. Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., Seidel, H.-P.: A statistical model of human pose and body shape. In: Computer Graphics Forum (Proceedings of the Eurographics 2008), vol. 2 (2009)

    Article  Google Scholar 

  6. Jain, A., Thormahlen, T., Seidel, H.-P., Theobalt, C.: MovieReshape: tracking and reshaping of humans in videos. Trans. Graph. 29(6), 148:1–148:10 (2010)

    Article  Google Scholar 

  7. Kraevoy, V., Sheffer, A., Van De Panne, M.: Modelling from contour drawings. In: SBIM (2009)

    Google Scholar 

  8. Kumar, H., Arora, N., Dhaliwal, J.S., Kalra, P., Chaudhuri, P.: Improved interactive reshaping of humans in images. In: 21st International Conference on Computer Graphics, Visualization and Computer Vision (2013)

    Google Scholar 

  9. Levin, A., Lischinski, D., Weiss, Y.: A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228–242 (2008)

    Article  Google Scholar 

  10. Newson, A., Almansa, A., Fradet, M., Gousseau, Y., Perez, P.: Video inpainting of complex scenes. SIAM J. Imaging Sci. 17(4), 1993–2019 (2014)

    Article  MathSciNet  Google Scholar 

  11. Richter, M., Varanasi, K., Hasler, N., Theobalt, C.: Real-time reshaping of humans. In: International Conference on 3D Imaging, Data Processing, Visualization and Transmission (3DIMPVT) (2012)

    Google Scholar 

  12. Starck, J., Hilton, A.: Surface capture for performance based animation. IEEE Comput. Graph. Appl. 27, 21–31 (2007)

    Article  Google Scholar 

  13. Yang, Y., Yu, Y., Zhou, Y., Du, S., Davis, J., Yang, R.: Semantic parametric reshaping of human body models. In: Second International Conference on 3D Vision (2014)

    Google Scholar 

  14. Zhou, S., Fu, H., Liu, L., Cohen-Or, D., Han, X.: Parametric reshaping of human bodies in images. ACM Trans. Graph. (Proc. ACM SIGGRAPH) 29(3), 1–10 (2010). Article No. 126

    Google Scholar 

  15. Schaefer, S., McPhail, T., Warren, J.: Image deformation using moving least squares. ACM Trans. Graph. 25(3), 533–540 (2006)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank University of Surrey (Gkalelis et al. [4]) for permitting us to use their video footages. We thank Hasler et al. [5] and Starck and Hilton [12] for making their datasets available. We also thank Newson et al. [10] for making their source code publicly available. We extend our gratitude to Ganesh and Aswinn for being the test subjects.

Author information

Authors and Affiliations

  1. IIT Delhi, Delhi, India

    Suresh Prakash & Prem Kalra

Authors
  1. Suresh Prakash
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prem Kalra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Suresh Prakash .

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Mandi, Mandi, Himachal Pradesh, India

    Renu Rameshan

  2. Indraprastha Institute of Information Technology, New Delhi, India

    Chetan Arora

  3. Indian Institute of Technology, New Delhi, India

    Sumantra Dutta Roy

1 Electronic supplementary material

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Prakash, S., Kalra, P. (2018). Parametric Reshaping of Humans in Videos Incorporating Motion Retargeting. In: Rameshan, R., Arora, C., Dutta Roy, S. (eds) Computer Vision, Pattern Recognition, Image Processing, and Graphics. NCVPRIPG 2017. Communications in Computer and Information Science, vol 841. Springer, Singapore. https://doi.org/10.1007/978-981-13-0020-2_11

Download citation

  • DOI: https://doi.org/10.1007/978-981-13-0020-2_11

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-13-0019-6

  • Online ISBN: 978-981-13-0020-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation