Abstract
This paper proposes a novel head pose estimation scheme that is based on image and wavelets input and conducts a coarse to fine regression. As wavelets provide low-level shape abstractions, we add them as extra channels to the input to help the neural network to make better estimation and converge. We design a coarse-to-fine regression framework that makes coarse-grained head pose classification followed by fine-grained angles estimation. This framework helps alleviate the influence of biased training sample distribution, and combines segment-wise mappings to form a better global fitting. Further, multiple streams are used in the neural network to extract a rich feature set for robust and accurate regression. Experiments show that the proposed method outperforms the state-of-the-art methods of the same type for the head pose estimation task.
Z. Li, W. Li—Both authors have contributed equally to this work.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Balasubramanian, V.N., Ye, J., Panchanathan, S.: Biased manifold embedding: a framework for person-independent head pose estimation. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–7. IEEE (2007)
Bulat, A., Tzimiropoulos, G.: How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230,000 3D facial landmarks). In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1021–1030 (2017)
Chang, F.J., Tuan Tran, A., Hassner, T., Masi, I., Nevatia, R., Medioni, G.: Faceposenet: making a case for landmark-free face alignment. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1599–1608 (2017)
Fanelli, G., Dantone, M., Gall, J., Fossati, A., Van Gool, L.: Random forests for real time 3D face analysis. Int. J. Comput. Vis. 101(3), 437–458 (2013). https://doi.org/10.1007/s11263-012-0549-0
Fanelli, G., Weise, T., Gall, J., Van Gool, L.: Real time head pose estimation from consumer depth cameras. In: Mester, R., Felsberg, M. (eds.) DAGM 2011. LNCS, vol. 6835, pp. 101–110. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23123-0_11
Gu, J., Yang, X., De Mello, S., Kautz, J.: Dynamic facial analysis: from Bayesian filtering to recurrent neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1548–1557 (2017)
Hu, N., Huang, W., Ranganath, S.: Head pose estimation by non-linear embedding and mapping. In: IEEE International Conference on Image Processing 2005. vol. 2, pp. II-342. IEEE (2005)
Hu, Y., Ji, C., Zhang, Q., Chen, L., Zhan, P., Li, X.: A novel multi-resolution representation for time series sensor data analysis. Soft. Comput. 24(14), 10535–10560 (2019). https://doi.org/10.1007/s00500-019-04562-7
Hu, Y., Ren, P., Luo, W., Zhan, P., Li, X.: Multi-resolution representation with recurrent neural networks application for streaming time series in IoT. Comput. Netw. 152, 114–132 (2019)
Hu, Y., Zhan, P., Xu, Y., Zhao, J., Li, Y., Li, X.: Temporal representation learning for time series classification. Neural Comput. Appl. 32, 1–13 (2020). https://doi.org/10.1007/s00521-020-05179-w
Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)
Koestinger, M., Wohlhart, P., Roth, P.M., Bischof, H.: Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV workshops), pp. 2144–2151. IEEE (2011)
Kumar, A., Alavi, A., Chellappa, R.: Kepler: keypoint and pose estimation of unconstrained faces by learning efficient H-CNN regressors. In: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), pp. 258–265. IEEE (2017)
Li, S., et al.: Deep-learning inversion of seismic data. IEEE Trans. Geosci. Remote Sens. 58(3), 2135–2149 (2020)
Liu, B., et al.: Deep learning inversion of electrical resistivity data. IEEE Trans. Geosci. Remote Sensing 58, 1–14 (2020). https://doi.org/10.1109/TGRS.2020.2969040
Mallat, S.: A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell. 11(7), 674–693 (1989)
Martin, M., Van De Camp, F., Stiefelhagen, R.: Real time head model creation and head pose estimation on consumer depth cameras. In: 2014 2nd International Conference on 3D Vision, vol. 1, pp. 641–648. IEEE (2014)
Meyer, G.P., Gupta, S., Frosio, I., Reddy, D., Kautz, J.: Robust model-based 3D head pose estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3649–3657 (2015)
Osuna, E., Freund, R., Girosit, F.: Training support vector machines: an application to face detection. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 130–136. IEEE (1997)
Ranjan, R., Patel, V.M., Chellappa, R.: Hyperface: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Trans. Pattern Anal. Mach. Intell. 41(1), 121–135 (2017)
Rowley, H.A., Baluja, S., Kanade, T.: Neural network-based face detection. IEEE Trans. Pattern Anal. Mach. Intell. 20(1), 23–38 (1998)
Ruiz, N., Chong, E., Rehg, J.M.: Fine-grained head pose estimation without keypoints. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2074–2083 (2018)
Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 397–403 (2013)
Song, X., et al.: Apollocar3D: a large 3D car instance understanding benchmark for autonomous driving. In: The IEEE Conference on Computer Vision and Pattern Recognition, CVPR, June 2019
Srinivasan, S., Boyer, K.L.: Head pose estimation using view based eigenspaces. In: Object Recognition Supported by User Interaction for Service Robots, vol. 4, pp. 302–305. IEEE (2002)
Yang, T.Y., Chen, Y.T., Lin, Y.Y., Chuang, Y.Y.: FSA-NET: learning fine-grained structure aggregation for head pose estimation from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1087–1096 (2019)
Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2879–2886. IEEE (2012)
Zhu, X., Lei, Z., Liu, X., Shi, H., Li, S.Z.: Face alignment across large poses: a 3D solution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 146–155 (2016)
Zhu, X., Liu, X., Lei, Z., Li, S.Z.: Face alignment in full pose range: a 3D total solution. IEEE Trans. Pattern Anal. Mach. Intell. 41, 78–92 (2019)
Acknowledgments
This paper is supported by “The Fundamental Research Funds of Shandong University”, NSFC of China (61907026) and Shandong Province Higher Educational Science and Technology Program (J18KA392).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, Z. et al. (2020). WC2FEst-Net: Wavelet-Based Coarse-to-Fine Head Pose Estimation from a Single Image. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12532. Springer, Cham. https://doi.org/10.1007/978-3-030-63830-6_53
Download citation
DOI: https://doi.org/10.1007/978-3-030-63830-6_53
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63829-0
Online ISBN: 978-3-030-63830-6
eBook Packages: Computer ScienceComputer Science (R0)