Abstract
Face alignment is to localize multiple facial landmarks for a given facial image. While convolutional neural networks based face alignment methods have achieved superior performance in recent years, the problem remains unresolved due to the fact that L2 loss function suffers from imbalance errors of different facial components caused by region-related changes in pose, illumination and occlusions. In this situation, the L2 loss function will be dominated by errors from those facial components on which the landmarks are hard predicted. To alleviate this issue, in this paper, we propose a facial landmarks detection method based on branched convolutional neural networks, which consists of the shared layers and component-aware branches. The proposed model first captures more dependencies among facial components through the shared layers. Then, by virtue of the knowledge of component-aware branches, different sub-networks can effectively detect the facial landmarks of different components. A series of empirical experiments are carried out on a benchmark dataset across different facial variations. Compared with the existing state-of-the-arts, our proposed method not only achieves the robustness with respect to normal face and occlusion, but also effectively improves the performance of detecting landmarks on corresponding facial components.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: Robust discriminative response map fitting with constrained local models. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3444–3451 (2013)
Burgos-Artizzu, X.P., Perona, P.: Robust face landmark estimation under occlusion. In: IEEE International Conference on Computer Vision, pp. 1513–1520 (2013)
Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. Int. J. Comput. Vis. 107(2), 177–190 (2014)
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: European Conference on Computer Vision, pp. 484–498 (1998)
Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models-their training and application. Comput. Vis. Image Underst. 61(1), 38–59 (1995)
Dollar, P., Welinder, P., Perona, P.: Cascaded pose regression. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1078–1085 (2010)
He, K., Xue, X.: Facial landmark localization by part-aware deep convolutional network. In: Chen, E., Gong, Y., Tie, Y. (eds.) PCM 2016. LNCS, vol. 9916, pp. 22–31. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48890-5_3
Jia, Y., et al:: Caffe: convolutional architecture for fast feature embedding. In: ACM International Conference on Multimedia, pp. 675–678. ACM (2014)
Kass, M., Witkin, A.P., Terzopoulos, D.: Snakes: active contour models. Int. J. Comput. Vis. 1(4), 321–331 (1988)
Lai, H., et al.: Deep recurrent regression for facial landmark detection. IEEE Trans. Circuits Syst. Video Technol. (2016)
Liu, H., Lu, J., Feng, J., Zhou, J.: Learning deep sharable and structural detectors for face alignment. IEEE Trans. Image Process. 26(4), 1666–1678 (2017)
Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2879–2886 (2012)
Ren, S., Cao, X., Wei, Y., Sun, J.: Face alignment via regressing local binary features. IEEE Trans. Image Process. 25(3), 1233 (2016)
Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: IEEE International Conference on Computer Vision Workshops, pp. 397–403 (2014)
Sun, P., Min, J.K., Xiong, G.: Globally tuned cascade pose regression via back propagation with application in 2D face pose estimation and heart segmentation in 3D CT images. In: IEEE International Conference on Computer and Information Technology Workshops, pp. 462–467 (2015)
Sun, Y., Wang, X., Tang, X.: Deep convolutional network cascade for facial point detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3476–3483 (2013)
Xing, J., Niu, Z., Huang, J., Hu, W., Xi, Z., Yan, S.: Towards robust and accurate multi-view and partially-occluded face alignment. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1 (2017)
Xiong, X., Torre, F.D.L.: Supervised descent method and its applications to face alignment. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 532–539 (2013)
Zhang, J., Shan, S., Kan, M., Chen, X.: Coarse-to-fine auto-encoder networks (CFAN) for real-time face alignment. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 1–16. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10605-2_1
Zhang, Q., Zhang, L.: Face alignment with two-layer shape regression. In: Ho, Y.-S., Sang, J., Ro, Y.M., Kim, J., Wu, F. (eds.) PCM 2015. LNCS, vol. 9314, pp. 125–134. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24075-6_13
Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Learning deep representation for face alignment with auxiliary attributes. IEEE Trans. Pattern Anal. Mach. Intell. 38(5), 918–930 (2016)
Zhu, S., Li, C., Chen, C.L., Tang, X.: Face alignment by coarse-to-fine shape searching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4998–5006 (2015)
Acknowledgments
This work is supported by Shenzhen Science and Technology Innovation Commission (SZSTI) project (No. JCYJ20170302153752613).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhu, M., Shi, D., Chen, S., Gao, J. (2018). Branched Convolutional Neural Networks for Face Alignment. In: Hong, R., Cheng, WH., Yamasaki, T., Wang, M., Ngo, CW. (eds) Advances in Multimedia Information Processing – PCM 2018. PCM 2018. Lecture Notes in Computer Science(), vol 11166. Springer, Cham. https://doi.org/10.1007/978-3-030-00764-5_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-00764-5_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00763-8
Online ISBN: 978-3-030-00764-5
eBook Packages: Computer ScienceComputer Science (R0)