Abstract
Lip image analysis plays a vital role in Traditional Chinese Medicine (TCM) and other visual and speech recognition applications. However, if the lip images contain weak color difference with background parts or the background is complicated, most of the current methods are difficult to robustly and accurately segment the lip regions. In this paper, we propose a lip segmentation method based on complexion mixture model to resolve this problem. Specifically, we use the pixels’ color of the upper (lip-free) part of the face as training data to build a corresponding complexion Gaussian Mixture Model (GMM) for each face image in Lab color space. Then by iteratively removing the complexion pixels not belonging to the lip region in the lower part of the face based on the GMM, an initial lip can be obtained. We further build GMMs on the initial lip and non-lip regions, respectively. The background probability map can be obtained based on the GMMs. Finally, we extract the optimal lip contour via a smooth operation. Experiments are performed on our dataset with 1000 face images. Experimental results demonstrate the efficacy of the proposed method compared with the state-of-art lip segmentation methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Shin, J., Jun, B., Kim, D.: Robust two-stage lip tracker. In: IEEE International Symposium on Signal Processing and Information Technology, pp. 265–270 (2009)
Zhang, Y., Levinson, S., Huang, T.: Speaker independent audio-visual speech recognition. In: IEEE International Conference on Multimedia and Expo, vol. 2, pp. 1073–1076 (2000)
Lucey, S., Sridharan, S., Chandran, V.: Initialised eigenlip estimator for fast lip tracking using linear regression. In: IEEE International Conference on Pattern Recognition, vol. 3, pp. 178–181 (2000)
Rabi, G., Lu, S.: Visual speech recognition by recurrent neural networks. In: Engineering Innovation: Voyage of Discovery Electrical and Computer Engineering, vol. 1, pp. 55–58. IEEE (1997)
Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., Ng, A.Y.: Multimodal deep learning. In: Proceedings of the 28th International Conference on Machine Learning, pp. 689–696 (2011)
Sun, C., Lu, H., Zhang, W., Qiu, X., Li, F., Zhang, H.: Lip segmentation based on facial complexion template. In: Ooi, W.T., Snoek, C.G.M., Tan, H.K., Ho, C.-K., Huet, B., Ngo, C.-W. (eds.) PCM 2014. LNCS, vol. 8879, pp. 193–202. Springer, Heidelberg (2014). doi:10.1007/978-3-319-13168-9_20
Ghaleh, V.E.C., Behrad, A.: Lip contour extraction using RGB color space and fuzzy c-means clustering. In: IEEE International Conference on Cybernetic Intelligent Systems, pp. 1–4 (2010)
Rahman, F.S., Nath, R., Nath, S., Basak, S., Audin, S.I., Fattah, S.A.: Lip contour extraction scheme based on k-means clustering in different color planes. In: IEEE International Conference on Informatics, Electronics & Vision, pp. 1–4 (2014)
Wang, S.-L., Lau, W.-H., Liew, A.W.-C., Leung, S.-H.: Robust lip region segmentation for lip images with complex background. Pattern Recognit. 40(12), 3481–3491 (2007)
Kalbkhani, H., Amirani, M.C.: An efficient algorithm for lip segmentation in color face images based on local information. J. World Elect. Eng. Technol. 1(1), 12–16 (2012)
Li, M., Cheung, Y.: Automatic segmentation of color lip images based on morphological filter. In: Diamantaras, K., Duch, W., Iliadis, L.S. (eds.) ICANN 2010. LNCS, vol. 6352, pp. 384–387. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15819-3_51
Rother, C., Kolmogorov, V., Blake, A.: “Grabcut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23(3), 309–314 (2004)
Bai, X., Wang, J., David, S., Sapiro, G.: Video snapcut: robust video object cutout using localized classifiers. ACM Trans. Graph. (TOG) 28(3) (2009)
Wang, J., Agrawala, M., Cohen, M.F.: Soft scissors: an interactive tool for realtime high quality matting. In: ACM Transactions on Graphics (TOG), vol. 26. ACM (2007)
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 1, pp. I–511 (2001)
Otsu, N.: A threshold selection method from gray-level histograms. Automatica 11(285–296), 23–27 (1975)
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China (No. 81373555), and Shanghai Committee of Science and Technology (14JC1402202 and 14441904403).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Hu, Y., Lu, H., Cheng, J., Zhang, W., Li, F., Zhang, W. (2016). Robust Lip Segmentation Based on Complexion Mixture Model. In: Chen, E., Gong, Y., Tie, Y. (eds) Advances in Multimedia Information Processing - PCM 2016. PCM 2016. Lecture Notes in Computer Science(), vol 9916. Springer, Cham. https://doi.org/10.1007/978-3-319-48890-5_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-48890-5_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-48889-9
Online ISBN: 978-3-319-48890-5
eBook Packages: Computer ScienceComputer Science (R0)