Abstract
Heuristic parameters such as width and height are usually obtained in audio-visual speech recognition. However, the presence of noise has an impact on such system. In the paper, we present a mathematical study investigating whether descriptive parameters derived from lip shapes can improve the performance of the system through the use of a mathematical model. The video database used consists of five separate pronunciations of the numbers ranging from 0 to 9. Three categories of data have been successfully classified; the polynomial coefficient (curving of the lips), width and height (both inner and outer) and also the raw data (coordinates). The results showed that the best classifier is the curving of the bottom lip contour with an accuracy of 90.91% and the weakest classifier is from points on the right upper lip contour with accuracy of 12.24%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Liu, H.: Study on lipreading recognition based on computer vision. In: Proceedings of the 2nd International Conference on Information Engineering and Computer Science (2010)
Liu, X., Cheung, Y.: A robust lip tracking algorithm using localized color active contours and deformable models. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1197–1200 (2011)
ur Rehman Butt, W., Lombardi, L.: A survey of automatic lip reading approaches. In: Proceednigs of the Eighth International Conference on Digital Information Management (ICDIM 2013), pp. 299–302 (2013)
Yargic, A., Dogan, M.: A lip reading application on MS Kinect camera. In: IEEE International Symposium on Innovations in Intelligent Systems and Applications, IEEE INISTA, pp. 1–5 (2013)
Ibrahim, M.Z.: A novel lip geometry approach for audio-visual speech recognition (2014)
Chi, E.C., Scott, D.W.: Robust Parametric Classification and Variable Selection by a Minimum Distance Criterion. Journal of Computational and Graphical Statistics 23, 111–128 (2014)
Essenwanger, O.: Curve Fitting. Wiley StatsRef: Statistics Reference Online (2014)
Bowden, R., Cox, S., Harvey, R., Lan, Y., Ong, E.J., Theobald, B.J.: Recent developments in automated lip-reading. In: Proc. SPIE 8901, Optics and Photonics for Counterterrorism, Crime Fighting and Defence IX; and Optical Materials and Biomaterials in Security and Defence Systems Technology X (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Zahari, A.M.R.P.H. (2016). Lip-Reading: Toward Phoneme Recognition Through Lip Kinematics. In: Lavangnananda, K., Phon-Amnuaisuk, S., Engchuan, W., Chan, J. (eds) Intelligent and Evolutionary Systems. Proceedings in Adaptation, Learning and Optimization, vol 5. Springer, Cham. https://doi.org/10.1007/978-3-319-27000-5_33
Download citation
DOI: https://doi.org/10.1007/978-3-319-27000-5_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26999-3
Online ISBN: 978-3-319-27000-5
eBook Packages: EngineeringEngineering (R0)