Abstract
Difficulty in pattern recognition is perceptible and neural networks approach the problem by way of learning from similar known patterns. Interest in Neural Networks started in the early 1980s when they were deemed to effectively model the human thought process. Speech recognition which first used Artificial Neural Networks (ANNs) to model the states of a Hidden Markov Models (HMMs) later started using Gaussian Mixture Models (GMMs). GMM-HMM systems have been the standard until recently when a new concept of Deep Neural Networks (DNNs) pre-trained using Restricted Boltzmann Machines (RBMs) came into existence. The discriminative capability of the resulting DNN is found to improve the performance of the recognition systems. The experimental work with DNN for recognizing patterns in handwriting and speech corpus has been carried out. In this work we implemented Deep Neural Networks for the above tasks and the pre trained DNN has been used for extracting bottleneck features and hereby improving the performance of the baseline systems with respect to recognition errors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dong, Y., Seide, F., Li, G., Deng, L.: Exploiting sparseness in deep neural networks for large vocabulary speech recognition. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 4409–4412 (2012)
Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)
Lee, H., Largman, Y., Pham, P., Ng, A.Y.: Unsupervised feature learning for audio classification using convolutional deep belief networks. In: Advances in Neural Information Processing Systems, pp. 1096–1104 (2009)
Price, P., Fisher, W.M., Bernstein, J., Pallett, D.S.: Resource Management RM2 2.0. Linguistic Data Consortium, Philadelphia (1993)
Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hannemann, M., Motlicek, P., Qian, Y., Schwarz, P., Silovsky, J.: The KALDI-speech recognition toolkit. In: Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding (2011)
Lishuang, Z., Zhiyan, H.: Speech recognition system based on integrating feature and HMM. In: Proceedings of the International Conference on Measuring Technology and Mechatronics Automation, vol. 3, pp. 449–452 (2010)
Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., Bengio, Y.: Theano: A CPU and GPU math expression compiler. In: Proceedings of the Conference on Python for Scientific Computing, pp. 1–7 (2010)
Grezl, F., Karafiat, M., Kontar, S., Cernocky, J.: Probabilistic and bottle-neck features for LVCSR of meetings. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 4, pp. 757–760 (2007)
Dahl, G.E., Ranzato, M., Mohamed, A., Hinton, G.E.: Phone recognition with the mean-covariance restricted Boltzmann machine. In: Advances in Neural Information Processing Systems (2010)
Mohamed, A., Yu, D., Deng, L.: Investigation of full-sequence training of deep belief networks for speech recognition. In: Proceedings of the Interspeech (2010)
Sainath, T.N., Ramabhadran, B., Picheny, M., Nahamoo, D., Kanevsky, D.: Exemplar-based sparse representation features: from TIMIT to LVCSR. IEEE Trans. Audio Speech Lang. Process. 19(8), 2598–2613 (2011)
Henry Charles, A.P., Devaraj, G.: Alaigal-A Tamil speech recognition. In: Tamil Internet Conference, Singapore (2004)
Abdel-Hamid, O., Mohamed, A., Jiang, H., Penn, G.: Applying convolutional neural networks concepts to hybrid NN-HMM model for speech recognition. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 4277–4280 (2012)
He, X., Deng, L., Chou, W.: Discriminative learning in sequential pattern recognition—a unifying review for optimization-oriented speech recognition. IEEE Signal Process. Mag. 25(5), 1436 (2008)
Bengio, Y., De Mori, R., Flammia, G., Kompe, F.: Global optimization of a neural network hidden Markov model hybrid. In: Proceedings of the EuroSpeech (1991)
Deng, L., Yu, D., Acero, A.: Structured speech modeling. EEE Trans. Audio Speech Lang. Process. 14, 1492–1504 (2006)
Dahl, G., Yu, D., Deng, L., Acero, A.: Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. EEE Trans. Audio Speech Lang. Process. 20(1), 30–42 (2012)
Siede, F., Li, G., Yu, D.: Conversational speech transcription using context-dependent deep neural networks. In: Proceedings of the Interspeech, pp. 437–440 (2011)
Yu, D., Deng, L., Li, G., Seide, F.: Discriminative pre-training of deep neural networks. U.S. Patent (2011)
Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.-A.: Stacked denoising auto encoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)
Rifai, S., Vincent, P., Muller, X., Glorot, X., Bengio, Y.: Contracting auto encoders: explicit invariance during feature extraction. In: Proceedings of the 28th International Conference on Machine Learning (2011)
Plahl, C., Sainath, T.N., Ramabhadran, B., Nahamoo, D.: Improved pre-training of deep belief networks using sparse encoding symmetric machines. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (2012)
Yu, D., Seltzer, M.L.: Improved bottleneck features using pre-trained deep neural networks. In: Proceedings of the Interspeech, pp. 237–240 (2011)
Hinton, G., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Mohamed, A., Hinton, G.: Phone recognition using restricted Boltzmann machines. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 4354–4357 (2010)
Hinton, G.: A Practical Guide to Training Restricted Boltzmann Machines. UTML TR 2010-003 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Padmanabhan, J., Melvin Jose Premkumar, J. (2018). Advanced Deep Neural Networks for Pattern Recognition: An Experimental Study. In: Abraham, A., Cherukuri, A., Madureira, A., Muda, A. (eds) Proceedings of the Eighth International Conference on Soft Computing and Pattern Recognition (SoCPaR 2016). SoCPaR 2016. Advances in Intelligent Systems and Computing, vol 614. Springer, Cham. https://doi.org/10.1007/978-3-319-60618-7_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-60618-7_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-60617-0
Online ISBN: 978-3-319-60618-7
eBook Packages: EngineeringEngineering (R0)