Skip to main content
SpringerLink
Log in

Hearing impaired speech recognition: Stockwell features and models

  • Published:
International Journal of Speech Technology Aims and scope Submit manuscript

Abstract

The development of speech recognition system for recognising the speeches of a reasonable person in various languages usually is in fashion and does not involve challenges to be faced by the researchers. For the past ten years, advances are taking place in analysing and recognising the speeches of the hearing impaired because of the deployment of sophisticated processing methods to study the characteristics of speech production. These technological advancements not only pave the way for developing algorithms for recognising the speeches of healthy persons, also recognising the utterances of the hearing impaired. The study on analysing the oral communication skills of hearing-impaired children has received the attention of the researchers, speech pathologists and audiologists to develop assistive tool/system because inadequacy of such skills dramatically affects the social, educational and career opportunities available to them at large. This paper mainly emphasises the need for the development of a more challenging speaker independent speech recognition system for hearing impaired so that the system can respond to the speech uttered by any HI. In this work, Modified Group Delay Features and Stockwell transform cepstral features are used at the front end and vector quantisation (VQ) and Multivariate Hidden Markov Models (MHMM) at the back end for recognising the speeches uttered by any person with hearing disability. Performance of the system is compared for the three modelling techniques VQ, Fuzzy C Means (FCM) clustering and MHMM for the recognition of isolated digits in Tamil. Recognition accuracy is 89.25% and 79.5% for speaker dependent and independent speech recognition system for the hearing impaired. Performance of the system reveals that this system may be deployed to understand the speeches uttered by any hearing impaired speaker, improve the social status of the people with hearing impairment and mitigate the social stigma in leading a normal life.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Arunachalam, R. (2019). A strategic approach to recognize the speech of the children with hearing impairment: Different sets of features and models. International Journal on Multimedia Tools and Applications,78(15), 20787–20808.

    Article  Google Scholar 

  • Brookes, C. (2000). Speech-to-text systems for deaf, deafened and hard-of-hearing people. In IEE seminar on speech and language processing for disabled and elderly people. Ref. No. 2000/025. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=846943.

  • Chee, L. S., Ai, O. C., & Hariharan, M. (2009). MFCC based recognition of repetitions and prolongations in stuttered speech using k-NN and LDA. In IEEE student conference on research and development (SCOReD) (Vol. 1, pp. 146–149).

  • Deller, J. R., Hsu, D., & Ferrier, L. J. (1991). On the use of hidden Markov modeling for recognition of dysarthric speech. International Journal on Computer Methods and Programs in Biomedicine,35(2), 125–139.

    Article  Google Scholar 

  • Girgin, M. C., & Ozsoy, B. (2008). The relationship between formant frequency and duration characteristics of vowels and speech intelligibility in Turkish hearing impaired children. World Applied Sciences Journal,4(6), 891–899.

    Google Scholar 

  • Gudi, A. B., Shreedhar, H. K., & Nagaraj, H. C. (2010). Signal processing techniques to estimate the speech disability in children. International Journal of Engineering and Technology,2(2), 169–176.

    Google Scholar 

  • Han, Z., Wang, X., & Wang, J. (2008). Pathological speech deformation degree assessment based on dynamic and static feature integration. In The 2nd international conference on bioinformatics and biomedical engineering (ICBBE) (pp. 2036–2039). https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4535718.

  • Han, Z., Wang, X., & Wang, J. (2009). Speech recognition system based on visual features and neural network for persons with speech impairments. International Journal on Modeling, Identification and Control,8(3), 240–247.

    Article  Google Scholar 

  • Hawley, M. S., Cunningham, S. P., Green, P. D., Enderby, P., Palmer, R., Sehgal, S., et al. (2012). A voice-input voice-output communication aid for people with severe speech Impairment. IEEE Transactions on Neural Systems and Rehabilitation Engineering,21(1), 23–31.

    Article  Google Scholar 

  • Healy, E. W., Vasko, J. L., & Wang, D. (2019). The optimal threshold for removing noise from speech is similar across normal and impaired hearing—a time-frequency masking study. The Journal of the Acoustical Society of America,145(6), EL581.

    Article  Google Scholar 

  • Hegde, R. M., Murthy, H. A., & Gadde, V. R. R. (2007). Significance of the modified group delay feature in speech recognition. IEEE Transactions on Audio, Speech and Language Processing,15(1), 190–202.

    Article  Google Scholar 

  • Jamil, M. H. M., Al-Haddad, S. A. R., & Ng, C. K. (2011). A flexible speech recognition system for cerebral palsy disabled. International Conference on Informatics Engineering and Information Science,251, 42–55.

    Article  Google Scholar 

  • Jeyalakshmi, C., Krishnamurthi, V., & Revathi, A. (2010a). Deaf speech assessment using digital processing techniques. International Journal On Signal And Image Processing,1(1), 14–25.

    Article  Google Scholar 

  • Jeyalakshmi, C., Krishnamurthi, V., & Revathi, A. (2010). Speech recognition of deaf and hard of hearing people using hybrid neural network. In 2nd International conference on mechanical and electronics engineering (ICMEE) (Vol. 1, pp. 83–87).

  • Jeyalakshmi, C., Krishnamurthi, V., & Revathi, A. (2014). Development of speech recognition system in native language for hearing impaired. Journal of Engineering Research, 2(2), Article 6.

  • Jeyalakshmi, C., & Revathi, A. (2018a). Efficient speech recognition system for hearing impaired children in classical Tamil language. International Journal of Biomedical Engineering and Technology (IJBET),26(1), 84.

    Article  Google Scholar 

  • Jeyalakshmi, C., & Revathi, A. (2018b). Efficient speech recognition system for hearing impaired children in classical Tamil language. Int. J. Biomedical Engineering and Technology,26(1), 84–100.

    Article  Google Scholar 

  • Jeyalakshmi, C., Revathi, A., & Krishnamurthi, V. (2012). Building robust HMM models for speech recognition of hearing impaired. International Journal on EE Times-India (pp. 1–11).

  • Jeyalakshmi, C., Revathi, A., & Krishnamurthi, V. (2013). Effect of states and mixtures in HMM model and connected word recognition of profoundly deaf and hard of hearing speech. International Journal of Engineering and Technology (IJET),5(6), 4938–4946.

    Google Scholar 

  • Jeyalakshmi, C., Revathi, A., & Krishnamurthi, V. (2015a). Investigation of voice disorders and recognising the speech of children with hearing impairment in classical Tamil language. International Journal of Biomedical Engineering and Technology,17(4), 356–370.

    Article  Google Scholar 

  • Jeyalakshmi, C., Revathi, A., & Krishnamurthi, V. (2015b). Alphabet model-based short vocabulary speech recognition for the assessment of profoundly deaf and hard of hearing speeches. International Journal of Modelling, Identification and Control (IJMIC),23(3), 278–286.

    Article  Google Scholar 

  • Karjalainen, M., Boda, P. P., Somervuo, P., & Altosaar, T. (1997). Applications for the hearing impaired: evaluation of Finnish phoneme recognition methods. In 5th European conference on speech communication and technology. https://pdfs.semanticscholar.org/f665/007d4a78cd901903ef0f41769774d6f6561a.pdf?_ga=2.122539531.1822128985.1528710252-1332967019.1520309814.

  • Le Prell, C. G., & Clavier, O. H. (2017). Performance evaluation of the hearing impaired speech recognition in noisy environment. International Journal on Hearing Research,349, 76–89.

    Article  Google Scholar 

  • Levitt, H. (1972). Acoustic analysis of deaf speech using digital processing techniques. IEEE Transactions on Audio and Electroacoustics,AU-20(1), 35–41.

    Article  Google Scholar 

  • Mahmoudi, Z., Rahati, S., & Ghasemi, M. M. (2010). Classification of voice disorder in children with cochlear implantation and a hearing aid using multiple classifier fusion. In 10th International conference on information sciences signal processing and their applications (ISSPA) (Vol. 1, pp. 304–307). https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5605466.

  • Mengistu, K. T., & Rudzicz, F. (2011). Adapting acoustic and lexical models to dysarthric speech. In: IEEE international conference on acoustics, speech and signal processing (ICASSP) (Vol. 1, pp. 4924–4927). https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5947460.

  • Newman, C. W., & Sandridge, S. A. (2004). Hearing loss is often undiscovered, but screening is easy. Cleveland: Audiology Research Laboratory, Department of Otolaryngology and Communicative Disorders, The Cleveland Clinic Foundation.

  • Pickett, J. (1969). Some applications of speech analysis to communication aids for the deaf. Transactions on Audio and Electroacoustics,17(4), 283–289.

    Article  Google Scholar 

  • Pitts, A. B. (2010). Comparing speech assessments: The usefulness of the DEAP as compared tothe GFTA-2. Independent Studies and Capstones, Program in Audiology and Communication Sciences. https://digitalcommons.wustl.edu/cgi/viewcontent.cgi?article=1604&context=pacs_capstones.

  • Polur, P. D., & Miller, G. E. (2005). Investigation of an HMM/ANN hybrid structure in pattern recognition application using cepstral analysis of dysarthric (distorted) speech signals. Medical Engineering & Physics,28(8), 741–748.

    Article  Google Scholar 

  • Rabiner, L., & Juang, B. H. (1993). Fundamentals of speech recognition. Englewood Cliffs: Prentice Hall.

    Google Scholar 

  • Revathi, A., & Jeyalakshmi, C. (2017). A challenging task in recognizing the speech of the hearing impaired using normal hearing models in classical Tamil language. Journal of Engineering Research,5(2), 110–128.

    Google Scholar 

  • Saoud, S., Bousselmi, S., Mohamed, B. N., & Cherif, A. (2016). New speech enhancement based on discrete orthonormal Stockwell transform. International Journal of Advanced Computer Science and Applications. https://doi.org/10.14569/ijacsa.2016.071026.

    Article  Google Scholar 

  • Stevens, C. G., & Bernstein, J. (1988). Bernstein and Jared, Automatic speech recognition of impaired speech. International Journal of Rehabilitation Research,11(4), 396–397.

    Article  Google Scholar 

  • Tseng, S.-C. (2011) Speech production of Mandarin-speaking children with hearing impairment and normal hearing. In 17th International congress of phonetic sciences (Vol. 1, pp. 2030–2033). https://www.internationalphoneticassociation.org/icphs-proceedings/ICPhS2011/index.htm.

  • Yamada, Y. (2000). Hector Javkin and Karen Youdelman (2000) Assistive speech technology for persons with speech impairments. International Journal on Speech Communication,30(2–3), 179–187.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of ECE/SEEE, SASTRA Deemed University, Thanjavur, India

    A. Revathi

  2. Department of CSE/SoC, SASTRA Deemed University, Thanjavur, India

    N. Sasikaladevi

Authors
  1. A. Revathi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. N. Sasikaladevi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to N. Sasikaladevi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Revathi, A., Sasikaladevi, N. Hearing impaired speech recognition: Stockwell features and models. Int J Speech Technol 22, 979–991 (2019). https://doi.org/10.1007/s10772-019-09644-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10772-019-09644-3

Keywords

Search

Navigation