Skip to main content
SpringerLink
Log in

A review into deep learning techniques for spoken language identification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Information Technology has touched new vistas for a couple of decades mostly to simplify the day-to-day life of the humans. One of the key contributions of Information Technology is the application of Artificial Intelligence to achieve better results. The advent of artificial intelligence has given rise to a new branch of Natural Language Processing (NLP) called Computational Linguistics, which generates frameworks for intelligently manipulating spoken language knowledge and has brought human-machine onto a new stage. In this context, speech has arisen to be one of the imperative forms of interfaces, which is the basic mode of communication for us, and generally the most preferred one. Language identification, being the front-end for various natural language processing tasks, plays an important role in language translation. Owing to this, the focus has been given on the field of speech recognition involving the identification & recognition of languages by a machine. Spoken language identification is the identification of language present in a speech segment despite its size (duration & speed), ambiance (topic & emotion), and moderator (gender, age, demographic region). This paper has investigated various existing spoken language identification models implemented using different deep learning approaches, datasets, and performance measures utilized for their analysis. It also highlights the main features and challenges faced by these models. A comprehensive comparative study of deep learning techniques has been carried out for spoken language identification. Moreover, this review analyzes the efficiency of the spoken language models that can help the researchers to propose new language identification models for speech signals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Agatonovic-Kustrin S, Beresford R (2000) Basic concepts of artificial neural network (ANN) modeling and its application in pharmaceutical research. J Pharm Biomed Anal 22(5):717–727

    Article  Google Scholar 

  2. Albadr MAA, Tiun S (2020) Spoken language identification based on particle swarm optimisation–extreme learning machine approach. Circ Syst Signal Process 1–27

  3. Albadr MAA, Tiun S, Ayob M, AL-Dhief FT (2019) Spoken language identification based on optimised genetic algorithm–extreme learning machine approach. Int J Speech Technol 22(3):711–727

    Article  Google Scholar 

  4. Albawi S, Mohammed TA, Al-Zawi S (2017) Understanding of a convolutional neural network. In 2017 international conference on engineering and Technology (ICET) (pp 1–6). Ieee

  5. Ambikairajah E, Li H, Wang L, Yin B, Sethu V (2011) Language identification: A tutorial. IEEE Circ Syst Mag 11(2):82–108

    Article  Google Scholar 

  6. Bartz C, Herold T, Yang H, Meinel C (2017) Language identification using deep convolutional recurrent neural networks. In international conference on neural information processing (pp 880–889). Springer, Cham

  7. Bastanfard A, Amirkhani D (2020) Detect hidden message in reverse timestamp in Farsi. In 2020 25th international computer conference, computer society of Iran (CSICC) (pp 1–8). IEEE

  8. Bastanfard A, Amirkhani D, Hasani M Increasing the accuracy of automatic speaker age estimation by using multiple UBMs. In 2019 5th conference on knowledge based engineering and innovation (KBEI) (pp 592–598). IEEE

  9. Bastanfard A, Amirkhani D, Naderi S (2020) A singing voice separation method from Persian music based on pitch detection methods. In 2020 6th Iranian conference on signal processing and intelligent systems (ICSPIS) (pp 1–7). IEEE

  10. Buhmann MD (2000) Radial basis functions. Acta Numer 9:1–38

    Article  MathSciNet  Google Scholar 

  11. Buscema M (1998) Back propagation neural networks. Subst Use Misuse 33(2):233–270

    Article  Google Scholar 

  12. Cai W, Cai D, Huang S, Li M (2019) Utterance-level end-to-end language identification using attention-based CNN-BLSTM. In ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp 5991–5995). IEEE

  13. Chowdhury AA, Borkar VS, Birajdar GK (2020) Indian language identification using time-frequency image textural descriptors and GWO-based feature selection. J Exp Theor Artif Intell 32(1):111–132

    Article  Google Scholar 

  14. Das A, Guha S, Singh PK, Ahmadian A, Senu N, Sarkar R (2020) A hybrid meta-heuristic feature selection method for identification of Indian spoken languages from audio signals. IEEE Access 8:181432–181449

    Article  Google Scholar 

  15. Das HS, Roy P (2019) Optimal prosodic feature extraction and classification in parametric excitation source information for Indian language identification using neural network based Q-learning algorithm. Int J Speech Technol 22(1):67–77

    Article  Google Scholar 

  16. Das HS, Roy P (2019) A deep dive into deep learning techniques for solving spoken language identification problems. In intelligent speech signal processing (pp 81–100). Academic Press

  17. Dehak N, Kenny PJ, Dehak R, Dumouchel P, Ouellet P (2010) Front-end factor analysis for speaker verification. IEEE Trans Audio Speech Lang Process 19(4):788–798

    Article  Google Scholar 

  18. Deshwal D, Sangwan P, Kumar D (2020) A language identification system using hybrid features and back-propagation neural network. Appl Acoust 164:107289

    Article  Google Scholar 

  19. Ferrer L, Lei Y, McLaren M, Scheffer N (2015) Study of senone-based deep neural network approaches for spoken language recognition. IEEE/ACM Trans Audio Speech Lang Process 24(1):105–116

    Article  Google Scholar 

  20. Fine TL (2006) Feedforward neural network methodology. Springer Science & Business Media

  21. Fu Z, Lu G, Ting KM, Zhang D (2010) A survey of audio-based music classification and annotation. IEEE Trans Multimed 13(2):303–319

    Article  Google Scholar 

  22. Gadekallu TR, Khare N, Bhattacharya S, Singh S, Reddy Maddikunta PK, Ra IH, Alazab M (2020) Early detection of diabetic retinopathy using PCA-firefly based deep learning model. Electronics 9(2):274

    Article  Google Scholar 

  23. Ganapathy S, Han K, Thomas S, Omar M, Segbroeck MV, Narayanan SS (2014) Robust language identification using convolutional neural network features. In Fifteenth annual conference of the international speech communication association

  24. Garain A, Singh PK, Sarkar R (2021) FuzzyGCP: a deep learning architecture for automatic spoken language identification from speech signals. Expert Syst Appl 168:114416

    Article  Google Scholar 

  25. Gelly G, Gauvain JL, Le VB, Messaoudi A (2016) A divide-and-conquer approach for language identification based on recurrent neural networks. In INTERSPEECH (pp 3231–3235)

  26. Geng W, Wang W, Zhao Y, Cai X, Xu B, Xinyuan C (2016) End-to-end language identification using attention-based recurrent neural networks. In Interspeech (pp 2944–2948)

  27. Gonzalez-Dominguez J, Lopez-Moreno I, Sak H, Gonzalez-Rodriguez J, Moreno PJ (2014) Automatic language identification using long short-term memory recurrent neural networks. In Fifteenth annual conference of the international speech communication association

  28. Gordon RG Jr (2005) Ethnologue, languages of the world. http://www.ethnologue.com/

  29. Guha S, Das A, Singh PK, Ahmadian A, Senu N, Sarkar R (2020) Hybrid feature selection method based on harmony search and naked mole-rat algorithms for spoken language identification from audio signals. IEEE Access 8:182868–182887

    Article  Google Scholar 

  30. Heracleous P, Takai K, Yasuda K, Mohammad Y, Yoneyama A (2018) Comparative study on spoken language identification based on deep learning. In 2018 26th European signal processing conference (EUSIPCO) (pp 2265–2269). IEEE

  31. Hinton G, Deng L, Yu D, Dahl GE, Mohamed AR, Jaitly N, … Kingsbury B (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82–97

    Article  Google Scholar 

  32. Hori C, Hori T, Watanabe S, Hershey JR (2016) Context-sensitive and role-dependent spoken language understanding using bidirectional and attention LSTMs. In Interspeech (pp 3236–3240)

  33. Jin M, Song Y, McLoughlin IV (2017) End-to-end dnn-cnn classification for language identification

  34. Jothilakshmi S, Ramalingam V, Palanivel S (2012) A hierarchical language identification system for Indian languages. Digit Signal Process 22(3):544–553

    Article  MathSciNet  Google Scholar 

  35. Kim H, Park JS (2020) Automatic language identification using speech rhythm features for multi-lingual speech recognition. Appl Sci 10(7):2225

    Article  Google Scholar 

  36. Lee CH (2008) Principles of spoken language recognition. In springer handbook of speech processing (pp 785–796). Springer, Berlin, Heidelberg

  37. Lei Y, Ferrer L, Lawson A, McLaren M, Scheffer N (2014) Application of Convolutional Neural Networks to Language Identification in Noisy Conditions. In Odyssey

  38. Li H, Ma B, Lee KA (2013) Spoken language recognition: from fundamentals to practice. Proc IEEE 101(5):1136–1159

    Article  Google Scholar 

  39. Liu B, Lane I (2016) Joint online spoken language understanding and language modeling with recurrent neural networks. arXiv preprint arXiv:1609.01462

  40. Lopez-Moreno I, Gonzalez-Dominguez J, Martinez D, Plchot O, Gonzalez-Rodriguez J, Moreno PJ (2016) On the use of deep feedforward neural networks for automatic language identification. Comput Speech Lang 40:46–59

    Article  Google Scholar 

  41. Lounnas K, Satori H, Hamidi M, Teffahi H, Abbas M, Lichouri M (2020) CLIASR: a combined automatic speech recognition and language identification system. In 2020 1st international conference on innovative research in applied science, engineering and Technology (IRASET) (pp 1–5). IEEE

  42. Lu X, Shen P, Tsao Y, Kawai H (2017) Regularization of neural network model with distance metric learning for i-vector based spoken language identification. Comput Speech Lang 44:48–60

    Article  Google Scholar 

  43. Ma B, Li H, Tong R (2007) Spoken language recognition using ensemble classifiers. IEEE Trans Audio Speech Lang Process 15(7):2053–2062

    Article  Google Scholar 

  44. Mahdavi R, Bastanfard A, Amirkhani D (2020) Persian accents identification using modeling of speech articulatory features. In 2020 25th international computer conference, Computer Society of Iran (CSICC) (pp 1–9). IEEE

  45. Masumura R, Asami T, Masataki H, Aono Y (2017) Parallel phonetically aware DNNs and LSTM-RNNs for frame-by-frame discriminative modeling of spoken language identification. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp 5260–5264). IEEE

  46. Miao X, McLoughlin I, Yan Y (2019) A new time-frequency attention mechanism for TDNN and CNN-LSTM-TDNN, with application to language identification. In Interspeech (pp 4080–4084)

  47. Mikolov T, Karafiát M, Burget L, Černocký J, Khudanpur S (2010) Recurrent neural network based language model. In Eleventh annual conference of the international speech communication association

  48. Mohamed AR, Dahl GE, Hinton G (2011) Acoustic modeling using deep belief networks. IEEE Trans Audio Speech Lang Process 20(1):14–22

    Article  Google Scholar 

  49. Montavon G (2009) Deep learning for spoken language identification. In NIPS workshop on deep learning for speech recognition and related applications (pp 1–4)

  50. Morchid M (2017) Internal memory gate for recurrent neural networks with application to spoken language understanding. In INTERSPEECH (pp 3316–3319)

  51. Morchid M Bidirectional internal memory gate recurrent neural networks for spoken language understanding. Int J Speech Technol 1–9

  52. Mukherjee H, Obaidullah SM, Santosh KC, Phadikar S, Roy K (2020) A lazy learning-based language identification from speech using MFCC-2 features. Int J Mach Learn Cybern 11(1):1–14

    Article  Google Scholar 

  53. Muthusamy YK, Barnard E, Cole RA (1994) Reviewing automatic language identification. IEEE Signal Process Mag 11(4):33–41

    Article  Google Scholar 

  54. Navratil J (2001) Spoken language recognition-a step toward multilinguality in speech processing. IEEE Trans Speech Audio Process 9(6):678–685

    Article  Google Scholar 

  55. Peche M, Davel MH, Barnard E (2009) Development of a spoken language identification system for south African languages. SAIEE Africa Res J 100(4):97–103

    Article  Google Scholar 

  56. Poncelet J, Renkens V (2021) Low resource end-to-end spoken language understanding with capsule networks. Comput Speech Lang 66:101142

    Article  Google Scholar 

  57. Qian Y, Ubale R, Lange P, Evanini K, Ramanarayanan V, Soong FK (2019) Spoken language understanding of human-machine conversations for language learning applications. J Signal Process Syst 1–13

  58. Richardson F, Reynolds D, Dehak N (2015) Deep neural network approaches to speaker and language recognition. IEEE Signal Process Lett 22(10):1671–1675

    Article  Google Scholar 

  59. Sharma V, Rai S, Dev A (2012) A comprehensive study of artificial neural networks. Int J Adv Res Comput Sci Softw Eng 2(10)

  60. Shen P, Lu X, Li S, Kawai H (2020) Knowledge distillation-based representation learning for short-utterance spoken language identification. IEEE/ACM Trans Audio Speech Lang Process 28:2674–2683

    Article  Google Scholar 

  61. Sim KC, Li H (2008) On acoustic diversification front-end for spoken language identification. IEEE Trans Audio Speech Lang Process 16(5):1029–1037

    Article  Google Scholar 

  62. Song Y, Cui R, Hong X, Mcloughlin I, Shi J, Dai L (2015) Improved language identification using deep bottleneck network. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp 4200–4204). IEEE

  63. Torres-Carrasquillo PA, Reynolds DA, Deller JR (2002) Language identification using Gaussian mixture model tokenization. In 2002 IEEE international conference on acoustics, speech, and signal processing (Vol 1, pp I–757). IEEE

  64. Tzanetakis G, Cook P (2002) Musical genre classification of audio signals. IEEE Trans Speech Audio Process 10(5):293–302

    Article  Google Scholar 

  65. Ultsch A (1993) Self-organizing neural networks for visualisation and classification. In information and classification (pp 307–313). Springer, Berlin, Heidelberg

  66. Vasan D, Alazab M, Wassan S, Safaei B, Zheng Q (2020) Image-based malware classification using ensemble of CNN architectures (IMCEC). Comput Secur 92:101748

    Article  Google Scholar 

  67. Veisi H, Ghoreishi SA, Bastanfard A (2021) Spoken term detection for Persian news of Islamic Republic of Iran broadcasting. Signal Data Process 17(4):67–88

    Article  Google Scholar 

  68. Vuddagiri RK, Vydana HK, Vuppala AK (2018) Curriculum learning based approach for noise robust language identification using DNN with attention. Expert Syst Appl 110:290–297

    Article  Google Scholar 

  69. Wikipedia, “https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons”

  70. Wong KYE (2004) Automatic spoken language identification utilizing acoustic and phonetic speech information (Doctoral dissertation, Queensland University of Technology)

  71. Wu WL, Lu RZ, Duan JY, Liu H, Gao F, Chen YQ (2010) Spoken language understanding using weakly supervised learning. Comput Speech Lang 24(2):358–382

    Article  Google Scholar 

  72. Yasmin G, Das AK, Nayak J, Pelusi D, Ding W (2020) Graph based feature selection investigating boundary region of rough set for language identification. Expert Syst Appl 158:113575

    Article  Google Scholar 

  73. Yu D, Deng L (2010) Deep learning and its applications to signal and information processing [exploratory dsp]. IEEE Signal Process Mag 28(1):145–154

    Article  Google Scholar 

  74. Zissman MA, Berkling KM (2001) Automatic language identification. Speech Commun 35(1-2):115–124

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Islamic University of Science & Technology, Awantipora, JK, India

    Irshad Ahmad Thukroo, Rumaan Bashir & Kaiser J. Giri

Authors
  1. Irshad Ahmad Thukroo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rumaan Bashir
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kaiser J. Giri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rumaan Bashir.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Thukroo, I.A., Bashir, R. & Giri, K.J. A review into deep learning techniques for spoken language identification. Multimed Tools Appl 81, 32593–32624 (2022). https://doi.org/10.1007/s11042-022-13054-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13054-0

Keywords

Search

Navigation