Abstract
We examine the efficacy of text and speech-based features for language identification in code-switched human-human dialog interactions at the turn level. We extract a variety of character- and word-based text features and pass them into multiple learners, including conditional random fields, logistic regressors and deep neural networks. We observe that our best-performing text system significantly outperforms a majority vote baseline. We further leverage the popular i-Vector approach in extracting features from the speech signal and show that this outperforms a traditional spectral feature-based front-end as well as the majority vote baseline.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Crutching refers to language learners relying on one language to fill in gaps in vocabulary or knowledge in the other [5].
- 2.
- 3.
We did experiment with oversampling the code-switched class as well, but this resulted in a degradation in performance. This could probably be due to the relatively few samples in the code-switched class to begin with.
- 4.
In order to roughly test this hypothesis, we ran experiments wherein we used the relatively cleaner Fisher corpora (of both Spanish and English speech) for both training and testing. In this case, the F1 score obtained was 0.96, highlighting both the mismatch between the Fisher and Bangor corpora as well as the effect of noise in the Bangor corpus. Of course, there is the possibility that the 2-class classification of English and Spanish turns from monolingual turns in code-switched speech might pose more challenges than LID in non-code-switched speech. Nevertheless, while this test was not a systematic one (and hence reported only as a footnote), this clearly points toward the effect of dataset quality on system performance.
References
Milroy L, Muysken P (1995) One speaker, two languages: cross-disciplinary perspectives on code-switching. Cambridge University Press
Wei L (2000) The bilingualism reader. Psychology Press
MacSwan J (2004) Code switching and grammatical theory. In: The handbook of bilingualism, vol 46, p 283
Myers-Scotton C (2006) Codeswitching with English: types of switching, types of communities. World Engl Crit Concepts Linguist 4(3):214
OConnor BH, Crawford LJ (2015) An art of being in between: the promise of hybrid language practices. In: Research on preparing inservice teachers to work effectively with emergent bilinguals. Emerald Group Publishing Limited, pp 149–173
Wheeler RS (2008) Code-switching. In: Educational leadership
Jiang Y-LB, García GE, Willis AI (2014) Code-mixing as a bilingual instructional strategy. Biling Res J 37(3):311–326
Ahmad BH, Jusoff K (2009) Teachers code-switching in classroom instructions for low English proficient learners. Engl Lang Teach 2(2):49
Littlewood W, Yu B (2011) First language and target language in the foreign language classroom. Lang Teach 44(1):64–77
Solorio T, Blair E, Maharjan S, Bethard S, Diab M, Gohneim M, Hawwari A, AlGhamdi F, Hirschberg J, Chang A et al (2014) Overview for the first shared task on language identification in code-switched data. In: Proceedings of the first workshop on computational approaches to code switching. Citeseer, pp 62–72
Bali K, Vyas Y, Sharma J, Choudhury M (2014) I am borrowing ya mixing? An analysis of English-Hindi code mixing in facebook. In: Proceedings of the first workshop on computational approaches to code switching, EMNLP 2014, p 116
Molina G, Rey-Villamizar N, Solorio T, AlGhamdi F, Ghoneim M, Hawwari A, Diab M (2016) Overview for the second shared task on language identification in code-switched data. In: EMNLP 2016, p 40
Solorio T, Liu Y (2008) Learning to predict code-switching points. In: Proceedings of the conference on empirical methods in natural language processing. Association for Computational Linguistics, pp 973–981
Barman U, Das A, Wagner J, Foster J (2014) Code mixing: a challenge for language identification in the language of social media. In: EMNLP 2014, vol 13
King B, Abney SP (2013) Labeling the languages of words in mixed-language documents using weakly supervised methods. In: HLT-NAACL, pp 1110–1119
Shirvani R, Piergallini M, Gautam GS, Chouikha M (2016) The Howard University system submission for the shared task in language identification in Spanish-English codeswitching. In: Proceedings of the second workshop on computational approaches to code switching, pp 116–120
Samih Y, Maharjan S, Attia M, Kallmeyer L, Solorio T (2016) Multilingual code-switching identification via LSTM recurrent neural networks. In: EMNLP 2016, p 50
Li Y, Yu Y, Fung P (2012) A Mandarin-English code-switching corpus. In: LREC, pp 2515–2519
Lyu D-C, Tan T-P, Chng E-S, Li H (2015) Mandarin-English code-switching speech corpus in South-East Asia: SEAME. Lang Resour Eval 49(3):581–600
Chan JY, Ching P, Lee T (2005) Development of a Cantonese-English code-mixing speech corpus. In: INTERSPEECH, pp 1533–1536
Dey A, Fung P (2014) A Hindi-English code-switching corpus. In: LREC, pp 2410–2413
Donnelly K, Deuchar M (2011) The bangor autoglosser: a multilingual tagger for conversational text. ITA11. Wrexham, Wales
Ramanarayanan V, Suendermann-Oeft D (2017) Jee haan, I’d like both, por favor: elicitation of a code-switched corpus of Hindi-English and Spanish-English human-machine dialog. Proc Interspeech 2017:47–51
Vu NT, Lyu DC, Weiner J, Telaar D, Schlippe T, Blaicher F, Chng E-S, Schultz T, Li H (2012) A first speech recognition system for Mandarin-English code-switch conversational speech. In: 2012 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 4889–4892
Yeh C-F, Sun L-C, Huang C-Y, Lee L-S (2011) Bilingual acoustic modeling with state mapping and three-stage adaptation for transcribing unbalanced code-mixed lectures. In: 2011 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 5020–5023
Eyben F, Weninger F, Gross F, Schuller B (2013) Recent developments in openSMILE, the Munich open-source multimedia feature extractor. In: Proceedings of the 21st ACM international conference on multimedia. ACM, pp 835–838
Qian Y, Evanini K, Wang X, Suendermann-Oeft D, Pugh RA, Lange PL, Molloy HR, Soong FK (2017) Improving sub-phone modeling for better native language identification with non-native English speech. Proc Interspeech 2017:2586–2590
Povey D, Ghoshal A, Boulianne G, Burget L, Glembek O, Goel N, Hannemann M, Motlicek P, Qian Y, Schwarz P et al (2011) The Kaldi speech recognition toolkit. In: IEEE 2011 workshop on automatic speech recognition and understanding, no. EPFL-CONF-192584. IEEE Signal Processing Society
Dehak N, Kenny PJ, Dehak R, Dumouchel P, Ouellet P (2011) Front-end factor analysis for speaker verification. IEEE Trans Audio Speech Lang Process 19(4):788–798
Martinez D, Plchot O, Burget L, Glembek O, Matějka P (2011) Language recognition in ivectors space. In: Twelfth annual conference of the international speech communication association
Cieri C, Miller D, Walker K (2004) The fisher corpus: a resource for the next generations of speech-to-text. In: LREC, vol 4, pp 69–71
Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ramanarayanan, V., Pugh, R., Qian, Y., Suendermann-Oeft, D. (2019). Automatic Turn-Level Language Identification for Code-Switched Spanish–English Dialog. In: D'Haro, L., Banchs, R., Li, H. (eds) 9th International Workshop on Spoken Dialogue System Technology. Lecture Notes in Electrical Engineering, vol 579. Springer, Singapore. https://doi.org/10.1007/978-981-13-9443-0_5
Download citation
DOI: https://doi.org/10.1007/978-981-13-9443-0_5
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-9442-3
Online ISBN: 978-981-13-9443-0
eBook Packages: Literature, Cultural and Media StudiesLiterature, Cultural and Media Studies (R0)