Abstract
The aim of this study is two-fold. First, we perform a series of experiments to examine the interference of different noises on speech processing. For that purpose, we concentrate on the Lombard effect, an involuntary tendency to raise speech level in the presence of background noise. Then, we apply this knowledge to detecting speech with the Lombard effect. This is for preparing a dataset for training a machine learning-based system for automatic speech conversion, mimicking a human way to make speech more intelligible in the presence of noise, i.e., to create Lombard speech. Several spectral descriptors are analyzed in the context of Lombard speech and various types of noise. In conclusion, pub-like and babble noises are most similar when comparing Spectral Entropy, Spectral RollOff, and Spectral Brightness. The larger values of these spectral descriptors, the more the speech-in-noise signal is degraded. To quantify the effect of noise on speech, containing the Lombard effect, an average formant track error is calculated as an objective image quality metric. For image quality assessment Structural SIMilarity (SSIM) index is employed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lombard, E.: Le signe de l’elevation de la voix. Ann. Mal. de L’Oreille et du Larynx, 101–119 (1911). Zollinger, S.A., Brumm, H.: The lombard effect. Current Biol. 21(16), 614–615 (2011)
Uma Maheswari, S., Shahina, A., Nayeemulla Khan, A.: Understanding Lombard speech: a review of compensation techniques towards improving speech based recognition systems. Artif. Intell. Rev. 54(4), 2495–2523 (2021)
Li, G., Hu, R., Zhang, R., Wang, X.: A mapping model of spectral tilt in normal-to-Lombard speech conversion for intelligibility enhancement. Multimed. Tools Appl. 79(27), 19471–19491 (2020)
Kakol, K., Korvel, G., Kostek, B.: Improving objective speech quality indicators in noise conditions. In: Data Science: New Issues, Challenges and Applications, pp. 199–218. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-39250-5_11
Bollepalli, B., Juvela, L., Airaksinen, M., Valentini-Botinhao, C., Alku, P.: Normal-to-Lombard adaptation of speech synthesis using long short-term memory recurrent neural networks. Speech Commun. 110, 64–75 (2019)
Paul, D., Shifas, M.P., Pantazis, Y., Stylianou, Y.: Enhancing speech intelligibility in text-to-speech synthesis using speaking style conversion. arXiv preprint arXiv:2008.05809 (2020)
Korvel, G., Kąkol, K., Kurasova, O., Kostek, B.: Evaluation of Lombard speech models in the context of speech in noise enhancement. IEEE Access 8, 155156–155170 (2020)
Novitasari, S., Sakti, S., Nakamura, S.: Dynamically adaptive machine speech chain inference for tts in noisy environment: listen and speak louder. Proc. Interspeech 2021, 4124–4128 (2021)
Yue, F., Deng, Y., He, L., Ko, T., Zhang, Y.: Exploring machine speech chain for domain adaptation. In: ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6757–6761 (2022)
Lampert, T.A., O’Keefe, S.E.: On the detection of tracks in spectrogram images. Pattern Recogn. 46(5), 1396–1408 (2013)
Bhattacharjee, M., Prasanna, S.M., Guha, P.: Speech/music classification using features from spectral peaks. IEEE/ACM Trans. Audio, Speech Lang. Process. 28, 1549–1559 (2020)
McAulay, R., Quatieri, T.: Speech analysis/synthesis based on a sinu-soidal representation. IEEE Trans. Acoust. Speech Signal Process. 34(4), 744–754 (1986)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Peng, J., et al.: Implementation of the structural SIMilarity (SSIM) index as a quantita- tive evaluation tool for dose distribution error detection. Med. Phys. 47(4), 1907–1919 (2020)
Zini, S., Bianco, S., Schettini, R.: Deep residual autoencoder for blind universal JPEG restoration. IEEE Access 8, 63283–63294 (2020)
Wei, Y., Zeng, Y., Li, C.: Single-channel speech enhancement based on subband spectral entropy. J. Audio Eng. Soc. 66(3), 100–113 (2018)
Czyzewski, A., Kostek, B., Bratoszewski, P., Kotus, J., Szykulski, M.: An audio-visual corpus for multimodal automatic speech recognition. J. Intell. Inf. Syst. 49(2), 167–192 (2017). https://doi.org/10.1007/s10844-016-0438-z
Barber, D.: Bayesian Reasoning and Machine Learning. Cambridge University Press (2012). ISBN 978-0-521-51814-7
Acknowledgments
This research is funded by the European Social Fund under the No 09.3.3-LMT-K-712 “Development of Competences of Scientists, other Researchers and Students through Practical Research Activities” measure.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Korvel, G., Kąkol, K., Treigys, P., Kostek, B. (2022). Investigating Noise Interference on Speech Towards Applying the Lombard Effect Automatically. In: Ceci, M., Flesca, S., Masciari, E., Manco, G., Raś, Z.W. (eds) Foundations of Intelligent Systems. ISMIS 2022. Lecture Notes in Computer Science(), vol 13515. Springer, Cham. https://doi.org/10.1007/978-3-031-16564-1_38
Download citation
DOI: https://doi.org/10.1007/978-3-031-16564-1_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16563-4
Online ISBN: 978-3-031-16564-1
eBook Packages: Computer ScienceComputer Science (R0)