research-article

Can You Understand Why I Am Crying? A Decision-making System for Classifying Infants’ Cry Languages Based on DeepSVM Model

Authors:

Hossein Ghayoumi Zadeh,

Hamidreza Rabiee,

Mohammad R. KhosraviAuthors Info & Claims

ACM Transactions on Asian and Low-Resource Language Information Processing, Volume 23, Issue 1

Article No.: 12, Pages 1 - 17

https://doi.org/10.1145/3579032

Published: 15 January 2024 Publication History

Abstract

Scientific and therapeutic advances in perinatology and neonatology have improved the survival prospects of preterm and extremely-low-birth-weight infants. Infants’ cries are a valuable noninvasive tool for monitoring their neurologic health, especially if they are premature. Automatic acoustic analysis and data mining are employed in this study to determine the discriminative features of preterm and full-term infant cries. The use of machine learning for recognizing sounds in a newborn's cry language has received less attention than previous methods for analyzing the sounds. Moreover, to extract appropriate features from infant cries, adequate knowledge and appropriate signal descriptors are required. Accordingly, to analyze infant cry language, we propose an approach that uses fractal descriptors to extract discriminant features from spectrograms of windowed signals, followed by iterative neighborhood component analysis (iNCA) to select appropriate features. Additionally, the improved deep support vector machine (DeepSVM) is used to classify the infants’ crying types and their meanings. The proposed method is verified using a newborn sound dataset. According to the classification of five types of crying perception based on various characteristics, 98.34% of all crying perceptions have been recognized. Although there are many classes examined, the feature extraction method based on the fractal method and our optimal classification have a much higher diagnostic accuracy compared with similar methods for analyzing baby crying language. The proposed method can overcome many problems associated with analyzing babies’ crying sounds and understanding their language, such as uncertainty and unusual errors in classification.

References

[1]

E. Geangu, O. Benga, D. Stahl, and T. Striano. 2010. Contagious crying beyond the first days of life. Infant Behavior and Development 33, 3 (2010), 279–288.

[2]

P. S. Douglas and H. Hiscock. 2010. The unsettled baby: Crying out for an integrated, multidisciplinary primary care approach. Medical Journal of Australia 193, 9 (2010), 533–536.

[3]

D. M. Zeifman and I. St James-Roberts. 2017. Parenting the crying infant. Current Opinion in Psychology 15 (2017), 149–154.

[4]

S. Orlandi, A. Bandini, F. F. Fiaschi, and C. Manfredi. 2017. Testing software tools for newborn cry analysis using synthetic signals. Biomedical Signal Processing and Control 37 (2017), 16–22.

[5]

L. Liu, W. Li, X. Wu, and B. X. Zhou. 2019. Infant cry language analysis and recognition: An experimental approach. IEEE/CAA Journal of Automatica Sinica 6, 3 (2019), 778–788.

[6]

H. Yoo, E. H. Buder, D. D. Bowman, G. M. Bidelman, and D. K. Oller. 2019. Acoustic correlates and adult perceptions of distress in infant speech-like vocalizations and cries. Frontiers in Psychology 10 (2019), 1154.

[7]

K. Tobin, D. King, S. Henderson, A. Bellocchi, and S. M. Ritchie. 2016. Expression of emotions and physiological changes during teaching. Cultural Studies of Science Education 11, 3 (2016), 669–692.

[8]

S. Ntalampiras. 2015. Audio pattern recognition of baby crying sound events. Journal of the Audio Engineering Society 63, 5 (2015), 358–369.

[9]

A. Bashiri and R. Hosseinkhani. 2020. Infant crying classification by using genetic algorithm and artificial neural network. Acta Medica Iranica (2020), 531–539.

[10]

T. Özseven. 2022. A review of infant cry recognition and classification based on computer-aided diagnoses. In 2022 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA’22). 1–11.

[11]

C. Y. Chang, S. Bhattacharya, P. M. Raj Vincent, K. Lakshmanna, and K. Srinivasan. 2021. An efficient classification of neonates cry using extreme gradient boosting-assisted grouped-support-vector network. Journal of Healthcare Engineering 2021 (2021).

[12]

I. A. Bănică, H. Cucu, A. Buzo, D. Burileanu, and C. Burileanu. 2016. Automatic methods for infant cry classification. In 2016 International Conference on Communications (COMM’16). 51–54.

Digital Library

[13]

S. Orlandi, C. A. R. Garcia, A. Bandini, G. Donzelli, and C. Manfredi. 2016. Application of pattern recognition techniques to the classification of full-term and preterm infant cry. Journal of Voice 30, 6 (2016), 656–663.

[14]

A. B. Moller, J. H. Patten, C. Hanson, A. Morgan, L. Say, T. Diaz, and A. C. Moran. 2019. Monitoring maternal and newborn health outcomes globally: A brief history of key events and initiatives. Tropical Medicine & International Health 24, 12 (2019), 1342–1368.

[15]

C. Ji, T. B. Mudiyanselage, Y. Gao, and Y. Pan. 2021. A review of infant cry analysis and classification. EURASIP Journal on Audio, Speech, and Music Processing 1 (2021), 1–17.

[16]

J. Saraswathy, M. Hariharan, S. Yaacob, and W. Khairunizam. 2012. Automatic classification of infant cry: A review. In 2012 International Conference on Biomedical Engineering (ICoBE’12). 543–548.

[17]

R. Cohen, D. Ruinskiy, J. Zickfeld, H. IJzerman, and Y. Lavner. 2020. Baby cry detection: Deep learning and classical approaches. In Development and Analysis of Deep Learning Architectures. Springer, Cham, 171–196.

[18]

F. S. Matikolaie and C. Tadj. 2020. On the use of long-term features in a newborn cry diagnostic system. Biomedical Signal Processing and Control 59 (2020), 101889.

[19]

F. S. Matikolaie and C. Tadj. 2022. Machine learning-based cry diagnostic system for identifying septic newborns. Journal of Voice (2022).

[20]

J. J. Parga, S. Lewin, J. Lewis, D. Montoya-Williams, A. Alwan, B. Shaul, et al. 2020. Defining and distinguishing infant behavioral states using acoustic cry analysis: Is colic painful?. Pediatric Research 87, 3 (2020), 576–580.

[21]

C. F. Z. Boukydis and B. M. Lester (Eds.). 2012. Infant crying: Theoretical and research perspectives. Berlin, Springer Science and Bussiness Media.

[22]

Y. Abdulaziz and S. M. S. Ahmad.2010. Infant cry recognition system: A comparison of system performance based on mel frequency and linear prediction cepstral coefficients. In 2010 International Conference on Information Retrieval & Knowledge Management (CAMP’10). 260–263.

[23]

V. R. Joshi, K. Srinivasan, P. D. R. Vincent, V. Rajinikanth, and C. Y. Chang. 2022. A multistage heterogeneous stacking ensemble model for augmented infant cry classification. Frontiers in Public Health 10 (2022).

[24]

C. Y. Chang, C. W. Chang, S. Kathiravan, C. Lin, and S. T. Chen. 2017. DAG-SVM based infant cry classification system using sequential forward floating feature selection. Multidimensional Systems and Signal Processing 28, 3 (2017), 961–976.

Digital Library

[25]

L. W. Jer, M. Hariharan, V. Vijean, H. Yazid, and L. C. Chin. 2018. Infant cry classification using dual tree complex wavelet transform features. Advanced Science Letters 24, 3 (2018), 1741–1744.

[26]

J. Saraswathy, M. Hariharan, W. Khairunizam, J. Sarojini, N. Thiyagar, Y. Sazali, and S. Nisha. 2018. Time–frequency analysis in infant cry classification using quadratic time frequency distributions. Biocybernetics and Biomedical Engineering 38, 3 (2018), 634–645.

[27]

B. X. Fang, J. T. Cen, T. Yuan, G. D. Yin, J. Gu, S. Q. Zhang, et al. 2020. Etiology of newborn hearing impairment in Guangdong province: 10-year experience with screening, diagnosis, and follow-up. World Journal of Pediatrics 16, 3 (2020), 305–313.

[28]

J. Saraswathy, M. Hariharan, W. Khairunizam, J. Sarojini, and S. Yaacob. 2020. Time-frequency analysis-based method for application of infant cry classification. International Journal of Medical Engineering and Informatics 12, 2 (2020), 119–134.

[29]

Y. Kheddache and C. Tadj. 2019. Identification of diseases in newborns using advanced acoustic features of cry signals. Biomedical Signal Processing and Control 50 (2019), 35–44.

[30]

H. F. Alaie, L. Abou-Abbas, and C. Tadj. 2016. Cry-based infant pathology classification using GMMs. Speech Communication 77 (2016), 28–52.

Digital Library

[31]

S. Lahmiri, C. Tadj, C. Gargour, and S. Bekiros. 2022. Deep learning systems for automatic diagnosis of infant cry signals. Chaos, Solitons & Fractals 154 (2022), 111700.

[32]

J. Xie, X. Long, R. A. Otte, and C. Shan. 2021. Convolutional neural networks for audio-based continuous infant cry monitoring at home. IEEE Sensors Journal 21, 24 (2021), 27710–27717.

[33]

D. Ferretti, M. Severini, E. Principi, A. Cenci, and S. Squartini. 2018. Infant cry detection in adverse acoustic environments by using deep neural networks. In 26th European Signal Processing Conference (EUSIPCO’18). 992–996.

[34]

C. Y. Chang and J. J. Li. 2016. Application of deep learning for recognizing infant cries. In 2016 IEEE International Conference on Consumer Electronics—Taiwan (ICCE-TW’16). 1–2.

[35]

C. A. Bratan, M. Gheorghe, I. Ispas, E. Franti, M. Dascalu, S. M. Stoicescu, et al. 2021. Dunstan baby language classification with CNN. In 2021 International Conference on Speech Technology and Human-Computer Dialogue (SpeD’21). 167–171.

[36]

R. Torres, D. Battaglino, and L. Lepauloux. 2017. Baby cry sound detection: A comparison of hand crafted features and deep learning approach. In International Conference on Engineering Applications of Neural Networks. Springer, Cham, 168–179.

[37]

V. Vaishnavi and P. Suveetha Dhanaselvam. 2022. Neonatal cry signal prediction and classification via dense convolution neural network. Journal of Intelligent & Fuzzy Systems (Preprint), 1–14.

Digital Library

[38]

S. Cabon, B. Met-Montot, F. Porée, O. Rosec, A. Simon, and G. Carrault. 2022. Extraction of premature newborns’ spontaneous cries in the real context of neonatal intensive care units. Sensors 22, 5 (2022), 1823.

[39]

G. Raboshchuk, C. Nadeu, P. Jančovič, A. P. Lilja, M. Köküer, B. M. Mahamud, and A. R. De Veciana. 2017. A knowledge-based approach to automatic detection of equipment alarm sounds in a neonatal intensive care unit environment. IEEE Journal of Translational Engineering in Health and Medicine 6 (2017), 1–10. http://doi.org/0.1109/JTEHM.2017.2781224

[40]

L. Abou-Abbas, C. Tadj, C. Gargour, and L. Montazeri. 2017. Expiratory and inspiratory cries detection using different signals’ decomposition techniques. Journal of Voice 31, 2 (2017), 259–e13.

[41]

G. Naithani, J. Kivinummi, T. Virtanen, O. Tammela, M. J. Peltola, and J. M. Leppänen. 2018. Automatic segmentation of infant cry signals using hidden Markov models. EURASIP Journal on Audio, Speech, and Music Processing 1, (2018), 1–14.

Digital Library

[42]

C. Mateo and J. A. Talavera. 2018. Short-time Fourier transform with the window size fixed in the frequency domain. Digital Signal Processing 77 (2018), 13–21.

[43]

J. B. Florindo and O. M. Bruno. 2014. Fractal descriptors based on the probability dimension: A texture analysis and classification approach. Pattern Recognition Letters 42 (2014), 107–114.

[44]

J. B. Florindo and O. M. Bruno. 2012. Fractal descriptors based on Fourier spectrum applied to texture analysis. Physica A: Statistical Mechanics and its Applications 391, 20 (2012), 4909–4922.

[45]

S. Chatterjee, D. Dey, and S. Munshi. 2019. Integration of morphological preprocessing and fractal based feature extraction with recursive feature elimination for skin lesion types classification. Computer Methods and Programs in Biomedicine 178 (2019), 201–218.

Digital Library

[46]

Z. Qi, B. Wang, Y. Tian, and P. Zhang. 2016. When ensemble learning meets deep learning: A new deep support vector machine for classification. Knowledge-Based Systems 107 (2016), 54–60.

Digital Library

[47]

G. Veres. 2015. Donate-a-cry Corpus [Online]. Retrieved February 2, 2023 from https://github.com/gveres/donateacry-corpus.

[48]

D. K. Oller, E. H. Buder, H. L. Ramsdell, A. S. Warlaumont, L. Chorna, and R. Bakeman. 2013. Functional flexibility of infant vocalization and the emergence of language. Proceedings of the National Academy of Sciences 110, 16 (2013), 6318–6323.

[49]

E. Geangu, P. Hauf, R. Bhardwaj, and W. Bentz. 2011. Infant pupil diameter changes in response to others’ positive and negative emotions. PLoS One 6, 11 (2011), e27132.

[50]

Y. Li and T. Zhang. 2017. Deep neural mapping support vector machines. Neural Networks 93 (2017), 185–194.

[51]

Y. Cho and L. Saul. 2009. Kernel methods for deep learning. Advances in Neural Information Processing Systems 22 (2009).

Digital Library

[52]

S. Bano and K. M. Ravikumar. 2015. Decoding baby talk: A novel approach for normal infant cry signal classification. In 2015 International Conference on Soft-Computing and Networks Security (ICSNS’15). 1–4.

[53]

T. N. Maghfira, T. Basaruddin, and A. Krisnadhi. 2020. Infant cry classification using CNN–RNN. In Journal of Physics: Conference Series. IOP Publishing 1528, 1 (2020), 012019.

[54]

E. Franti, I. Ispas, and M. Dascalu. 2018. Testing the universal baby language hypothesis-automatic infant speech recognition with CNNs. In 41st International Conference on Telecommunications and Signal Processing (TSP’18). 1–4.

[55]

S. Bano and K. M. Ravikumar. 2015. Decoding baby talk: Basic approach for normal classification of infant cry signal. International Journal of Computer Applications 975 (2015), 8887.

[56]

L. Abou-Abbas, H. F. Alaie, and C. Tadj. 2015. Automatic detection of the expiratory and inspiratory phases in newborn cry signals. Biomedical Signal Processing and Control 19 (2015), 35–43.

[57]

N. S. A. Wahid, P. Saad, and M. Hariharan. 2016. Automatic infant cry pattern classification for a multiclass problem. Journal of Telecommunication, Electronic and Computer Engineering (JTEC) 8, 9 (2016), 45–52.

[58]

L. Le, A. N. M. Kabir, C. Ji, S. Basodi, and Y. Pan.2019. Using transfer learning, SVM, and ensemble classification to classify baby cries based on their spectrogram images. In 2019 IEEE 16th International Conference on Mobile Ad Hoc and Sensor Systems Workshops (MASSW). 106–110.

[59]

K. Sharma, C. Gupta, and S. Gupta. 2019. Infant weeping calls decoder using statistical feature extraction and Gaussian mixture models. In 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT’19). 1–6.

[60]

F. Anders, M. Hlawitschka, and M. Fuchs. 2020. Automatic classification of infant vocalization sequences with convolutional neural networks. Speech Communication 119 (2020), 36–45.

Cited By

Li FCui CHu Y(2024)Classification of Infant Crying Sounds Using SE-ResNet-TransformerSensors10.3390/s2420657524:20(6575)Online publication date: 12-Oct-2024
https://doi.org/10.3390/s24206575
Ghurroh Setyoningrum NUtami EKusrini Wahyu Wibowo F(2024)A Systematic Literature Review of Audio Signal Processing Methods for Infant Cry Recognition and Interpretation2024 International Conference on Computer Engineering, Network, and Intelligent Multimedia (CENIM)10.1109/CENIM64038.2024.10882830(1-6)Online publication date: 19-Nov-2024
https://doi.org/10.1109/CENIM64038.2024.10882830
Cederic LMaximilian BFabian RChristian HAnsgar T(2024)Contextual Anomaly Detection in Hot Forming Production Line using PINN Architecture2024 IEEE International Conference on Advanced Intelligent Mechatronics (AIM)10.1109/AIM55361.2024.10637234(1020-1025)Online publication date: 15-Jul-2024
https://doi.org/10.1109/AIM55361.2024.10637234
Show More Cited By

Index Terms

Can You Understand Why I Am Crying? A Decision-making System for Classifying Infants’ Cry Languages Based on DeepSVM Model
1. Applied computing
  1. Arts and humanities
    1. Sound and music computing

Recommendations

Infant cry reliability: Acoustic homogeneity of spontaneous cries and pain-induced cries

Infant cries can indicate certain developmental disorders and therefore may be suited for early diagnosis. An open research question is which type of crying (spontaneous, pain-induced) is best suited for infant cry analysis. For estimating the degree of ...
Mobile media use by infants and toddlers
Abstract
This study examined mobile media (MM) use patterns by infants and toddlers (36 months old and younger, N = 326) from a family media ecology perspective. Parents' frequency of MM use, their motives for child's use, and child's age ...
Highlights
- Parents' frequency of mobile media (MM) use predicted children's total MM use on our online survey.
Newborn infant's cry analysis

The very first cry or the birth cry of an infant carries significant information about the health of an infant and hence, it is considered as the vital parameter in deciding the Apgar count. As an infant grows, the cry acoustics changes with the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Asian and Low-Resource Language Information Processing

ACM Transactions on Asian and Low-Resource Language Information Processing Volume 23, Issue 1

January 2024

385 pages

EISSN:2375-4702

DOI:10.1145/3613498

Editor:
Imed Zitoun
Google, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 January 2024

Online AM: 23 January 2023

Accepted: 19 December 2022

Revised: 26 November 2022

Received: 14 September 2022

Published in TALLIP Volume 23, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
685
Total Downloads

Downloads (Last 12 months)276
Downloads (Last 6 weeks)36

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li FCui CHu Y(2024)Classification of Infant Crying Sounds Using SE-ResNet-TransformerSensors10.3390/s2420657524:20(6575)Online publication date: 12-Oct-2024
https://doi.org/10.3390/s24206575
Ghurroh Setyoningrum NUtami EKusrini Wahyu Wibowo F(2024)A Systematic Literature Review of Audio Signal Processing Methods for Infant Cry Recognition and Interpretation2024 International Conference on Computer Engineering, Network, and Intelligent Multimedia (CENIM)10.1109/CENIM64038.2024.10882830(1-6)Online publication date: 19-Nov-2024
https://doi.org/10.1109/CENIM64038.2024.10882830
Cederic LMaximilian BFabian RChristian HAnsgar T(2024)Contextual Anomaly Detection in Hot Forming Production Line using PINN Architecture2024 IEEE International Conference on Advanced Intelligent Mechatronics (AIM)10.1109/AIM55361.2024.10637234(1020-1025)Online publication date: 15-Jul-2024
https://doi.org/10.1109/AIM55361.2024.10637234
Reddy KRithani MRangarajan PMohan G(2023)A Comparative Analysis: Enhancing Baby Cry Detection with Hybrid Deep Learning Techniques2023 International Conference on Next Generation Electronics (NEleX)10.1109/NEleX59773.2023.10421119(1-6)Online publication date: 14-Dec-2023
https://doi.org/10.1109/NEleX59773.2023.10421119
Kumar Pareek PK VN Prasad ADiggi PA D(2023)Classification for Jaundice Symptoms Using Improved Dragonfly with XGBoost Model2023 International Conference on Data Science and Network Security (ICDSNS)10.1109/ICDSNS58469.2023.10245611(1-7)Online publication date: 28-Jul-2023
https://doi.org/10.1109/ICDSNS58469.2023.10245611
Lockhart-Bouron MAnikin APisanski KCorvin SCornec CPapet LLevréro FFauchon CPatural HReby DMathevon N(2023)Infant cries convey both stable and dynamic information about age and identityCommunications Psychology10.1038/s44271-023-00022-z1:1Online publication date: 2-Oct-2023
https://doi.org/10.1038/s44271-023-00022-z
Charola MRathod SPatil H(2023)Robustness of Whisper Features for Infant Cry ClassificationSpeech and Computer10.1007/978-3-031-48312-7_34(421-433)Online publication date: 29-Nov-2023
https://dl.acm.org/doi/10.1007/978-3-031-48312-7_34

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents