Abstract
Autism Spectrum Disorder (ASD) is a congenital neurodevelopmental disorder, and the number of ASD has been increasing in recent decades worldwide. Early screening is essential for proper treatment and intervention in toddlers with ASD. However, manual early screening methods for ASD are costly and inefficient. Stereotyped behavior is one of the clinical manifestations of ASD toddlers. In this paper, we propose a vision-based action detection network, named OstAD, for response-to-instruction (RTI) protocol to assist professional clinicians with an early screening. Our network adopts a temporal attention branch to aggregate contextual features, and proposes a spatial attention branch to generate local frame-level features of the toddlers. Experimental results demonstrate that the proposed OstAD model can detect typical actions of ASD toddler with mAP 72.6% and 75.9% accuracy, and achieves the excellent results in the RTI screening.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Wang, Z., Liu, J., He, K., Xu, Q., Xu, X., Liu, H.: Screening early children with autism spectrum disorder via response-to-name protocol. IEEE Trans. Industr. Inf. 17(1), 587–595 (2019)
Zwaigenbaum, L., et al.: Early identification of autism spectrum disorder: recommendations for practice and research. Pediatrics 136(Supplement_1), S10–S40 (2015)
Fernell, E., Eriksson, M.A., Gillberg, C.: Early diagnosis of autism and impact on prognosis: a narrative review. Clin. Epidemiol. 5, 33 (2013)
Lord, C., Cook, E.H., Leventhal, B.L., Amaral, D.G.: Autism spectrum disorders. Neuron 28(2), 355–363 (2000)
Jiang, S., Qi, Y., Zhang, H., Bai, Z., Lu, X., Wang, P.: D3D: dual 3-D convolutional network for real-time action recognition. IEEE Trans. Industr. Inf. 17(7), 4584–4593 (2020)
Q. Liu and Z. Wang, “Progressive boundary refinement network for temporal action detection," in AAAI Conference on Artificial Intelligence, vol. 34, no. 07, 2020, pp. 11 612–11 619
Heilbron, F.C., Escorcia, V., Ghanem, B., Niebles, J.C.: ActivityNet: a large-scale video benchmark for human activity understanding. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 961–970 (2015)
Jiang, Y.-G., et al.: Thumos challenge: action recognition with a large number of classes (2014)
Liu, J., et al.: Early screening of autism in toddlers via response-to-instructions protocol. IEEE Trans. Cybern. (2020)
Tawhid, M.N.A., Siuly, S., Wang, H., Whittaker, F., Wang, K., Zhang, Y.: A spectrogram image based intelligent technique for automatic detection of autism spectrum disorder from EEG. PLoS ONE 16(6), e0253094 (2021)
Husna, R.N.S., Syafeeza, A., Hamid, N.A., Wong, Y., Raihan, R.A.: Functional magnetic resonance imaging for autism spectrum disorder detection using deep learning. Jurnal Teknologi 83(3), 45–52 (2021)
Krupa, N., Anantharam, K., Sanker, M., Datta, S., Sagar, J.V.: Recognition of emotions in autistic children using physiological signals. Heal. Technol. 6(2), 137–147 (2016). https://doi.org/10.1007/s12553-016-0129-3
Rad, N.M., et al.: Deep learning for automatic stereotypical motor movement detection using wearable sensors in autism spectrum disorders. Signal Process. 144, 180–191 (2018)
Y. Tian, X. Min, G. Zhai, and Z. Gao, “Video-based early asd detection via temporal pyramid networks," in 2019 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2019, pp. 272–277
Zhang, W., Wang, Z., Cai, H., Liu, H.: Detection for joint attention based on a multi-sensor visual system. In: 2018 25th International Conference on Mechatronics and Machine Vision in Practice (M2VIP), pp. 1–6. IEEE (2018)
Zhou, J., Jiang, Z., Yoo, J.-H., Hwang, J.-N.: Hierarchical pose classification for infant action analysis and mental development assessment. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1340–1344. IEEE (2021)
Qing, Z., et al.: Temporal context aggregation network for temporal action proposal refinement. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 485–494 (2021)
Wang, H., Yuan, C., Shen, J., Yang, W., Ling, H.: Action unit detection and key frame selection for human activity prediction. Neurocomputing 318, 109–119 (2018)
Wang, D., Yuan, Y., Wang, Q.: Gated forward refinement network for action segmentation. Neurocomputing 407, 63–71 (2020)
Li, Y., et al.: Efficient two-step networks for temporal action segmentation. Neurocomputing 454, 373–381 (2021)
Wang, T., Chen, Y., Lv, H., Teng, J., Snoussi, H., Tao, F.: Online detection of action start via soft computing for smart city. IEEE Trans. Industr. Inf. 17(1), 524–533 (2020)
Redmon, J., Divvala, S.K., Girshick, R.B., Farhadi, A.: You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016. IEEE Computer Society, pp. 779–788 (2016). https://doi.org/10.1109/CVPR.2016.91
Carreira, J., Zisserman, A.: Quo vadis, action recognition? A new model and the kinetics dataset. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 6299–6308 (2017)
Shou, Z., Wang, D., Chang, S.-F.: Temporal action localization in untrimmed videos via multi-stage CNNs. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1049–1058 (2016)
Yang, L., Han, J., Zhang, D.: Colar: effective and efficient online action detection by consulting exemplars. arXiv preprint arXiv:2203.01057 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shi, Y., Ren, W., Jiang, W., Xu, Q., Xu, X., Liu, H. (2022). Vision-Based Action Detection for RTI Protocol of ASD Early Screening. In: Liu, H., et al. Intelligent Robotics and Applications. ICIRA 2022. Lecture Notes in Computer Science(), vol 13455. Springer, Cham. https://doi.org/10.1007/978-3-031-13844-7_36
Download citation
DOI: https://doi.org/10.1007/978-3-031-13844-7_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-13843-0
Online ISBN: 978-3-031-13844-7
eBook Packages: Computer ScienceComputer Science (R0)