Abstract
An increasing number of studies argue for the necessity of an objective method to quantify autism spectrum disorder (ASD) symptoms to replace time-consuming assessments and interviews by trained clinicians. Several attempts at digital phenotyping have been reported, some of which have succeeded in predicting ASD risk based on quantified symptoms by machine learning. Color information analysis is a promising tool for digital phenotyping of ASD symptoms. The potential field of application ranges from detection of atypicality in autonomic function and pupillary response to evaluation of gastrointestinal symptoms. However, some technical and ethical hurdles remain that hamper the ready application of color analysis to digital phenotyping of ASD.


Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders, 5th edn. American Psychiatric Association, Arlington
Jl Matson, Kozlowski AM (2011) The increasing prevalence of autism spectrum disorders. Res Autism Spectr Disord 5(1):418–425
Kanner L (1943) Autistic disturbances of affective contact. Nerv Child 2:217–250
Asperger H (1944) Die “Autistischen Psychopathen” im Kindesalter. Archiv für Psychiatrie und Nervenkrankheiten. 117(1):132–135
Anderson GM (2015) Autism biomarkers: challenges, pitfalls and possibilities. J Autism Dev Disord 45(4):1103–1113
Asif M et al (2020) Identification of biological mechanisms underlying a multidimensional ASD phenotype using machine learning. Transl Psychiatry 10(1):43. https://doi.org/10.1038/s41398-020-0721-1
Washington P et al (2019) Data-driven diagnostics and the potential of mobile artificial intelligence for digital therapeutic phenotyping in computational psychiatry. Biol Psychiatry Cogn Neurosci Neuroimaging. https://doi.org/10.1016/j.bpsc.2019.11.015(Epub ahead of print)
Yahata N et al (2016) A small number of abnormal brain connections predicts adult autism spectrum disorder. Nat Commun 7:11254. https://doi.org/10.1038/ncomms11254
Emerson RW et al (2017) Functional neuroimaging of high-risk 6-month-old infants predicts a diagnosis of autism at 24 months of age. Sci Transl Med. https://doi.org/10.1126/scitranslmed.aag2882
Baranek GT (1999) Autism during infancy: a retrospective video analysis of sensory-motor and social behaviors at 9–12 months of age. J Autism Dev Disord 29(3):213–224
Osterling JA, Dawson G, Munson JA (2002) Early recognition of 1-year-old infants with autism spectrum disorder versus mental retardation. Dev Psychopathol 14(2):239–251
Elsabbagh M, Mercure E, Hudry K, Chandler S, Pasco G, Charman T, Pickles A, Baron-Cohen S, Bolton P, Johnson MH, Team BASIS (2012) Infant neural sensitivity to dynamic eye gaze is associated with later emerging autism. Curr Biol 22(4):338–342
Dawson G, Rogers S, Munson J, Smith M, Winter J, Greenson J, Donaldson A, Varley J (2010) Randomized, controlled trial of an intervention for toddlers with autism: the Early Start Denver Model. Pediatrics 125(1):e17–e23
Baron-Cohen S, Scott FJ, Allison C, Williams J, Bolton P, Matthews FE, Brayne C (2009) Prevalence of autism-spectrum conditions: UK school-based population study. Br J Psychiatry 194(6):500–509
Foran LL (2018) Identifying autism spectrum disorder in undiagnosed adults. Nurse Pract 43(9):14–18
van Niekerk ME, Groen W, Vissers CT, van Driel-de Jong D, Kan CC, Oude Voshaar RC (2011) Diagnosing autism spectrum disorders in elderly people. Int Psychogeriatr 23(5):700–710
Grandin T, Panek R (2013) The autistic brain: thinking across the spectrum, 1st edn. Houghton Mifflin Harcourt, Boston
Loth L, Garrido L, Ahmad J, Watson E, Duff A, Duchaine B (2018) Facial expression recognition as a candidate marker for autism spectrum disorder: how frequent and severe are deficits? Mol Autism 30(9):7. https://doi.org/10.1186/s13229-018-0187-7
Doi H, Fujisawa TX, Kanai C, Ohta H, Yokoi H, Iwanami A, Kato N, Shinohara K (2013) Recognition of facial expressions and prosodic cues with graded emotional intensities in adults with Asperger syndrome. J Autism Dev Disord 43(9):2099–2113. https://doi.org/10.1007/s10803-013-1760-8
Ekman P, Friesen W (1978) Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto
Metallinou A, Grossman RB, Narayanan S (2013) Quantifying atypicality in affective facial expressions of children with autism spectrum disorders. Proc (IEEE Int Conf Multimed Expo) 2013:1–6
Grossard C, Dapogny A, Cohen D, Bernheim S, Juillet E, Hamel F, Hun S, Bourgeois J, Pellerin H, Serret S, Bailly K, Chaby L (2010) Children with autism spectrum disorder produce more ambiguous and less socially meaningful facial expressions: an experimental study using random forest classifiers. Mol Autism 11:5. https://doi.org/10.1186/s13229-020-0312-2
Teitelbaum P, Teitelbaum O, Nye J, Fryman J, Maurer RG (1998) Movement analysis in infancy may be useful for early diagnosis of autism. PNAS 95(23):13982–13987
Fournier KA, Hass CJ, Naik SK, Lodha N, Cauraugh JH (2010) Motor coordination in autism spectrum disorders: a synthesis and meta-analysis. J Autism Dev Disord 40(10):1227–1240
Dawson G, Campbell K, Hashemi J, Lippmann SJ, Smith V, Carpenter K, Egger H, Espinosa S, Vermeer S, Baker J, Sapiro G (2018) Atypical postural control can be detected via computer vision analysis in toddlers with autism spectrum disorder. Sci Rep 8:art.no.17008
Wu D, José JV, Nurnberger JI, Torres EB (2018) A biomarker characterizing neurodevelopment with applications in autism. Sci Rep 8:614
Ardalan A, Assadi AH, Surgent OJ, Travers BG (2019) Whole-body movement during videogame play distinguishes youth with autism from youth with typical development. Sci Rep 9(1):20094. https://doi.org/10.1038/s41598-019-56362-6
Anzulewicz A, Sobota K, Delafield-Butt JT (2016) Toward the autism motor signature: gesture patterns during smart tablet gameplay identify children with autism. Sci Rep 24(6):31107. https://doi.org/10.1038/srep31107
Millar L, McConnachie A, Minnis H, Wilson P, Thompson L, Anzulewicz A, Sobota K, Rowe P, Gillberg C, Delafield-Butt J (2019) Phase 3 diagnostic evaluation of a smart tablet serious game to identify autism in 760 children 3–5 years old in Sweden and the United Kingdom. BMJ Open 9(7):e026226. https://doi.org/10.1136/bmjopen-2018-026226
Guy L, Souders M, Bradstreet L, DeLussey C, Herringto JD (2014) Brief report: emotion regulation and respiratory sinus arrhythmia in autism spectrum disorder. J Autism Dev Disord 44(10):2614–2620
Matsushima K, Matsubayashi J, Toichi M, Funabiki Y, Kato K, Awaya T, Kato T (2016) Unusual sensory features are related to resting-state cardiac vagus nerve activity in autism spectrum disorders. Res Autism Spectr Disord 25:37–46
Kushki A, Drumm E, Pla Mobarak M, Tanel N, Dupuis A, Chau T, Anagnostou E (2013) Investigating the autonomic nervous system response to anxiety in children with autism spectrum disorders. PLos One 8(4):e59730. https://doi.org/10.1371/journal.pone.0059730
Harder R, Malow BA, Goodpaster RL, Iqbal F, Halbower A, Goldman SE, Fawkes DB, Wang L, Shi Y, Baudenbacher F, Diedrich A (2016) Heart rate variability during sleep in children with autism spectrum disorder. Clin Auton Res 26(6):423–432
Smeekens I, Didden R, Verhoeven EW (2015) Exploring the relationship of autonomic and endocrine activity with social functioning in adults with autism spectrum disorders. J Autism Dev Disord 45(2):495–505. https://doi.org/10.1007/s10803-013-1947-z
Jansen LM, Gispen-de Wied CC, Wiegant VM, Westenberg HG, Lahuis BE, van Engeland H (2006) Autonomic and neuroendocrine responses to a psychosocial stressor in adults with autistic spectrum disorder. J Autism Dev Disord 36(7):891–899
Watson LR, Roberts JE, Baranek GT, Mandulak KC, Dalton JC (2012) Behavioral and physiological responses to child-directed speech of children with autism spectrum disorders or typical development. J Autism Dev Disord 42(8):1616–1629. https://doi.org/10.1007/s10803-011-1401-z
Louwerse A, Tulen JH, van der Geest JN, van der Ende J, Verhulst FC, Greaves-Lord K (2014) Autonomic responses to social and nonsocial pictures in adolescents with autism spectrum disorder. Autism Res 7(1):17–27
Poh MZ, McDuff DJ, Picard RW (2010) Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt Express 18(10):10762–10774
Fukunishi M, Kurita K, Yamamoto S, Tsumura N (2017) Non-contact video based estimation of heart rate variability spectrogram from hemoglobin composition. Artif Life Robot 22(4):457–463. https://doi.org/10.1007/s10015-017-0382-1
Fukunishi M, Kurita K, Yamamoto S, Tsumura N (2018) Video based measurement of heart rate and heart rate variability spectrogram from estimated hemoglobin information, 2018. In: IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW), Salt Lake City, UT, pp 1405–14057. https://doi.org/10.1109/cvprw.2018.00180
Bal E, Harden E, Lamb D, Van Hecke AV, Denver JW, Porges SW (2010) Emotion recognition in children with autism spectrum disorders: relations to eye gaze and autonomic state. J Autism Dev Disord 40(3):358–370
Larsen RS, Waters J (2018) Neuromodulatory correlates of pupil dilation. Front Neural Circuit 12:21. https://doi.org/10.3389/fncir.2018.00021
Turi M, Burr DC, Binda P (2018) Pupillometry reveals perceptual differences that are tightly linked to autistic traits in typical adults. eLife 6:7. https://doi.org/10.7554/elife.32399
Nyström P, Gliga T, Nilsson Jobs E, Gredebäck G, Charman T, Johnson MH, Bölte S, Falck-Ytter T (2018) Enhanced pupillary light reflex in infancy is associated with autism diagnosis in toddlerhood. Nat Commun 9(1):1678. https://doi.org/10.1038/s41467-018-03985-4
McAnany JJ, Smith BM, Garland A, Kagen SL (2018) iPhone-based pupillometry: a novel approach for assessing the pupillary light reflex. Optom Vis Sci 95(10):953–958. https://doi.org/10.1097/OPX.0000000000001289
Ferguson BJ, Dovgan K, Takahashi N, Beversdorf DQ (2019) The relationship among gastrointestinal symptoms, problem behaviors, and internalizing symptoms in children and adolescents with autism spectrum disorder. Front Psychiatry 10:194. https://doi.org/10.3389/fpsyt.2019.00194
Wu TC, Wu KL, Hu WL, Sheen JM, Lu CN, Chiang JY, Hung YC (2018) Tongue diagnosis indices for upper gastrointestinal disorders: protocol for a cross-sectional, case-controlled observational study. Medicine (Baltimore) 97(2):e9607. https://doi.org/10.1097/MD.0000000000009607
Li N, Yang J, Zhang J, Liang C, Wang Y, Chen B, Zhao C, Wang J, Zhang G, Zhao D, Liu Y, Zhang L, Yang J, Li G, Gai Z, Zhang L, Zhao G (2019) Correlation of gut microbiome between ASD children and mothers and potential biomarkers for risk assessment. Genomics Proteomics Bioinf 17(1):26–38. https://doi.org/10.1016/j.gpb.2019.01.002(Epub 2019 Apr 23)
Yamamoto S, Tsumura N, Nakaguchi T, Namiki T, Kasahara Y, Terasawa K, Miyake Y (2011) Regional image analysis of the tongue color spectrum. Int J Comput Assist Radiol Surg 6:143–152
Alshaigi K, Albraheem R, Alsaleem K, Zakaria M, Jobeir A, Aldhalaan H (2020) Stigmatization among parents of autism spectrum disorder children in Riyadh, Saudi Arabia. Int J Pediatr Adolesc Med (in press)
Acknowledgements
The author would like to thank Dr Norimichi Tsumura for giving me the permission to use images generated in his study.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Doi, H. Digital phenotyping of autism spectrum disorders based on color information: brief review and opinion. Artif Life Robotics 25, 329–334 (2020). https://doi.org/10.1007/s10015-020-00614-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10015-020-00614-6