Skip to main content
Springer Nature Link
Log in

Digital phenotyping of autism spectrum disorders based on color information: brief review and opinion

  • Original Article
  • Published:
Artificial Life and Robotics Aims and scope Submit manuscript

Abstract

An increasing number of studies argue for the necessity of an objective method to quantify autism spectrum disorder (ASD) symptoms to replace time-consuming assessments and interviews by trained clinicians. Several attempts at digital phenotyping have been reported, some of which have succeeded in predicting ASD risk based on quantified symptoms by machine learning. Color information analysis is a promising tool for digital phenotyping of ASD symptoms. The potential field of application ranges from detection of atypicality in autonomic function and pupillary response to evaluation of gastrointestinal symptoms. However, some technical and ethical hurdles remain that hamper the ready application of color analysis to digital phenotyping of ASD.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders, 5th edn. American Psychiatric Association, Arlington

    Book  Google Scholar 

  2. Jl Matson, Kozlowski AM (2011) The increasing prevalence of autism spectrum disorders. Res Autism Spectr Disord 5(1):418–425

    Article  Google Scholar 

  3. Kanner L (1943) Autistic disturbances of affective contact. Nerv Child 2:217–250

    Google Scholar 

  4. Asperger H (1944) Die “Autistischen Psychopathen” im Kindesalter. Archiv für Psychiatrie und Nervenkrankheiten. 117(1):132–135

    Article  Google Scholar 

  5. Anderson GM (2015) Autism biomarkers: challenges, pitfalls and possibilities. J Autism Dev Disord 45(4):1103–1113

    Article  Google Scholar 

  6. Asif M et al (2020) Identification of biological mechanisms underlying a multidimensional ASD phenotype using machine learning. Transl Psychiatry 10(1):43. https://doi.org/10.1038/s41398-020-0721-1

    Article  Google Scholar 

  7. Washington P et al (2019) Data-driven diagnostics and the potential of mobile artificial intelligence for digital therapeutic phenotyping in computational psychiatry. Biol Psychiatry Cogn Neurosci Neuroimaging. https://doi.org/10.1016/j.bpsc.2019.11.015(Epub ahead of print)

    Article  Google Scholar 

  8. Yahata N et al (2016) A small number of abnormal brain connections predicts adult autism spectrum disorder. Nat Commun 7:11254. https://doi.org/10.1038/ncomms11254

    Article  Google Scholar 

  9. Emerson RW et al (2017) Functional neuroimaging of high-risk 6-month-old infants predicts a diagnosis of autism at 24 months of age. Sci Transl Med. https://doi.org/10.1126/scitranslmed.aag2882

    Article  Google Scholar 

  10. Baranek GT (1999) Autism during infancy: a retrospective video analysis of sensory-motor and social behaviors at 9–12 months of age. J Autism Dev Disord 29(3):213–224

    Article  Google Scholar 

  11. Osterling JA, Dawson G, Munson JA (2002) Early recognition of 1-year-old infants with autism spectrum disorder versus mental retardation. Dev Psychopathol 14(2):239–251

    Article  Google Scholar 

  12. Elsabbagh M, Mercure E, Hudry K, Chandler S, Pasco G, Charman T, Pickles A, Baron-Cohen S, Bolton P, Johnson MH, Team BASIS (2012) Infant neural sensitivity to dynamic eye gaze is associated with later emerging autism. Curr Biol 22(4):338–342

    Article  Google Scholar 

  13. Dawson G, Rogers S, Munson J, Smith M, Winter J, Greenson J, Donaldson A, Varley J (2010) Randomized, controlled trial of an intervention for toddlers with autism: the Early Start Denver Model. Pediatrics 125(1):e17–e23

    Article  Google Scholar 

  14. Baron-Cohen S, Scott FJ, Allison C, Williams J, Bolton P, Matthews FE, Brayne C (2009) Prevalence of autism-spectrum conditions: UK school-based population study. Br J Psychiatry 194(6):500–509

    Article  Google Scholar 

  15. Foran LL (2018) Identifying autism spectrum disorder in undiagnosed adults. Nurse Pract 43(9):14–18

    Article  Google Scholar 

  16. van Niekerk ME, Groen W, Vissers CT, van Driel-de Jong D, Kan CC, Oude Voshaar RC (2011) Diagnosing autism spectrum disorders in elderly people. Int Psychogeriatr 23(5):700–710

    Article  Google Scholar 

  17. Grandin T, Panek R (2013) The autistic brain: thinking across the spectrum, 1st edn. Houghton Mifflin Harcourt, Boston

    Google Scholar 

  18. Loth L, Garrido L, Ahmad J, Watson E, Duff A, Duchaine B (2018) Facial expression recognition as a candidate marker for autism spectrum disorder: how frequent and severe are deficits? Mol Autism 30(9):7. https://doi.org/10.1186/s13229-018-0187-7

    Article  Google Scholar 

  19. Doi H, Fujisawa TX, Kanai C, Ohta H, Yokoi H, Iwanami A, Kato N, Shinohara K (2013) Recognition of facial expressions and prosodic cues with graded emotional intensities in adults with Asperger syndrome. J Autism Dev Disord 43(9):2099–2113. https://doi.org/10.1007/s10803-013-1760-8

    Article  Google Scholar 

  20. Ekman P, Friesen W (1978) Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto

    Google Scholar 

  21. Metallinou A, Grossman RB, Narayanan S (2013) Quantifying atypicality in affective facial expressions of children with autism spectrum disorders. Proc (IEEE Int Conf Multimed Expo) 2013:1–6

    Google Scholar 

  22. Grossard C, Dapogny A, Cohen D, Bernheim S, Juillet E, Hamel F, Hun S, Bourgeois J, Pellerin H, Serret S, Bailly K, Chaby L (2010) Children with autism spectrum disorder produce more ambiguous and less socially meaningful facial expressions: an experimental study using random forest classifiers. Mol Autism 11:5. https://doi.org/10.1186/s13229-020-0312-2

    Article  Google Scholar 

  23. Teitelbaum P, Teitelbaum O, Nye J, Fryman J, Maurer RG (1998) Movement analysis in infancy may be useful for early diagnosis of autism. PNAS 95(23):13982–13987

    Article  Google Scholar 

  24. Fournier KA, Hass CJ, Naik SK, Lodha N, Cauraugh JH (2010) Motor coordination in autism spectrum disorders: a synthesis and meta-analysis. J Autism Dev Disord 40(10):1227–1240

    Article  Google Scholar 

  25. Dawson G, Campbell K, Hashemi J, Lippmann SJ, Smith V, Carpenter K, Egger H, Espinosa S, Vermeer S, Baker J, Sapiro G (2018) Atypical postural control can be detected via computer vision analysis in toddlers with autism spectrum disorder. Sci Rep 8:art.no.17008

    Article  Google Scholar 

  26. Wu D, José JV, Nurnberger JI, Torres EB (2018) A biomarker characterizing neurodevelopment with applications in autism. Sci Rep 8:614

    Article  Google Scholar 

  27. Ardalan A, Assadi AH, Surgent OJ, Travers BG (2019) Whole-body movement during videogame play distinguishes youth with autism from youth with typical development. Sci Rep 9(1):20094. https://doi.org/10.1038/s41598-019-56362-6

    Article  Google Scholar 

  28. Anzulewicz A, Sobota K, Delafield-Butt JT (2016) Toward the autism motor signature: gesture patterns during smart tablet gameplay identify children with autism. Sci Rep 24(6):31107. https://doi.org/10.1038/srep31107

    Article  Google Scholar 

  29. Millar L, McConnachie A, Minnis H, Wilson P, Thompson L, Anzulewicz A, Sobota K, Rowe P, Gillberg C, Delafield-Butt J (2019) Phase 3 diagnostic evaluation of a smart tablet serious game to identify autism in 760 children 3–5 years old in Sweden and the United Kingdom. BMJ Open 9(7):e026226. https://doi.org/10.1136/bmjopen-2018-026226

    Article  Google Scholar 

  30. Guy L, Souders M, Bradstreet L, DeLussey C, Herringto JD (2014) Brief report: emotion regulation and respiratory sinus arrhythmia in autism spectrum disorder. J Autism Dev Disord 44(10):2614–2620

    Article  Google Scholar 

  31. Matsushima K, Matsubayashi J, Toichi M, Funabiki Y, Kato K, Awaya T, Kato T (2016) Unusual sensory features are related to resting-state cardiac vagus nerve activity in autism spectrum disorders. Res Autism Spectr Disord 25:37–46

    Article  Google Scholar 

  32. Kushki A, Drumm E, Pla Mobarak M, Tanel N, Dupuis A, Chau T, Anagnostou E (2013) Investigating the autonomic nervous system response to anxiety in children with autism spectrum disorders. PLos One 8(4):e59730. https://doi.org/10.1371/journal.pone.0059730

    Article  Google Scholar 

  33. Harder R, Malow BA, Goodpaster RL, Iqbal F, Halbower A, Goldman SE, Fawkes DB, Wang L, Shi Y, Baudenbacher F, Diedrich A (2016) Heart rate variability during sleep in children with autism spectrum disorder. Clin Auton Res 26(6):423–432

    Article  Google Scholar 

  34. Smeekens I, Didden R, Verhoeven EW (2015) Exploring the relationship of autonomic and endocrine activity with social functioning in adults with autism spectrum disorders. J Autism Dev Disord 45(2):495–505. https://doi.org/10.1007/s10803-013-1947-z

    Article  Google Scholar 

  35. Jansen LM, Gispen-de Wied CC, Wiegant VM, Westenberg HG, Lahuis BE, van Engeland H (2006) Autonomic and neuroendocrine responses to a psychosocial stressor in adults with autistic spectrum disorder. J Autism Dev Disord 36(7):891–899

    Article  Google Scholar 

  36. Watson LR, Roberts JE, Baranek GT, Mandulak KC, Dalton JC (2012) Behavioral and physiological responses to child-directed speech of children with autism spectrum disorders or typical development. J Autism Dev Disord 42(8):1616–1629. https://doi.org/10.1007/s10803-011-1401-z

    Article  Google Scholar 

  37. Louwerse A, Tulen JH, van der Geest JN, van der Ende J, Verhulst FC, Greaves-Lord K (2014) Autonomic responses to social and nonsocial pictures in adolescents with autism spectrum disorder. Autism Res 7(1):17–27

    Article  Google Scholar 

  38. Poh MZ, McDuff DJ, Picard RW (2010) Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt Express 18(10):10762–10774

    Article  Google Scholar 

  39. Fukunishi M, Kurita K, Yamamoto S, Tsumura N (2017) Non-contact video based estimation of heart rate variability spectrogram from hemoglobin composition. Artif Life Robot 22(4):457–463. https://doi.org/10.1007/s10015-017-0382-1

    Article  Google Scholar 

  40. Fukunishi M, Kurita K, Yamamoto S, Tsumura N (2018) Video based measurement of heart rate and heart rate variability spectrogram from estimated hemoglobin information, 2018. In: IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW), Salt Lake City, UT, pp 1405–14057. https://doi.org/10.1109/cvprw.2018.00180

  41. Bal E, Harden E, Lamb D, Van Hecke AV, Denver JW, Porges SW (2010) Emotion recognition in children with autism spectrum disorders: relations to eye gaze and autonomic state. J Autism Dev Disord 40(3):358–370

    Article  Google Scholar 

  42. Larsen RS, Waters J (2018) Neuromodulatory correlates of pupil dilation. Front Neural Circuit 12:21. https://doi.org/10.3389/fncir.2018.00021

    Article  Google Scholar 

  43. Turi M, Burr DC, Binda P (2018) Pupillometry reveals perceptual differences that are tightly linked to autistic traits in typical adults. eLife 6:7. https://doi.org/10.7554/elife.32399

    Article  Google Scholar 

  44. Nyström P, Gliga T, Nilsson Jobs E, Gredebäck G, Charman T, Johnson MH, Bölte S, Falck-Ytter T (2018) Enhanced pupillary light reflex in infancy is associated with autism diagnosis in toddlerhood. Nat Commun 9(1):1678. https://doi.org/10.1038/s41467-018-03985-4

    Article  Google Scholar 

  45. McAnany JJ, Smith BM, Garland A, Kagen SL (2018) iPhone-based pupillometry: a novel approach for assessing the pupillary light reflex. Optom Vis Sci 95(10):953–958. https://doi.org/10.1097/OPX.0000000000001289

    Article  Google Scholar 

  46. Ferguson BJ, Dovgan K, Takahashi N, Beversdorf DQ (2019) The relationship among gastrointestinal symptoms, problem behaviors, and internalizing symptoms in children and adolescents with autism spectrum disorder. Front Psychiatry 10:194. https://doi.org/10.3389/fpsyt.2019.00194

    Article  Google Scholar 

  47. Wu TC, Wu KL, Hu WL, Sheen JM, Lu CN, Chiang JY, Hung YC (2018) Tongue diagnosis indices for upper gastrointestinal disorders: protocol for a cross-sectional, case-controlled observational study. Medicine (Baltimore) 97(2):e9607. https://doi.org/10.1097/MD.0000000000009607

    Article  Google Scholar 

  48. Li N, Yang J, Zhang J, Liang C, Wang Y, Chen B, Zhao C, Wang J, Zhang G, Zhao D, Liu Y, Zhang L, Yang J, Li G, Gai Z, Zhang L, Zhao G (2019) Correlation of gut microbiome between ASD children and mothers and potential biomarkers for risk assessment. Genomics Proteomics Bioinf 17(1):26–38. https://doi.org/10.1016/j.gpb.2019.01.002(Epub 2019 Apr 23)

    Article  Google Scholar 

  49. Yamamoto S, Tsumura N, Nakaguchi T, Namiki T, Kasahara Y, Terasawa K, Miyake Y (2011) Regional image analysis of the tongue color spectrum. Int J Comput Assist Radiol Surg 6:143–152

    Article  Google Scholar 

  50. Alshaigi K, Albraheem R, Alsaleem K, Zakaria M, Jobeir A, Aldhalaan H (2020) Stigmatization among parents of autism spectrum disorder children in Riyadh, Saudi Arabia. Int J Pediatr Adolesc Med (in press)

Download references

Acknowledgements

The author would like to thank Dr Norimichi Tsumura for giving me the permission to use images generated in his study.

Author information

Authors and Affiliations

  1. School of Science and Engineering, Kokushikan University, 4-28-1 Setagaya, Setagaya, Tokyo, Japan

    Hirokazu Doi

Authors
  1. Hirokazu Doi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hirokazu Doi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Doi, H. Digital phenotyping of autism spectrum disorders based on color information: brief review and opinion. Artif Life Robotics 25, 329–334 (2020). https://doi.org/10.1007/s10015-020-00614-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10015-020-00614-6

Keywords

Search

Navigation