Skip to main content
Springer Nature Link
Log in

Two-Step Fine-Tuned Convolutional Neural Networks for Multi-label Classification of Children’s Drawings

  • Conference paper
  • First Online:
Document Analysis and Recognition – ICDAR 2021 (ICDAR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12822))

Included in the following conference series:

  • 3840 Accesses

Abstract

Developmental psychologists employ several drawing-based tasks to measure the cognitive maturity of a child. Manual scoring of such tests is time-consuming and prone to scorer bias. A computerized analysis of digitized samples can provide efficiency and standardization. However, the inherent variability of hand-drawn traces and lack of sufficient training samples make it challenging for both feature engineering and feature learning. In this paper, we present a two-step fine-tuning based method to train a multi-label Convolutional Neural Network (CNN) architecture, for the scoring of a popular drawing-based test ‘Draw-A-Person’ (DAP). Our proposed two-step fine-tuned CNN architecture outperforms conventional pre-trained CNNs by achieving an accuracy of 81.1% in scoring of Gross Details, 99.2% in scoring of Attachments, and 79.3% in scoring of Head Details categories of DAP samples.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bender, L.: A visual motor gestalt test and its clinical use. Research Monographs, American Orthopsychiatric Association (1938)

    Google Scholar 

  2. Buck, J.N.: The htp technique; a qualitative and quantitative scoring manual. Journal of Clinical Psychology (1948)

    Google Scholar 

  3. Chindaro, S., Guest, R., Fairhurst, M., Potter, J.: Assessing visuo-spatial neglect through feature selection from shape drawing performance and sequence analysis. Int. J. Pattern Recogn. Artif. Intell. 18(07), 1253–1266 (2004)

    Article  Google Scholar 

  4. De Waal, E., Pienaar, A.E., Coetzee, D.: Influence of different visual perceptual constructs on academic achievement among learners in the nw-child study. Percept. Motor Skills 125(5), 966–988 (2018)

    Article  Google Scholar 

  5. Diaz, M., Ferrer, M.A., Impedovo, D., Pirlo, G., Vessio, G.: Dynamically enhanced static handwriting representation for parkinson’s disease detection. Pattern Recogn. Lett. 128, 204–210 (2019)

    Article  Google Scholar 

  6. Drotár, P., Mekyska, J., Rektorová, I., Masarová, L., Smékal, Z., Faundez-Zanuy, M.: Evaluation of handwriting kinematics and pressure for differential diagnosis of parkinson’s disease. Artif. Intell. Med. 67, 39–46 (2016)

    Article  Google Scholar 

  7. Eitz, M., Hays, J., Alexa, M.: How do humans sketch objects? ACM Trans. Graph. (TOG) 31(4), 1–10 (2012)

    Google Scholar 

  8. Eitz, M., Hildebrand, K., Boubekeur, T., Alexa, M.: Sketch-based image retrieval: benchmark and bag-of-features descriptors. IEEE Trans. Visual. Comput. Graph. 17(11), 1624–1636 (2011)

    Article  Google Scholar 

  9. El Shafie, A.M., El Lahony, D.M., Omar, Z.A., El Sayed, S.B., et al.: Screening the intelligence of primary school children using ‘draw a person’ test. Menoufia Med. J. 31(3), 994 (2018)

    Google Scholar 

  10. Fairhurst, M.C., Linnell, T., Glenat, S., Guest, R., Heutte, L., Paquet, T.: Developing a generic approach to online automated analysis of writing and drawing tests in clinical patient profiling. Behav. Res. Methods 40(1), 290–303 (2008)

    Article  Google Scholar 

  11. Farokhi, M., Hashemi, M.: The analysis of children’s drawings: social, emotional, physical, and psychological aspects. Procedia-Soc. Behav. Sci. 30, 2219–2224 (2011)

    Article  Google Scholar 

  12. Gazda, M., Hireš, M., Drotár, P.: Multiple-fine-tuned convolutional neural networks for parkinson’s disease diagnosis from offline handwriting. IEEE Transactions on Systems, Man, and Cybernetics: Systems (2021)

    Google Scholar 

  13. Goodenough, F.L.: Measurement of intelligence by drawings (1926)

    Google Scholar 

  14. Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 855–868 (2008)

    Article  Google Scholar 

  15. Harbi, Z., Hicks, Y., Setchi, R.: Clock drawing test interpretation system. Procedia Comput. Sci. 112, 1641–1650 (2017)

    Article  Google Scholar 

  16. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  17. Khalid, P.I., Yunus, J., Adnan, R., Harun, M., Sudirman, R., Mahmood, N.H.: The use of graphic rules in grade one to help identify children at risk of handwriting difficulties. Res. Dev. Disabil. 31(6), 1685–1693 (2010)

    Article  Google Scholar 

  18. Kornmeier, J., Bach, M.: The necker cube–an ambiguous figure disambiguated in early visual processing. Vis. Res. 45(8), 955–960 (2005)

    Article  Google Scholar 

  19. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)

    Article  Google Scholar 

  20. Larner, A..J.. (ed.): Cognitive Screening Instruments. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-44775-9

    Book  Google Scholar 

  21. Moetesum, M., Aslam, T., Saeed, H., Siddiqi, I., Masroor, U.: Sketch-based facial expression recognition for human figure drawing psychological test. In: 2017 International Conference on Frontiers of Information Technology (FIT), pp. 258–263. IEEE (2017)

    Google Scholar 

  22. Moetesum, M., Siddiqi, I., Ehsan, S., Vincent, N.: Deformation modeling and classification using deep convolutional neural networks for computerized analysis of neuropsychological drawings. Neural Comput. Appl. 32(16), 12909–12933 (2020). https://doi.org/10.1007/s00521-020-04735-8

    Article  Google Scholar 

  23. Moetesum, M., Siddiqi, I., Masroor, U., Djeddi, C.: Automated scoring of bender gestalt test using image analysis techniques. In: 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 666–670. IEEE (2015)

    Google Scholar 

  24. Moetesum, M., Siddiqi, I., Vincent, N.: Deformation classification of drawings for assessment of visual-motor perceptual maturity. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 941–946. IEEE (2019)

    Google Scholar 

  25. Moetesum, M., Siddiqi, I., Vincent, N., Cloppet, F.: Assessing visual attributes of handwriting for prediction of neurological disorders–a case study on parkinson’s disease. Pattern Recogn. Lett. 121, 19–27 (2019)

    Article  Google Scholar 

  26. Naseer, A., Rani, M., Naz, S., Razzak, M.I., Imran, M., Xu, G.: Refining parkinson’s neurological disorder identification through deep transfer learning. Neural Comput. Appl. 32(3), 839–854 (2020)

    Article  Google Scholar 

  27. Nazar, H.B., et al.: Classification of graphomotor impressions using convolutional neural networks: an application to automated neuro-psychological screening tests. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 432–437. IEEE (2017)

    Google Scholar 

  28. Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1717–1724 (2014)

    Google Scholar 

  29. Pereira, C.R., Weber, S.A., Hook, C., Rosa, G.H., Papa, J.P.: Deep learning-aided parkinson’s disease diagnosis from handwritten dynamics. In: 2016 29th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 340–346. IEEE (2016)

    Google Scholar 

  30. Pratt, H.D., Greydanus, D.E.: Intellectual disability (mental retardation) in children and adolescents. Primary Care Clin. Office Pract. 34(2), 375–386 (2007)

    Article  Google Scholar 

  31. Pullman, S.L.: Spiral analysis: a new technique for measuring tremor with a digitizing tablet. Mov. Disord. 13(S3), 85–89 (1998)

    Article  Google Scholar 

  32. Rémi, C., Frélicot, C., Courtellemont, P.: Automatic analysis of the structuring of children’s drawings and writing. Pattern Recogn. 35(5), 1059–1069 (2002)

    Article  Google Scholar 

  33. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: Towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497 (2015)

  34. Shin, M.S., Park, S.Y., Park, S.R., Seol, S.H., Kwon, J.S.: Clinical and empirical applications of the rey-osterrieth complex figure test. Nat. Protoc. 1(2), 892 (2006)

    Article  Google Scholar 

  35. Shulman, K.I., Shedletsky, R., Silver, I.L.: The challenge of time: clock-drawing and cognitive function in the elderly. Int. J. Geriatr. Psychiatry 1(2), 135–140 (1986)

    Article  Google Scholar 

  36. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  37. Smith, A.D.: On the use of drawing tasks in neuropsychological assessment. Neuropsychology 23(2), 231 (2009)

    Article  Google Scholar 

  38. Smith, S.L., Hiller, D.L.: Image analysis of neuropsychological test responses. In: Medical Imaging 1996: Image Processing, vol. 2710, pp. 904–915. International Society for Optics and Photonics (1996)

    Google Scholar 

  39. Smith, S.L., Lones, M.A.: Implicit context representation cartesian genetic programming for the assessment of visuo-spatial ability. In: 2009 IEEE Congress on Evolutionary Computation, pp. 1072–1078. IEEE (2009)

    Google Scholar 

  40. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)

    Google Scholar 

  41. Tabatabaey-Mashadi, N., Sudirman, R., Guest, R.M., Khalid, P.I.: Analyses of pupils’ polygonal shape drawing strategy with respect to handwriting performance. Pattern Anal. Appl. 18(3), 571–586 (2015)

    Article  MathSciNet  Google Scholar 

  42. Tabatabaey, N., Sudirman, R., Khalid, P.I., et al.: An evaluation of children’s structural drawing strategies. Jurnal Teknologi, vol. 61, no. 2 (2013)

    Google Scholar 

  43. Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? arXiv preprint arXiv:1411.1792 (2014)

Download references

Author information

Authors and Affiliations

  1. Bahria University, Islamabad, Pakistan

    Muhammad Osama Zeeshan, Imran Siddiqi & Momina Moetesum

Authors
  1. Muhammad Osama Zeeshan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Imran Siddiqi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Momina Moetesum
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Momina Moetesum .

Editor information

Editors and Affiliations

  1. Universitat Autònoma de Barcelona, Barcelona, Spain

    Josep Lladós

  2. Lehigh University, Bethlehem, PA, USA

    Daniel Lopresti

  3. Kyushu University, Fukuoka-shi, Japan

    Seiichi Uchida

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zeeshan, M.O., Siddiqi, I., Moetesum, M. (2021). Two-Step Fine-Tuned Convolutional Neural Networks for Multi-label Classification of Children’s Drawings. In: Lladós, J., Lopresti, D., Uchida, S. (eds) Document Analysis and Recognition – ICDAR 2021. ICDAR 2021. Lecture Notes in Computer Science(), vol 12822. Springer, Cham. https://doi.org/10.1007/978-3-030-86331-9_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86331-9_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86330-2

  • Online ISBN: 978-3-030-86331-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships