Skip to main content

Advertisement

SpringerLink
Log in

Detecting Children’s Fine Motor Skill Development using Machine Learning

  • ARTICLE
  • Published:
International Journal of Artificial Intelligence in Education Aims and scope Submit manuscript

Abstract

Children’s fine motor skills are linked not only to drawing ability but also to cognitive, social-emotional, self-regulatory, and academic development Suggate et al. Journal of Research in Reading, 41(1), 1–19 (2018), Benedetti et al. (2014), Liew et al. Early Education & Development, 22(4), 549–573 (2011), Liew (2012) and Xie et al. (2014). Current educators are assessing children’s fine motor skills by either determining their shape drawing correctness Meisels et al. (1997) or measuring their drawing time duration Kochanska et al. (1997) and Liew et al. (2011) through paper-based assessments. However, these methods involve human experts manually analyzing children’s fine motor skills, which can be time consuming and prone to human error or bias Kim et al. (2013) and Lotz et al. (2005). With many children using sketch-based applications on mobile devices like smartphones or tablets Anthony et al. (2012), computer-based fine motor skill assessment has the potential to address limitations of paper-based assessment by using automated measurements. In this work, we introduce a machine learning approach for analyzing aspects of children’s fine motor skill development. We performed a study with 60 young children (aged 3 to 8 years old), and we implemented classifiers that determine children’s age category based on features related to fine motor skill, predominantly for curvature- and corner-based drawing skills, surpassing the performance of our previous work Kim et al. (2013) and of human evaluators. We also present dedicated discussion and statistical testing of sketch recognition features which will further enhance automated fine motor assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Alamudun, F.T., Hammond, T., Yoon, H.J., & Tourassi, G.D. (2017). Geometry and gesture-based features from saccadic eye-movement as a biometric in radiology. In International conference on augmented cognition, pp. 123–138. Springer.

  • Anthony, L., Brown, Q., Nias, J., Tate, B., & Mohan, S. (2012). Interaction and recognition challenges in interpreting children’s touch and gesture input on mobile devices. In Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces, pp. 225–234.

  • Benedetti, L., Winnemöller, H., Corsini, M., & Scopigno, R. (2014). Painting with bob: Assisted creativity for novices. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST ’14 (pp. 419–428). USA: ACM, New York, NY.

  • Bhat, A., & Hammond, T. (2009). Using entropy to distinguish shape versus text in hand-drawn diagrams. In IJCAI, vol. 9, pp. 1395–400.

  • Blagojevic, R., Chang, S.H.H., & Plimmer, B. (2010). The power of automatic feature selection: Rubine on steroids. In Proceedings of the Seventh Sketch-Based Interfaces and Modeling Symposium, SBIM ’10 (pp. 79–86). Switzerland: Aire-la-Ville.

  • Cahill, S.M. (2009). Where does handwriting fit in? strategies to support academic achievement. Intervention in school and clinic, 44(4), 223–228.

    Article  Google Scholar 

  • Calhoun, C., Stahovich, T.F., Kurtoglu, T., & Kara, L.B. (2002). Recognizing multi-stroke symbols. In AAAI Spring symposium on sketch understanding, pp. 15–23.

  • Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1), 37–46.

    Article  Google Scholar 

  • Cohen, J., Penney, D.L., Davis, R., Libon, D.J., Swenson, R.A., Ajilore, O., Kumar, A., & Lamar, M. (2014). Digital clock drawing: differentiating “thinking” versus “doing” in younger and older adults with depression. Journal of the International Neuropsychological Society, 20(9), 920–928.

    Article  Google Scholar 

  • Crosser, S. (2009). When children draw. Early Childhood News http://www.earlychildhoodnews.com/earlychildhood/article_view. aspx.

  • Delaye, A., & Anquetil, E. (2013). Hbf49 feature set: a first unified baseline for online symbol recognition. Pattern Recognition, 46(1), 117–130.

    Article  Google Scholar 

  • Dennis, T.A. (2007). Effortful control, social competence, and adjustment problems in children at risk for psychopathology. In Journal of clinical child and adolescent psychology, vol. 36, pp. 442–454.

  • DiCiccio, T.J., & Efron, B. (1996). Bootstrap confidence intervals. Statistical science, pp 189–212.

  • on Children with Disabilities, C., et al. (2001). Developmental surveillance and screening of infants and young children. Pediatrics, 108(1), 192–195.

    Article  Google Scholar 

  • Duncan, G.J., Dowsett, C.J., Claessens, A., Magnuson, K., Huston, A.C., Klebanov, P., Pagani, L.S., Feinstein, L., Engel, M., Brooks-Gunn, J., & et al. (2007). School readiness and later achievement. Developmental psychology, 43(6), 1428.

    Article  Google Scholar 

  • Fields, M.V. (1998). Your child learns to read & write ERIC.

  • Fleiss, J.L., Levin, B., & Paik, M.C. (2013). Statistical methods for rates and proportions john wiley & sons.

  • Fonseca, M.J., Pimentel, C., & Jorge, J.A. (2002). Cali: an online scribble recognizer for calligraphic interfaces. In Sketch understanding, papers from the 2002 AAAI spring symposium, pp. 51–58.

  • Forbus, K., Usher, J., Lovett, A., Lockwood, K., & Wetzel, J. (2011). Cogsketch: Sketch understanding for cognitive science research and for education. Topics in Cognitive Science, 3(4), 648–666.

    Article  Google Scholar 

  • Glascoe, F.P. (2000). Early detection of developmental and behavioral problems. Pediatrics in Review, 21(8), 272–280.

    Article  Google Scholar 

  • Grissmer, D., Grimm, K.J., Aiyer, S.M., Murrah, W.M., & Steele, J.S. (2010). Fine motor skills and early comprehension of the world: two new school readiness indicators. Developmental psychology, 46(5), 1008.

    Article  Google Scholar 

  • Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I.H. (2009). The weka data mining software: an update. SIGKDD Explor Newsl., 11(1), 10–18.

    Article  Google Scholar 

  • Hammond, T., Kumar, S.P.A., Runyon, M., Cherian, J., Williford, B., Keshavabhotla, S., Valentine, S., Li, W., & Linsey, J. (2018). It’s not just about accuracy: Metrics that matter when modeling expert sketching ability. ACM Transactions on Interactive Intelligent Systems (TiiS), 8(3), 1–47.

    Article  Google Scholar 

  • Hse, H., & Newton, A.R. (2004). Sketched symbol recognition using zernike moments. In Proceedings of the Pattern Recognition, 17th International Conference on (ICPR’04) Volume 1 - Volume 01, ICPR ’04 (pp. 367–370). USA: IEEE Computer Society, Washington, DC.

  • Kilday, C.R., Kinzie, M.B., Mashburn, A.J., & Whittaker, J.V. (2012). Accuracy of teacher judgments of preschoolers’ math skills. Journal of Psychoeducational Assessment, 30(2), 148–159.

    Article  Google Scholar 

  • Kim, H., Taele, P., Valentine, S., McTigue, E., & Hammond, T. (2013). Kimchi: a sketch-based developmental skill classifier to enhance pen-driven educational interfaces for children. In Proceeding SBIM 1́3 Proceedings of the International Symposium on Sketch-Based Interfaces and Modeling, pp. 33–42.

  • Kim, H., Valentine, S., Taele, P., & Hammond, T. (2014). Easysketch: a sketch-based educational interface to support children’s self-regulation and school readiness. In Proceedings of the Workshop on the Impact of Pen and Touch Technology on Education (WIPTTE) 2014.

  • Kochanska, G., Murray, K., & Coy, K.C. (1997). Inhibitory control as a contributor to con- science in childhood: from toddler to early school age. In Child development, vol. 68, pp. 263–277.

  • Landis, J.R., & Koch, G.G. (1977). The measurement of observer agreement for categorical data. biometrics pp 159–174.

  • Lara-Garduno, R., & Igarashi, T. (2019). Hammond, t.: 3d-trail-making test: a touch-tablet cognitive test to support intelligent behavioral recognition. In Proceedings of the 45th Graphics Interface Conference on Proceedings of Graphics Interface 2019, pp. 1–8. Canadian Human-Computer Communications Society.

  • Lara-Garduno, R., Leslie, N., & Hammond, T. (2016). Smartstrokes: digitizing paper-based neuropsychological tests. In Revolutionizing education with digital ink, pp. 163–175. Springer.

  • Lara-Garduno, R.A. (2018). Machine learning behavioral recognition to support neuropsychological diagnosis of cognitive decline. In 23Rd international conference on intelligent user interfaces, pp. 667–668.

  • Leung, W.H., & Chen, T. (2002). User-independent retrieval of free-form hand-drawn sketches. In 2002 IEEE International conference on acoustics, speech, and signal processing, vol. 2, pp. II–2029. IEEE.

  • Levy, R. (2009). ‘you have to understand words… but not read them’: Young children becoming readers in a digital age. Journal of Research in Reading, 32(1), 75–91.

    Article  Google Scholar 

  • Liew, J. (2012). Effortful control, executive functions, and education: Bringing self-regulatory and social-emotional competencies to the table. In Child development perspectives, vol. 6, pp. 105–111.

  • Liew, J., Johnson, A.Y., Smith, T.R., & Thoemmes, F. (2011). Parental expressivity, child physiological and behavioral regulation, and child adjustment: Testing a three-path mediation model. Early Education & Development, 22(4), 549–573.

    Article  Google Scholar 

  • Liikanen, P. (1975). Increasing creativity through art education among pre-school children. Jyväskylä studies in education, psychology, and social research. University of Jyväskylä. http://books.google.com/books?id=fvANAQAAIAAJ.

  • Long, A.C., Landay, J.A., Rowe, L.A., & Michiels, J. (2000). Visual similarity of pen gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’00 (pp. 360–367). USA: ACM, New York, NY.

  • Lotz, L., Loxton, H., & Naidoo, A. (2005). Visual-motor integration functioning in a south african middle childhood sample. Journal of Child & Adolescent Mental Health, pp 63–67.

  • Mashburn, A.J., Hamre, B.K., Downer, J.T., & Pianta, R.C. (2006). Teacher and classroom characteristics associated with teachers’ ratings of prekindergartners’ relationships and behaviors. Journal of psychoeducational assessment, 24(4), 367–380.

    Article  Google Scholar 

  • Meisels, S.J., Marsden, D.B., Wiske, M.S., & Henderson, L.W. (1997). Early screening Inventory-Revised NY: pearson early learning.

  • Morphett, M.V., & Washburne, C. (1931). When should children begin to read? The Elementary School Journal, 31(7), 496–503.

    Article  Google Scholar 

  • Newcombe, R.G. (1998). Two-sided confidence intervals for the single proportion: comparison of seven methods. Statistics in medicine, 17(8), 857–872.

    Article  Google Scholar 

  • Patel, R., Plimmer, B., Grundy, J., & Ihaka, R. (2007). Ink features for diagram recognition. In: Proceedings of the Fourth Eurographics Workshop on Sketch-based Interfaces and Modeling, SBIM ’07, pp. 131–138. ACM, New York, NY, USA. https://doi.org/10.1145/1384429.1384457.

  • Paulson, B., & Hammond, T. (2007). A system for recognizing and beautifying low-level sketch shapes using ndde and dcr. In ACM Symposium on user interface software and technology (UIST), p. 2 pages. ACM, Newport Rhode Island, USA.

  • Paulson, B., & Hammond, T. (2008). Paleosketch: Accurate primitive sketch recognition and beautification. In Proceedings of the 13th International Conference on Intelligent User Interfaces (IUI), p. 10 pages. ACM, Canary Islands, Spain, Spain. https://doi.org/10.1145/1378773.1378775. ISBN: 978-1-59593-987-6.

  • Paulson, B., Rajan, P., Davalos, P., Osuna, R., & Hammond, T. (2008). What!?! no rubine features?: Using geometric-based features to produce normalized confidence values for sketch recognition. In VL/HCC Workshop: sketch tools for diagramming herrsching am ammersee, pp. 56–63.

  • Piaget, J. (1983). Piaget’s Theory. In P. Mussen (Ed.) Handbook of Child Psychology. Wiley & Sons, New York, NY USA.

  • Rubine, D. (1991). Specifying gestures by example. In Proceeding of the 18th annual conference on computer graphics and interactive techniques, SIGGRAPH 91, pp. 329–337.

  • Satlow, E., & Newcombe, N. (1998). When is a triangle not a triangle? young children’s developing concepts of geometric shape. Cognitive Development, 13(4), 547–559.

    Article  Google Scholar 

  • Sezgin, T.M., Stahovich, T., & Davis, R. (2006). Sketch based interfaces: Early processing for sketch understanding. In ACM SIGGRAPH 2006 Courses, SIGGRAPH ’06. ACM, New York, NY, USA.

  • Simpson, A., Al Ruwaili, R., Jolley, R., Leonard, H., Geeraert, N., & Riggs, K.J. (2019). Fine motor control underlies the association between response inhibition and drawing skill in early development. Child development, 90(3), 911–923.

    Article  Google Scholar 

  • Suggate, S., Pufke, E., & Stoeger, H. (2018). Do fine motor skills contribute to early reading development? Journal of Research in Reading, 41(1), 1–19.

    Article  Google Scholar 

  • Taele, P., Koh, J.I., & Hammond, T. (2020). Kanji workbook: a writing-based intelligent tutoring system for learning proper japanese kanji writing technique with instructor-emulated assessment. In AAAI, pp. 13382–13389.

  • Taranta, E.M., & LaViola, Jr J.J. (2015). Math boxes: a pen-based user interface for writing difficult mathematical expressions. In Proceedings of the 20th International Conference on Intelligent User Interfaces, pp. 87–96.

  • Toomela, A. (2002). Drawing as a verbally mediated activity: a study of relationships between verbal, motor, and visuospatial skills and drawing in children. International Journal of Behavioral Development, 26(3), 234–247.

    Article  Google Scholar 

  • Valentine, S., Vides, F., Lucchese, G., Turner, D., Kim, H.H., Li, W., Linsey, J., & Hammond, T. (2012). Mechanix: A sketch-based tutoring system for statics courses. In Twenty-fourth IAAI conference.

  • Vatavu, R., Cramariuc, G., & Schipor, D. (2015). Touch interaction for children aged 3 to 6 years: Experimental findings and relationship to motor skills. International Journal of Human-Computer Studies, 74, 54–76.

    Article  Google Scholar 

  • Voigt, R.G., Llorente, A.M., Jensen, C.L., Fraley, J.K., Barbaresi, W.J., & Heird, W.C. (2007). Comparison of the validity of direct pediatric developmental evaluation versus developmental screening by parent report. Clinical pediatrics, 46(6), 523–529.

    Article  Google Scholar 

  • Williford, B. (2017). Sketchtivity: Improving creativity by learning sketching with an intelligent tutoring system. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition, pp. 477–483.

  • Williford, B., Doke, A., Pahud, M., Hinckley, K., & Hammond, T. (2019). Drawmyphoto: Assisting novices in drawing from photographs. In Proceedings of the 2019 on Creativity and Cognition, pp. 198–209. ACM.

  • Wolin, A., Eoff, B., & Hammond, T. (2008). Shortstraw: a simple and effective corner finder for polylines. In SBM, pp. 33–40.

  • Wolin, A., Eoff, B., & Hammond, T. (2008). Shortstraw: a simple and effective corner finder for polylines. In Proceedings of the Fifth Eurographics Conference on Sketch-Based Interfaces and Modeling, SBM’08 (pp. 33–40). Switzerland: Eurographics Association, Aire-la-Ville, Switzerland.

  • Xie, J., Hertzmann, A., Li, W., & Winnemöller, H. (2014). Portraitsketch: Face sketching assistance for novices. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST ’14 (pp. 407–417). USA: ACM, New York, NY.

  • Xiong, Y., & LaViola, Jr J.J. (2010). A shortstraw-based algorithm for corner finding in sketch-based interfaces. Computers & Graphics, 34(5), 513–527.

Download references

Acknowledgements

The authors are grateful to Juliet Nyanamba for lending her time and expertise to the task of labeling children’s sketches.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Texas A&M University, College Station, TX, 77843, USA

    Seth Polsley, Larry Powell, Xien Thomas & Tracy Hammond

  2. Samsung Research America, Burlington, MA, 01803, USA

    Hong-Hoe Kim

  3. Department of Educational Psychology, Texas A&M University, College Station, TX, 77843, USA

    Jeffrey Liew

Authors
  1. Seth Polsley
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Larry Powell
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hong-Hoe Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xien Thomas
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jeffrey Liew
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tracy Hammond
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seth Polsley.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: Feature List

Appendix: Feature List

Table 6 List of features with their descriptions and sources
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Polsley, S., Powell, L., Kim, HH. et al. Detecting Children’s Fine Motor Skill Development using Machine Learning. Int J Artif Intell Educ 32, 991–1024 (2022). https://doi.org/10.1007/s40593-021-00279-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40593-021-00279-7

Keywords

Search

Navigation