Skip to main content
SpringerLink
Log in

The Importance of Pen Motion Pattern Groups for Semi-Automatic Classification of Handwriting into Mental Workload Classes

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

In this paper, we introduce the pen motion pattern groups (PMPGs) and their contribution to the classification of handwriting into cognitive mental workload classes. We demonstrate the importance of PMPGs by providing an efficient semi-automatic machine learning-based classification framework that distinguishes between handwritten texts written by the same person under different mental workloads. Our evaluation framework is non-language-dependent since we used stroke features, which are not language-specific, and it takes into account the variability in behavioral biometrics between different writers. The handwritten text data was collected using the Computerized Penmanship Evaluation Tool. This digitizer provided accurate temporal measures throughout the writing. As a first stage, the participants were asked to write a given text in the Hebrew language. Then, as a second stage, the participants’ cognitive workload was manipulated by asking them to hold a number in their memory during the entire writing task. In our experiments, we show that incorporating the PMPGs into the classification process yielded an average cognitive load discrimination accuracy of 92.16%, which decreased to 72.90% when the PMPGs were not considered. The separation of handwritten strokes into PMPGs allows us to account for the fact that the strokes are affected differently under different cognitive mental workloads. This novel distinction between PMPGs is important since the handwriting process in each PMPG is different from a perceptual motor and brain-hand control point of view. Moreover, most of the features that are influenced by cognitive workload are those that cannot be discerned by an expert when looking at a handwritten text on paper, such as azimuth, tilt, velocity, acceleration, and pressure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Alonso-Martinez C, Faundez-Zanuy M, Mekyska J. A comparative study of in-air trajectories at short and long distances in online handwriting. Cogn Comput 2017;9(5):712–20.

    Article  Google Scholar 

  2. Breiman L. Random forests. Mach Learn 2001;45(1):5–32.

    Article  Google Scholar 

  3. Cambria E, Hussain A. 2012. Sentic computing: techniques, tools, and applications, vol 2. Springer Science & Business Media.

  4. Drotar P, Mekyska J, Rektorova I, Masarova L, Smekal Z, Faundez-Zanuy M. Decision support framework for Parkinson’s disease based on novel handwriting markers. IEEE Trans Neural Syst Rehabil Eng 2015;23(3):508–16.

    PubMed  Google Scholar 

  5. Drotár P, Mekyska J, Rektorová I, Masarová L, Smékal Z, Faundez-Zanuy M. Evaluation of handwriting kinematics and pressure for differential diagnosis of Parkinson’s disease. Artif Intell Med 2016;67:39–46.

    Article  PubMed  Google Scholar 

  6. Faundez-Zanuy M, Sesa-Nogueras E, Roure-Alcobé J. On the relevance of age in handwritten biometric recognition. IEEE international carnahan conference on security technology (ICCST); 2012. p. 105–109.

  7. Faundez-Zanuy M, Sesa-Nogueras E, Roure-Alcobé J, Esposito A, Mekyska J, López de Ipiña K. A preliminary study on aging examining online handwriting. 5th IEEE conference on cognitive infocommunications (CogInfoCom); 2014. p. 221–224.

  8. Faundez-Zanuy M, Hussain A, Mekyska J, Sesa-Nogueras E, Monte-Moreno E, Esposito A, Chetouani M, Garre-Olmo J, Abel A, Smekal Z, Lopez de Ipina K. Biometric applications related to human beings: there is life beyond security. Cogn Comput 2013;5(1):136–51.

    Article  Google Scholar 

  9. Fierrez J, Ortega-Garcia J, Ramos D, Gonzalez-Rodriguez J. Hmm-based on-line signature verification: feature extraction and signature modeling. Pattern Recogn Lett 2007;28(16):2325–34.

    Article  Google Scholar 

  10. Froese T, Suzuki K, Ogai Y, Ikegami T. Using human–computer interfaces to investigate ‘mind-as-it-could-be’ from the first-person perspective. Cogn Comput 2012;4(3):365–82.

    Article  Google Scholar 

  11. Gomez-Barrero M, Galbally J, Fierrez J, Ortega-Garcia J, Plamondon R. Enhanced on-line signature verification based on skilled forgery detection using sigma-lognormal features. 2015 international conference on biometrics (ICB). IEEE; 2015. p. 501–506.

  12. Górska Z, Janicki A. Recognition of extraversion level based on handwriting and support vector machines. Percept Mot Skills 2012;114(3):857–69.

    Article  PubMed  Google Scholar 

  13. Gramzow RH, Gaertner L, Sedikides C. Memory for in-group and out-group information in a minimal group context: the self as an informational base. J Pers Soc Psychol 2001;80(2):188–205.

    Article  CAS  PubMed  Google Scholar 

  14. Hussain A, Niazi M. Toward a formal, visual framework of emergent cognitive development of scholars. Cogn Comput 2013;6(1):113–24.

    Article  Google Scholar 

  15. Kandel S, Valdois S. Syllables as functional units in a copying task. Lang Cogn Process 2006;21(4):432–52.

    Article  Google Scholar 

  16. Koole SL, Dijksterhuis A, van Knippenberg A. What’s in a name: implicit self-esteem and the automatic self. J Pers Soc Psychol 2001;80(4):669–85.

    Article  CAS  PubMed  Google Scholar 

  17. Kotsavasiloglou C, Kostikis N, Hristu-Varsakelis D, Arnaoutoglou M. Machine learning-based classification of simple drawing movements in Parkinson’s disease. Biomed Signal Process Control 2016;31(1):174–80.

    Google Scholar 

  18. Kotsiantis SB. Supervised machine learning: a review of classification techniques. Proceedings of the 2007 conference on emerging artificial intelligence applications in computer engineering. Amsterdam, The Netherlands; 2007. p. 3–24.

  19. Likforman-Sulem L, Esposito A, Faundez-Zanuy M, Clémençon S, Cordasco G. Emothaw: a novel database for emotional state recognition from handwriting and drawing. IEEE Transactions on Human-Machine Systems 2017;47(2):273–84.

    Article  Google Scholar 

  20. Likforman-Sulem L, Esposito A, Faundez-Zanuy M, Clémençon S, Bassis S, Esposito A, Morabito CF. Extracting style and emotion from handwriting. Advances in neural networks: computational and theoretical issues; 2015. p. 347–355.

  21. Longstaff MG, Heath RA. The influence of motor system degradation on the control of handwriting movements: a dynamical systems analysis. Hum Mov Sci 2003;22(1):91–110.

    Article  CAS  PubMed  Google Scholar 

  22. López-de Ipiña K, Alonso JB, Solé-Casals J, Barroso N, Henriquez P, Faundez-Zanuy M, Travieso CM, Ecay-Torres M, Martínez-Lage P, Eguiraun H. On automatic diagnosis of Alzheimer’s disease based on spontaneous speech analysis and emotional temperature. Cogn Comput 2015;7(1):44–55.

    Article  Google Scholar 

  23. Luria G, Kahana A, Rosenblum S. Detection of deception via handwriting behaviors using a computerized tool: toward an evaluation of malingering. Cogn Comput 2014;6(4):849–55.

    Article  Google Scholar 

  24. Luria G, Rosenblum S. Comparing the handwriting behaviours of true and false writing with computerized handwriting measures. Appl Cogn Psychol 2010;24(8):1115–28.

    Article  Google Scholar 

  25. Luria G, Rosenblum S. A computerized multidimensional measurement of mental workload via handwriting analysis. Behav Res Methods 2012;44(2):575–86.

    Article  PubMed  Google Scholar 

  26. Malik ZK, Hussain A, Wu J. Novel biologically inspired approaches to extracting online information from temporal data. Cogn Comput 2014;6(3):595–607.

    Article  Google Scholar 

  27. McClelland JL. Is a machine realization of truly human-like intelligence achievable? Cogn Comput 2009;1(1):17–21.

    Article  Google Scholar 

  28. Mergl R, Tigges P, Schröter A, Möller H -J, Hegerl U. Digitized analysis of handwriting and drawing movements in healthy subjects: methods, results and perspectives. J Neurosci Methods 1999;90(2):157–69.

    Article  CAS  PubMed  Google Scholar 

  29. Meulenbroek RGJ, Vinter A, Mounoud P. Development of the start-rotation principle in circle production. Br J Dev Psychol 1993;11(3):307–20.

    Article  Google Scholar 

  30. Mohan V, Morasso P, Sandini G, Kasderidis S. Inference through embodied simulation in cognitive robots. Cogn Comput 2013;5(3):355–82.

    Article  Google Scholar 

  31. Pan SJ, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng 2010;22(10):1345–59.

    Article  Google Scholar 

  32. Paulhus DL, Graf P, Van Selst M. Attentional load increases the positivity of self-presentation. Soc Cogn 1989;7(4):389–400.

    Article  Google Scholar 

  33. Poon KW, Li-Tsang CWP, Weiss TPL, Rosenblum S. The effect of a computerized visual perception and visual-motor integration training program on improving chinese handwriting of children with handwriting difficulties. Res Dev Disabil 2010;31(6):1552–60.

    Article  CAS  PubMed  Google Scholar 

  34. Rosenblum S. Handwriting measures as reflectors of executive functions among adults with developmental coordination disorders (dcd). Front Psychol 2013;4(1):357–67.

    PubMed  PubMed Central  Google Scholar 

  35. Rosenblum S. Do motor ability and handwriting kinematic measures predict organizational ability among children with developmental coordination disorders? Hum Mov Sci 2015;43(1):201–15.

    Article  PubMed  Google Scholar 

  36. Rosenblum S, Epsztein L, Josman N. Handwriting performance of children with attention deficit hyperactive disorders: a pilot study. Physical & Occupational Therapy In Pediatrics 2008;28(3):219–34.

    Article  Google Scholar 

  37. Rosenblum S, Livneh-Zirinski M. Do relationships exist between brain-hand language and daily function characteristics of children with a hidden disability? Recent advances of neural network models and applications: proceedings of the 23rd workshop of the italian neural networks society (SIREN); 2014. p. 269–281.

  38. Rosenblum S, Luria G. Applying a handwriting measurement model for capturing cognitive load implications through complex figure drawing. Cogn Comput 2016;8(1):69–77.

    Article  Google Scholar 

  39. Rosenblum S, Parush S, Weiss PL. Computerized temporal handwriting characteristics of proficient and non-proficient handwriters. Am J Occup Ther 2003;57(2):129–38.

    Article  PubMed  Google Scholar 

  40. Rosenblum S, Simhon HAB, Gal E. Unique handwriting performance characteristics of children with high-functioning autism spectrum disorder. Research in Autism Spectrum Disorders 2016;23(1):235–44.

    Article  Google Scholar 

  41. Rosenblum S, Werner P. Assessing the handwriting process in healthy elderly persons using a computerized system. Aging Clin Exp Res 2006;18(5):433–39.

    Article  PubMed  Google Scholar 

  42. Rosenblum S, Werner P, Dekel T, Gurevitz I, Heinik J. Handwriting process variables among elderly people with mild major depressive disorder: a preliminary study. Aging Clin Exp Res 2010;22(2):141–7.

    Article  PubMed  Google Scholar 

  43. Sesa-Nogueras E, Faundez-Zanuy M, Mekyska J. An information analysis of in-air and on-surface trajectories in online handwriting. Cogn Comput 2012;4(2):195–205.

    Article  Google Scholar 

  44. Sesa-Nogueras E, Faundez-Zanuy M, Roure-Alcobé J. Gender classification by means of online uppercase handwriting: a text-dependent allographic approach. Cogn Comput 2016;8(1):15–29.

    Article  Google Scholar 

  45. Smekal Z, Mekyska J, Rektorova I, Faundez-Zanuy M. Analysis of neurological disorders based on digital processing of speech and handwritten text. International symposium on signals circuits and systems (ISSCS); 2013. p. 1–6.

  46. Squartini S, Esposito A. Co-worker: toward real-time and context-aware systems for human collaborative knowledge building. Cogn Comput 2012;4(2):157–71.

    Article  Google Scholar 

  47. Squartini S, Hu S, Liu Q. Advances on brain inspired computing. Cogn Comput 2013;5(2):161–3.

    Article  Google Scholar 

  48. Stone JV. Learning perceptually salient visual parameters using spatiotemporal smoothness constraints. Neural Comput 1996;8(7):1463–92.

    Article  CAS  PubMed  Google Scholar 

  49. Suzuki Y, Kazuo M, Jiro T. Stylus enhancement to enrich interaction with computers. Proceedings of the 12th international conference on human-computer interaction: interaction platforms and techniques, HCI’07; 2007. p. 133–142.

  50. Swann WB, Hixon JG, Stein-Seroussi A, Gilbert DT. The fleeting gleam of praise: cognitive processes underlying behavioral reactions to self-relevant feedback. J Pers Soc Psychol 1990;59(1):17.

    Article  PubMed  Google Scholar 

  51. Taylor BT, Bove V M Jr. Graspables: Grasp-recognition as a user interface. Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’09; 2009. p. 917–926.

  52. Tabatabaey-Mashadi N, Sudirman R, Guest RM, Khalid PI. Analyses of pupils’ polygonal shape drawing strategy with respect to handwriting performance. Pattern Anal Applic 2015;18(3):571–86.

    Article  Google Scholar 

  53. Taylor JG. Cognitive computation. Cogn Comput 2009;1(1):4–16.

    Article  Google Scholar 

  54. Teulings H -L, Contreras-Vidal JL, Stelmach GE, Adler CH. Parkinsonism reduces coordination of fingers, wrist, and arm in fine motor control. Exp Neurol 1997;146(1):159–70.

    Article  CAS  PubMed  Google Scholar 

  55. Travieso CM, Alonso JB. Special issue on advanced cognitive systems based on nonlinear analysis. Cogn Comput 2013;5(4):397–8.

    Article  Google Scholar 

  56. Werner P, Rosenblum S, Bar-On G, Heinik J, Korczyn A. Handwriting process variables discriminating mild Alzheimer’s disease and mild cognitive impairment. J Gerontol Ser B Psychol Sci Soc Sci 2006;61 (4):228–36.

    Article  Google Scholar 

  57. Xiaojun B, Tomer M, Gonzalo R, Ravin B, Ken H. An exploration of pen rolling for pen-based interaction. Proceedings of the 21st annual ACM symposium on user interface software and technology, UIST ’08; 2008. p. 191–200.

  58. Zhao Z, Chen Z, Chen Y, Wang S, Wang H. A class incremental extreme learning machine for activity recognition. Cogn Comput 2014;6(3):423–31.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Systems, Faculty of Social Sciences, University of Haifa, Haifa, Israel

    Murad Badarna & Ilan Shimshoni

  2. Department of Human Services, Faculty of Social Welfare and Health Sciences, University of Haifa, Haifa, Israel

    Gil Luria

  3. Laboratory of Complex Human Activity and Participation (CHAP), Department of Occupational Therapy, Faculty of Social Welfare and Health Sciences, University of Haifa, Haifa, Israel

    Sara Rosenblum

Authors
  1. Murad Badarna
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ilan Shimshoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gil Luria
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sara Rosenblum
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Murad Badarna.

Ethics declarations

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee.

Conflict of Interest

The authors declare that they have no conflict of interest.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Badarna, M., Shimshoni, I., Luria, G. et al. The Importance of Pen Motion Pattern Groups for Semi-Automatic Classification of Handwriting into Mental Workload Classes. Cogn Comput 10, 215–227 (2018). https://doi.org/10.1007/s12559-017-9520-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-017-9520-2

Keywords

Search

Navigation