Skip to main content
Log in

Online separation of handwriting from freehand drawing using extreme learning machines

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Online separation between handwriting and freehand drawing is still an active research area in the field of sketch-based interfaces. In the last years, most approaches in this area have been focused on the use of statistical separation methods, which have achieved significant results in terms of performance. More recently, Machine Learning (ML) techniques have proven to be even more effective by treating the separation problem like a classification task. Despite this, also in the use of these techniques several aspects can be still considered open problems, including: 1) the trade-off between separation performance and training time; 2) the separation of handwriting from different types of freehand drawings. To address the just reported drawbacks, in this paper a novel separation algorithm based on a set of original features and an Extreme Learning Machine (ELM) is proposed. Extensive experiments on a wide range of sketched schemes (i.e., text and graphical symbols), more numerous than those usually tested in any key work of the current literature, have highlighted the effectiveness of the proposed approach. Finally, measurements on accuracy and speed of computation, during both training and testing stages, have shown that the ELM can be considered, in this research area, the better choice even if compared with other popular ML techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Al Kabary I, Schuldt H (2014) Enhancing sketch-based sport video retrieval by suggesting relevant motion paths. In: Proceedings of the international ACM SIGIR conference on research & development in information retrieval (SIGIR), pp 1227–1230

  2. Alvarado C, Davis R (2004) Sketchread: A multi-domain sketch recognition engine. In: Proceedings of the annual ACM symposium on user interface software and technology (UIST), pp 23–32

  3. Avola D, Caschera MC, Grifoni P (2006) Solving ambiguities for sketch-based interaction in mobile environments. In: Proceedings of the international conference on the move to meaningful internet systems (OTM), pp 904–915

    Chapter  Google Scholar 

  4. Avola D, Ferri F, Grifoni P (2007a) Formalizing recognition of sketching styles in human centered systems. In: Proceedings of the international conference on knowledge-based intelligent information and engineering systems (KES), pp 369–376

    Google Scholar 

  5. Avola D, Ferri F, Grifoni P, Caschera MC (2007b) Ambiguities in sketch-based interfaces. In: Proceedings of the annual Hawaii international conference on system sciences (HICSS), pp 1–10

  6. Avola D, Del Buono A, Del Nostro P, Wang R (2009a) A novel online textual/graphical domain separation approach for sketch-based interfaces. In: Proceedings of the international conference on new directions in intelligent interactive multimedia systems and services (KES-IIMSS), pp 167–176

  7. Avola D, Del Buono A, Gianforme G, Paolozzi S, Wang R (2009b) Sketchml a representation language for novel sketch recognition approach. In: Proceedings of the international conference on pervasive technologies related to assistive environments (PETRA), pp 1–8

  8. Avola D, Cinque L, Placidi G (2013) Sketchspore: A sketch based domain separation and recognition system for interactive interfaces. In: Proceedings of the international conference on image analysis and processing (ICIAP), pp 181–190

    Chapter  Google Scholar 

  9. Avola D, Bernardi M, Cinque L, Foresti GL, Marini MR, Massaroni C (2017) A machine learning approach for the online separation of handwriting from freehand drawing. In: Proceedings of the international conference on image analysis and processing (ICIAP), pp 223–232

    Chapter  Google Scholar 

  10. Bhat A, Hammond T (2009) Using entropy to distinguish shape versus text in hand-drawn diagrams. In: Proceedings of the international jont conference on artifical intelligence (IJCAI), pp 1395–1400

  11. Bishop CM, Svensen M, Hinton GE (2004) Distinguishing text from graphics in on-line handwritten ink. In: Proceedings of the international workshop on frontiers in handwriting recognition (ICFHR), pp 142–147

  12. Blagojevic R, Plimmer B, Grundy J, Wang Y (2011) Using data mining for digital ink recognition: Dividing text and shapes in sketched diagrams. Comput Graph 35(5):976–991

    Article  Google Scholar 

  13. Boniardi F, Valada A, Burgard W, Tipaldi G (2016) Autonomous indoor robot navigation using a sketch interface for drawing maps and routes. In: Proceedings of the international conference on robotics and automation (ICRA), pp 2896–2901

  14. Bucurica M, Dogaru R, Dogaru I (2015) A comparison of extreme learning machine and support vector machine classifiers. In: Proceedings of the international conference on intelligent computer communication and processing (ICCP), pp 471–474

  15. Cao J, Zhang K, Luo M, Yin C, Lai X (2016) Extreme learning machine and adaptive sparse representation for image classification. Neural Netw 81:91–102

    Article  Google Scholar 

  16. Costagliola G, Rosa MD, Fuccella V (2014) Local context-based recognition of sketched diagrams. J Vis Lang Comput 25(6):955–962

    Article  Google Scholar 

  17. Dahake D, Sharma RK, Singh H (2017) On segmentation of words from online handwritten gurmukhi sentences. In: Proceedings of the international conference on man and machine interfacing (MAMI), pp 1–6

  18. Deufemia V, Risi M, Tortora G (2014) Sketched symbol recognition using latent-dynamic conditional random fields and distance-based clustering. Pattern Recogn 47(3):1159–1171

    Article  Google Scholar 

  19. Ding C, Liu L (2016) A survey of sketch based modeling systems. Front Comp Sci 10(6):985–999

    Article  Google Scholar 

  20. Eglin V, Bres S, Rivero C (2004) Multiscale handwriting characterization for writers’ classification. In: Proceedings of the international conference on document analysis systems (DAS), pp 337–341

    Chapter  Google Scholar 

  21. Eitrich T, Lang B (2005) Parallel tuning of support vector machine learning parameters for large and unbalanced data sets. In: Proceedings of the international conference on computational life sciences (ICCLS), pp 253–264

    Google Scholar 

  22. Hammond T, Logsdon D, Peschel J, Johnston J, Taele P, Wolin A, Paulson B (2010) A sketch recognition interface that recognizes hundreds of shapes in course-of-action diagrams. In: Extended abstracts on human factors in computing systems (CHI-EA), pp 4213–4218

  23. Hammond TA, Logsdon D, Paulson B, Johnston J, Peschel J, Wolin A, Taele P (2010) A sketch recognition system for recognizing free-hand course of action diagrams. In: Proceedings of the innovative applications of artificial intelligence conference (IAAI), pp 1–6

  24. Herold J, Stahovich TF (2014) A machine learning approach to automatic stroke segmentation. Comput Graph 38:357–364

    Article  Google Scholar 

  25. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: Theory and applications. Neurocomputing 70(1):489–501

    Article  Google Scholar 

  26. Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2(2):107–122

    Article  Google Scholar 

  27. Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42(2):513–529

    Article  Google Scholar 

  28. Huang Z, Yu Y, Gu J, Liu H (2017) An efficient method for traffic sign recognition based on extreme learning machine. IEEE Trans Cybern 47(4):920–933

    Article  Google Scholar 

  29. Hussain T, Siniscalchi SM, Lee CC, Wang SS, Tsao Y, Liao WH (2017) Experimental study on extreme learning machine applications for speech enhancement. IEEE Access 5:25542–25554

    Article  Google Scholar 

  30. Jahani-Fariman H, Kavakli M, Boyali A (2018) Matrack: block sparse bayesian learning for a sketch recognition approach. Multimed Tools Appl 77(2):1997–2012

    Article  Google Scholar 

  31. Keysers D, Deselaers T, Rowley HA, Wang L, Carbune V (2017) Multi-language online handwriting recognition. IEEE Trans Pattern Anal Mach Intell 39(6):1180–1194

    Article  Google Scholar 

  32. Kim HH, Taele P, Valentine S, McTigue E, Hammond T (2013) Kimchi: A sketch-based developmental skill classifier to enhance pen-driven educational interfaces for children. In: Proceedings of the international symposium on sketch-based interfaces and modeling (SBIM), pp 33–42

  33. Lan Y, Hu Z, Soh YC, Huang GB (2013) An extreme learning machine approach for speaker recognition. Neural Comput Appl 22(3-4):417–425

    Article  Google Scholar 

  34. Landay JA, Myers BA (2001) Sketching interfaces: toward more human interface design. Computer 34(3):56–64

    Article  Google Scholar 

  35. Liu X, Gao C, Li P (2012) A comparative analysis of support vector machines and extreme learning machines. Neural Netw 33:58–66

    Article  Google Scholar 

  36. Lu T, Guan Y, Zhang Y, Qu S, Xiong Z (2018) Robust and efficient face recognition via low-rank supported extreme learning machine. Multimed Tools Appl 77 (9):11219–11240

    Article  Google Scholar 

  37. Machii K, Fukushima H, Nakagawa M (1993) On-line text/drawings segmentation of handwritten patterns. In: Proceedings of the international conference on document analysis and recognition (DAR), pp 710–713

  38. Mahdiyah U, Irawan MI, Imah EM (2015) Integrating data selection and extreme learning machine for imbalanced data. Procedia Comput Sci 59:221–229

    Article  Google Scholar 

  39. Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Proceedings of the international conference on machine learning (ICML), pp 807–814

  40. Ouyang TY, Davis R (2007) Recognition of hand drawn chemical diagrams. In: Proceedings of the national conference on artificial intelligence (AAAI), pp 846–851

  41. Parvez MT, Mahmoud SA (2013) Arabic handwriting recognition using structural and syntactic pattern attributes. Pattern Recogn 46(1):141–154

    Article  Google Scholar 

  42. Patil U, Begum M (2012) Word level handwritten and printed text separation based on shape features. Int J Emerging Technol Adv Eng 2(4):590–594

    Google Scholar 

  43. Phan AV, Nguyen ML, Bui LT (2017) Feature weighting and svm parameters optimization based on genetic algorithms for classification problems. Appl Intell 46 (2):455–469

    Article  Google Scholar 

  44. Phang SK, Lai S, Wang F, Lan M, Chen BM (2015) Systems design and implementation with jerk-optimized trajectory generation for uav calligraphy. Mechatronics 30:65–75

    Article  Google Scholar 

  45. Qin S (2005) Intelligent classification of sketch strokes. In: Proceedings of the international conference on computer as a tool (EUROCON), pp 1374–1377

  46. Shi LC, Lu BL (2013) Eeg-based vigilance estimation using extreme learning machines. Neurocomputing 102:135–143

    Article  Google Scholar 

  47. Sun P, Chen Y, Lyu X, Wang B, Qu J, Tang Z (2018) A free-sketch recognition method for chemical structural formula. In: Proceedings of the international workshop on document analysis systems (DAS), pp 157–162

  48. Suresh S, Babu RV, Kim H (2009) No-reference image quality assessment using modified extreme learning machine classifier. Appl Soft Comput 9(2):541–552

    Article  Google Scholar 

  49. Tang J, Deng C, Huang GB, Zhao B (2015) Compressed-domain ship detection on spaceborne optical image using deep neural network and extreme learning machine. IEEE Trans Geosci Remote Sens 53(3):1174–1185

    Article  Google Scholar 

  50. Unser M, Aldroubi A, Eden M (1993) B-spline signal processing. i. theory. IEEE Trans Signal Process 41(2):821–833

    Article  Google Scholar 

  51. Verma K, Sharma RK (2017) Comparison of hmm- and svm-based stroke classifiers for gurmukhi script. Neural Comput Appl 28(1):51–63

    Article  Google Scholar 

  52. Wadhwa D, Verma K (2012) Online handwriting recognition of hindi numerals using svm. Int J Comput Appl 48(11):590–594

    Google Scholar 

  53. Yank E, Sezgin TM (2015) Active learning for sketch recognition. Comput Graph 52:93–105

    Article  Google Scholar 

  54. Zhang XY, Bengio Y, Liu CL (2017) Online and offline handwritten chinese character recognition: A comprehensive study and new benchmark. Pattern Recogn 61:348–360

    Article  Google Scholar 

  55. Zheng X, Miao Q, Shi Z, Fan Y, Shui W (2016) A new artistic information extraction method with multi channels and guided filters for calligraphy works. Multimed Tools Appl 75(14):8719–8744

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the MIUR under grant “Departments of Excellence 2018-2022” of the Department of Computer Science of Sapienza University.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Danilo Avola.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Avola, D., Bernardi, M., Cinque, L. et al. Online separation of handwriting from freehand drawing using extreme learning machines. Multimed Tools Appl 79, 4463–4481 (2020). https://doi.org/10.1007/s11042-019-7196-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7196-1

Keywords

Navigation