Abstract
Online separation between handwriting and freehand drawing is still an active research area in the field of sketch-based interfaces. In the last years, most approaches in this area have been focused on the use of statistical separation methods, which have achieved significant results in terms of performance. More recently, Machine Learning (ML) techniques have proven to be even more effective by treating the separation problem like a classification task. Despite this, also in the use of these techniques several aspects can be still considered open problems, including: 1) the trade-off between separation performance and training time; 2) the separation of handwriting from different types of freehand drawings. To address the just reported drawbacks, in this paper a novel separation algorithm based on a set of original features and an Extreme Learning Machine (ELM) is proposed. Extensive experiments on a wide range of sketched schemes (i.e., text and graphical symbols), more numerous than those usually tested in any key work of the current literature, have highlighted the effectiveness of the proposed approach. Finally, measurements on accuracy and speed of computation, during both training and testing stages, have shown that the ELM can be considered, in this research area, the better choice even if compared with other popular ML techniques.
Similar content being viewed by others
References
Al Kabary I, Schuldt H (2014) Enhancing sketch-based sport video retrieval by suggesting relevant motion paths. In: Proceedings of the international ACM SIGIR conference on research & development in information retrieval (SIGIR), pp 1227–1230
Alvarado C, Davis R (2004) Sketchread: A multi-domain sketch recognition engine. In: Proceedings of the annual ACM symposium on user interface software and technology (UIST), pp 23–32
Avola D, Caschera MC, Grifoni P (2006) Solving ambiguities for sketch-based interaction in mobile environments. In: Proceedings of the international conference on the move to meaningful internet systems (OTM), pp 904–915
Avola D, Ferri F, Grifoni P (2007a) Formalizing recognition of sketching styles in human centered systems. In: Proceedings of the international conference on knowledge-based intelligent information and engineering systems (KES), pp 369–376
Avola D, Ferri F, Grifoni P, Caschera MC (2007b) Ambiguities in sketch-based interfaces. In: Proceedings of the annual Hawaii international conference on system sciences (HICSS), pp 1–10
Avola D, Del Buono A, Del Nostro P, Wang R (2009a) A novel online textual/graphical domain separation approach for sketch-based interfaces. In: Proceedings of the international conference on new directions in intelligent interactive multimedia systems and services (KES-IIMSS), pp 167–176
Avola D, Del Buono A, Gianforme G, Paolozzi S, Wang R (2009b) Sketchml a representation language for novel sketch recognition approach. In: Proceedings of the international conference on pervasive technologies related to assistive environments (PETRA), pp 1–8
Avola D, Cinque L, Placidi G (2013) Sketchspore: A sketch based domain separation and recognition system for interactive interfaces. In: Proceedings of the international conference on image analysis and processing (ICIAP), pp 181–190
Avola D, Bernardi M, Cinque L, Foresti GL, Marini MR, Massaroni C (2017) A machine learning approach for the online separation of handwriting from freehand drawing. In: Proceedings of the international conference on image analysis and processing (ICIAP), pp 223–232
Bhat A, Hammond T (2009) Using entropy to distinguish shape versus text in hand-drawn diagrams. In: Proceedings of the international jont conference on artifical intelligence (IJCAI), pp 1395–1400
Bishop CM, Svensen M, Hinton GE (2004) Distinguishing text from graphics in on-line handwritten ink. In: Proceedings of the international workshop on frontiers in handwriting recognition (ICFHR), pp 142–147
Blagojevic R, Plimmer B, Grundy J, Wang Y (2011) Using data mining for digital ink recognition: Dividing text and shapes in sketched diagrams. Comput Graph 35(5):976–991
Boniardi F, Valada A, Burgard W, Tipaldi G (2016) Autonomous indoor robot navigation using a sketch interface for drawing maps and routes. In: Proceedings of the international conference on robotics and automation (ICRA), pp 2896–2901
Bucurica M, Dogaru R, Dogaru I (2015) A comparison of extreme learning machine and support vector machine classifiers. In: Proceedings of the international conference on intelligent computer communication and processing (ICCP), pp 471–474
Cao J, Zhang K, Luo M, Yin C, Lai X (2016) Extreme learning machine and adaptive sparse representation for image classification. Neural Netw 81:91–102
Costagliola G, Rosa MD, Fuccella V (2014) Local context-based recognition of sketched diagrams. J Vis Lang Comput 25(6):955–962
Dahake D, Sharma RK, Singh H (2017) On segmentation of words from online handwritten gurmukhi sentences. In: Proceedings of the international conference on man and machine interfacing (MAMI), pp 1–6
Deufemia V, Risi M, Tortora G (2014) Sketched symbol recognition using latent-dynamic conditional random fields and distance-based clustering. Pattern Recogn 47(3):1159–1171
Ding C, Liu L (2016) A survey of sketch based modeling systems. Front Comp Sci 10(6):985–999
Eglin V, Bres S, Rivero C (2004) Multiscale handwriting characterization for writers’ classification. In: Proceedings of the international conference on document analysis systems (DAS), pp 337–341
Eitrich T, Lang B (2005) Parallel tuning of support vector machine learning parameters for large and unbalanced data sets. In: Proceedings of the international conference on computational life sciences (ICCLS), pp 253–264
Hammond T, Logsdon D, Peschel J, Johnston J, Taele P, Wolin A, Paulson B (2010) A sketch recognition interface that recognizes hundreds of shapes in course-of-action diagrams. In: Extended abstracts on human factors in computing systems (CHI-EA), pp 4213–4218
Hammond TA, Logsdon D, Paulson B, Johnston J, Peschel J, Wolin A, Taele P (2010) A sketch recognition system for recognizing free-hand course of action diagrams. In: Proceedings of the innovative applications of artificial intelligence conference (IAAI), pp 1–6
Herold J, Stahovich TF (2014) A machine learning approach to automatic stroke segmentation. Comput Graph 38:357–364
Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: Theory and applications. Neurocomputing 70(1):489–501
Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2(2):107–122
Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42(2):513–529
Huang Z, Yu Y, Gu J, Liu H (2017) An efficient method for traffic sign recognition based on extreme learning machine. IEEE Trans Cybern 47(4):920–933
Hussain T, Siniscalchi SM, Lee CC, Wang SS, Tsao Y, Liao WH (2017) Experimental study on extreme learning machine applications for speech enhancement. IEEE Access 5:25542–25554
Jahani-Fariman H, Kavakli M, Boyali A (2018) Matrack: block sparse bayesian learning for a sketch recognition approach. Multimed Tools Appl 77(2):1997–2012
Keysers D, Deselaers T, Rowley HA, Wang L, Carbune V (2017) Multi-language online handwriting recognition. IEEE Trans Pattern Anal Mach Intell 39(6):1180–1194
Kim HH, Taele P, Valentine S, McTigue E, Hammond T (2013) Kimchi: A sketch-based developmental skill classifier to enhance pen-driven educational interfaces for children. In: Proceedings of the international symposium on sketch-based interfaces and modeling (SBIM), pp 33–42
Lan Y, Hu Z, Soh YC, Huang GB (2013) An extreme learning machine approach for speaker recognition. Neural Comput Appl 22(3-4):417–425
Landay JA, Myers BA (2001) Sketching interfaces: toward more human interface design. Computer 34(3):56–64
Liu X, Gao C, Li P (2012) A comparative analysis of support vector machines and extreme learning machines. Neural Netw 33:58–66
Lu T, Guan Y, Zhang Y, Qu S, Xiong Z (2018) Robust and efficient face recognition via low-rank supported extreme learning machine. Multimed Tools Appl 77 (9):11219–11240
Machii K, Fukushima H, Nakagawa M (1993) On-line text/drawings segmentation of handwritten patterns. In: Proceedings of the international conference on document analysis and recognition (DAR), pp 710–713
Mahdiyah U, Irawan MI, Imah EM (2015) Integrating data selection and extreme learning machine for imbalanced data. Procedia Comput Sci 59:221–229
Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Proceedings of the international conference on machine learning (ICML), pp 807–814
Ouyang TY, Davis R (2007) Recognition of hand drawn chemical diagrams. In: Proceedings of the national conference on artificial intelligence (AAAI), pp 846–851
Parvez MT, Mahmoud SA (2013) Arabic handwriting recognition using structural and syntactic pattern attributes. Pattern Recogn 46(1):141–154
Patil U, Begum M (2012) Word level handwritten and printed text separation based on shape features. Int J Emerging Technol Adv Eng 2(4):590–594
Phan AV, Nguyen ML, Bui LT (2017) Feature weighting and svm parameters optimization based on genetic algorithms for classification problems. Appl Intell 46 (2):455–469
Phang SK, Lai S, Wang F, Lan M, Chen BM (2015) Systems design and implementation with jerk-optimized trajectory generation for uav calligraphy. Mechatronics 30:65–75
Qin S (2005) Intelligent classification of sketch strokes. In: Proceedings of the international conference on computer as a tool (EUROCON), pp 1374–1377
Shi LC, Lu BL (2013) Eeg-based vigilance estimation using extreme learning machines. Neurocomputing 102:135–143
Sun P, Chen Y, Lyu X, Wang B, Qu J, Tang Z (2018) A free-sketch recognition method for chemical structural formula. In: Proceedings of the international workshop on document analysis systems (DAS), pp 157–162
Suresh S, Babu RV, Kim H (2009) No-reference image quality assessment using modified extreme learning machine classifier. Appl Soft Comput 9(2):541–552
Tang J, Deng C, Huang GB, Zhao B (2015) Compressed-domain ship detection on spaceborne optical image using deep neural network and extreme learning machine. IEEE Trans Geosci Remote Sens 53(3):1174–1185
Unser M, Aldroubi A, Eden M (1993) B-spline signal processing. i. theory. IEEE Trans Signal Process 41(2):821–833
Verma K, Sharma RK (2017) Comparison of hmm- and svm-based stroke classifiers for gurmukhi script. Neural Comput Appl 28(1):51–63
Wadhwa D, Verma K (2012) Online handwriting recognition of hindi numerals using svm. Int J Comput Appl 48(11):590–594
Yank E, Sezgin TM (2015) Active learning for sketch recognition. Comput Graph 52:93–105
Zhang XY, Bengio Y, Liu CL (2017) Online and offline handwritten chinese character recognition: A comprehensive study and new benchmark. Pattern Recogn 61:348–360
Zheng X, Miao Q, Shi Z, Fan Y, Shui W (2016) A new artistic information extraction method with multi channels and guided filters for calligraphy works. Multimed Tools Appl 75(14):8719–8744
Acknowledgements
This work was supported in part by the MIUR under grant “Departments of Excellence 2018-2022” of the Department of Computer Science of Sapienza University.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Avola, D., Bernardi, M., Cinque, L. et al. Online separation of handwriting from freehand drawing using extreme learning machines. Multimed Tools Appl 79, 4463–4481 (2020). https://doi.org/10.1007/s11042-019-7196-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-019-7196-1