Skip to main content

Advertisement

SpringerLink
Log in

MATRACK: block sparse Bayesian learning for a sketch recognition approach

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Human-computer interaction has become increasingly easy and popular using widespread smart devices. Gestures and sketches as the trajectory of hands in 3D space are among the popular interaction media. Therefore, their recognition is essential. However, diversity of human gestures along with the lack of visual cues make the sketch recognition process challenging. This paper aims to develop an accurate sketch recognition algorithm using Block Sparse Bayesian Learning (BSBL). Sketches are acquired from three datasets using a Wii-mote in a virtual-reality environment. We evaluate the performance of the proposed sketch recognition approach (MATRACK) on diverse sketch datasets. Comparisons are drawn with three high accuracy classifiers namely, Hidden Markov Model (HMM), Principle Component Analysis (PCA) and K-Nearest Neighbour (K-NN). MATRACK, the developed BSBL based sketch recognition system, outperforms k-NN, HMM and PCA. Specifically, for the most diverse dataset, MATRACK reaches the accuracy of 93.5%, where other three classifiers approximately catches 80% accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Akl A, Valaee S (2010) Accelerometer-based gesture recognition via dynamic-time warping, affinity propagation, & compressive sensing. in Acoustics Speech and Signal Processing (ICASSP), 2010 I.E. International Conference on, p 2270–2273

  2. Álvaro F, Sánchez J-A, Benedí J-M (2014) Recognition of on-line handwritten mathematical expressions using 2d stochastic context-free grammars and hidden markov models. Pattern Recognition Letters 35:58–67

  3. Andreopoulos A, Tsotsos JK (2013) 50 Years of object recognition: directions forward. Comput Vis Image Underst 117:827–891

  4. Bae S-H, Balakrishnan R, Singh K (2008) ILoveSketch: as-natural-as-possible sketching system for creating 3d curve models. in Proceedings of the 21st annual ACM symposium on User interface software and technology, p 151–160

  5. Baum LE, Petrie T (1966) Statistical inference for probabilistic functions of finite state Markov chains. Ann Math Stat 37:1554–1563

  6. Billinghurst M, Piumsomboon T, Bai H (2014) Hands in space: gesture interaction with augmented-reality interfaces. IEEE Comput Graph Appl 34:77–80

  7. Boyali A and Hashimoto N (2014) "Block-Sparse Representation Classification based gesture recognition approach for a robotic wheelchair," in Intelligent Vehicles Symposium Proceedings, 2014 IEEE, pp. 1133–1138.

  8. Boyali A, Kavakli M (2012) A robust gesture recognition algorithm based on sparse representation, random projections and compressed sensing. in Industrial Electronics and Applications (ICIEA), 2012 7th IEEE Conference on, p 243–249

  9. Braga-Neto UM, Dougherty ER (2004) Is cross-validation valid for small-sample microarray classification?, Bioinformatics 20:374–380

  10. Candè EJ, Wakin MB (2008) "an introduction to compressive sampling," Signal Processing Magazine. IEEE 25:21–30

  11. Candes EJ, Tao T (2006) "near-optimal signal recovery from random projections: universal encoding strategies?," Information Theory. IEEE Transactions 52:5406–5425

  12. Candès EJ, Romberg J, Tao T (2006) "robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information," Information Theory. IEEE Transactions 52:489–509

  13. Company P, Contero M, Varley P, Aleixos N, Naya F (2009) Computer-aided sketching as a tool to promote innovation in the new product development process. Comput Ind 60:592–603

  14. Cunningham P, Delany SJ (2007) k-Nearest neighbour classifiers. Multiple Classifier Systems, pp. 1–17

  15. Dix A (2009) Human-computer interaction. Springer, Berlin

  16. Donoho DL (2006) "compressed sensing," Information Theory. IEEE Trans 52:1289–1306

    MathSciNet  MATH  Google Scholar 

  17. Donoho DL, Elad M (2003) Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization. Proc Natl Acad Sci 100:2197–2202

    Article  MathSciNet  MATH  Google Scholar 

  18. Eitz M, Hays J (2011) Learning to classify human object sketches. in ACM SIGGRAPH 2011 Talks, p 30

  19. Eitz M, Hildebrand K, Boubekeur T, Alexa M (2011) "sketch-based image retrieval: benchmark and bag-of-features descriptors," Visualization and Computer Graphics. IEEE Trans 17:1624–1636

    Google Scholar 

  20. Eitz M, Hays J, Alexa M (2012) How do humans sketch objects? ACM Trans Graph 31:44

    Google Scholar 

  21. Fariman HJ, Ahmad SA, Marhaban MH, Ghasab MA, Chappell PH (2015) Hand movements classification for myoelectric control system using adaptive resonance theory. Australas Phys Eng Sci Med 39:1–18

  22. Freina L, Ott M (2015) A literature review on immersive virtual reality in education: state of the art and perspectives. in The International Scientific Conference eLearning and Software for Education, p. 133

  23. Ghorbel A, Anquetil E, Camillerapp J, Lemaitre A (2014) IMISketch: an interactive method for sketch recognition. Pattern Recogn Lett 35:78–90

  24. Ghorbel A, Lemaitre A, Anquetil E, Fleury S, Jamet E (2015) Interactive interpretation of structured documents: application to the recognition of handwritten architectural plans. Pattern Recogn 48:2446–2458

  25. Hammond TA, Logsdon D, Paulson B, Johnston J, Peschel JM, Wolin A, et al. (2010) A sketch recognition system for recognizing free-hand course of action diagrams, in IAAI

  26. Hu R, Collomosse J (2013) A performance evaluation of gradient field hog descriptor for sketch based image retrieval. Comput Vis Image Underst 117:790–806

  27. Johnson G, Gross MD, Hong J, Yi-Luen Do E (2009) Computational support for sketching in design: a review. Found Trends on Hum-Comput Interact 2:1–93

  28. Karam M (2006) Investigating user tolerance for errors in vision-enabled gesture-based interactions. in Proceedings of the working conference on Advanced visual interfaces, p 225–232

  29. Kavakli M (2008) Gesture recognition in virtual reality. International Journal of Arts and Technology 1:215–229.

  30. Kavakli M, Boyali A (2012) De-SIGN: robust gesture recognition in conceptual design, sensor analysis and synthesis. in Advances in Robotics and Virtual Reality, ed: Springer, p 201–225

  31. Kavakli M, Jayarathna D (2005) Virtual hand: an interface for interactive sketching in virtual reality. in Computational intelligence for modelling, control and automation, 2005 and international conference on intelligent agents, Web Technologies and Internet Commerce, International Conference on, p 613–618

  32. Kim KI, Jung K, Kim HJ (2002) "face recognition using kernel principal component analysis." Signal Processing Letters. IEEE 9:40–42.

    Google Scholar 

  33. Kim KS, Choi HH, Moon CS, Mun CW (2011) Comparison of k-nearest neighbor, quadratic discriminant and linear discriminant analysis in classification of electromyogram signals based on the wrist-motion directions. Curr Appl Phys 11:740–745

    Article  Google Scholar 

  34. Landay J, Myers B (2001) Sketching interfaces: toward more human interface design. IEEE Journal of Computer 34:56–64

    Article  Google Scholar 

  35. LaViola JJ Jr (2015) Context aware 3D gesture recognition for games and virtual reality. in ACM SIGGRAPH 2015 Courses, p 10

  36. Li Y, Hospedales TM, Song Y-Z, Gong S (2015) Free-hand sketch recognition by multi-kernel feature learning. Comput Vis Image Underst 137:1–11

    Article  Google Scholar 

  37. Liu N (2003) Lovell BC. Gesture classification using hidden markov models and viterbi path counting. in VIIth Digital Image Computing, Techniques and Applications

    Google Scholar 

  38. Mäntyjärvi J, Kallio S, Korpipää P, Kela J, Plomp J (2005) Gesture interaction for small handheld devices to support multimedia applications. Journal of Mobile Multimedia 1:92–111

    Google Scholar 

  39. Mitra S, Acharya T (2007) "gesture recognition: a survey," Systems, Man, and Cybernetics, Part C: Applications and Reviews. IEEE Transactions 37:311–324

    Google Scholar 

  40. Pantic M, Nijholt A, Pentland A, Huanag TS (2008) Human-Centred intelligent human? Computer interaction (HCI2): how far are we from attaining it? International Journal of Autonomous and Adaptive Communication Systems 1:168–187

    Article  Google Scholar 

  41. Rahman MH, Afrin J (2013) Hand gesture recognition using multiclass support vector machine. Int J Comput Appl 74:39

    Google Scholar 

  42. Ramani K (2015) A gesture-free geometric approach for mid-air expression of design intent in 3D virtual pottery. Comput Aided Des 69:11–24

    Article  Google Scholar 

  43. Rautaray SS, Agrawal A (2015) Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Rev 43:1–54

    Article  Google Scholar 

  44. Rehm M, Bee N, André E (2008) Wave like an Egyptian: accelerometer based gesture recognition for culture specific interactions. in Proceedings of the 22nd British HCI Group Annual Conference on People and Computers: Culture, Creativity, Interaction-Volume 1, p 13–22

  45. Seddati O, Dupont S, Mahmoudi S (2015) Deepsketch: deep convolutional neural networks for sketch recognition and similarity search. in Content-Based Multimedia Indexing (CBMI), 2015 13th International Workshop on, p 1–6

  46. Su M-C, Hsio T-H, Hsieh Y-Z, Lin S-C, Chou C-H (2012) A neural-network-based sketch recognition system. in Intelligent Signal Processing and Communications Systems (ISPACS), 2012 International Symposium on, p 420–423

  47. Tang X, Wang X (2004) "face sketch recognition," Circuits and Systems for Video Technology. IEEE Transactions 14:50–57

    Google Scholar 

  48. Tseng K-Y, Lin Y-L, Chen Y-H, Hsu WH (2012) Sketch-based image retrieval on mobile devices using compact hash bits. in Proceedings of the 20th ACM international conference on Multimedia, p 913–916

  49. Wiethoff A, Schneider H, Rohs M, Butz A, Greenberg S (2012) Sketch-a-TUI: low cost prototyping of tangible interactions using cardboard and conductive ink. in Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction, p 309–312

  50. Wilson AD, Bobick AF (2001) Hidden Markov models for modeling and recognizing gesture under variation. Int J Pattern Recognit Artif Intell 15:123–160

    Article  Google Scholar 

  51. Wobbrock JO, Wilson AD, Li Y (2007) Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. in Proceedings of the 20th annual ACM symposium on User interface software and technology, p 159–168

  52. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) "robust face recognition via sparse representation," Pattern Analysis and Machine Intelligence. IEEE Transactions 31:210–227

    Google Scholar 

  53. Yang K, Li Z, Ye J (2012) Freely-drawn sketches interpretation using SVMs-chain modeling. Engineering Applications of Artificial Intelligence 25:392–403

  54. Zhang Z, Rao BD (2012) Recovery of block sparse signals using the framework of block sparse Bayesian learning. in Acoustics, Speech and Signal Processing (ICASSP), 2012 I.E. International Conference on, p 3345–3348

  55. Zhang Z, Rao BD (2013) Extension of SBL algorithms for the recovery of block sparse signals with intra-block correlation. IEEE Trans Signal Process 61:2009–2015

    Article  Google Scholar 

Download references

Acknowledgements

This paper has been funded by iMQRES scholarship from Macquarie University, Sydney, NSW, Australia.

Author information

Authors and Affiliations

  1. VISOR Research Group, Department of Computing, Faculty of Science & Engineering, Macquarie University, Room 349, Building E6A, Sydney, NSW, 2109, Australia

    Hessam Jahani-Fariman & Manolya Kavakli

  2. National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Ibaraki, 305-8561, Japan

    Ali Boyali

Authors
  1. Hessam Jahani-Fariman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manolya Kavakli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ali Boyali
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hessam Jahani-Fariman.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jahani-Fariman, H., Kavakli, M. & Boyali, A. MATRACK: block sparse Bayesian learning for a sketch recognition approach. Multimed Tools Appl 77, 1997–2012 (2018). https://doi.org/10.1007/s11042-017-4368-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4368-8

Keywords

Search

Navigation