Skip to main content
Log in

A music symbols recognition method using pattern matching along with integrated projection and morphological operation techniques

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Optical Music Recognition (OMR) can be divided into three main phases: (i) staff line detection and removal. The goal of this phase is to detect and to remove staff lines from sheet music images. (ii) music symbol detection and segmentation. The propose of this phase is to detect the remaining musical symbols such as single symbols and group symbols, then segment the group symbols to single or primitive symbols after removing staff lines. (iii) musical symbols recognition. In this phase, recognition of musical symbols is the main objective. The method presented in this paper, covers all three phases. One advantage of the first phase of the proposed method is that it is robust to staff lines rotation and staff lines which have curvature in sheet music images. Moreover, the staff lines are removed accurately and quickly and also fewer details of the musical symbols are omitted. The proposed method in the first phase focuses on the hand-written documents databases which have been introduced in the CVC-MUSCIMA and ICDAR 2013. It has the lowest error rate among well-known methods and outperforms the state of the art in CVC-MUSCIMA database. In ICDAR 2013, the specificity measure of this method is 99.71% which is the highest specificity among available methods. Also, in terms of accuracy, recall rate and f-measure is only slightly less than the best method. Therefor our method is comparable favorably to the existing methods. In the second phase, the symbols are divided into two categories, single and group. In the recognition phase, we use a pattern matching method to identify single symbols. For recognizing group symbols, a hierarchical method is proposed. The proposed method in the third phase has several advantages over the previous methods. It is quite robust to skewness of musical group symbols. Furthermore, it provides high accuracy in recognition of the symbols.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26

Similar content being viewed by others

Notes

  1. https://en.wikipedia.org/wiki/Solfege

  2. http://www.cvc.uab.es/cvcmuscima/index_database.html

  3. http://dag.cvc.uab.es/muscima/

  4. http://gamera.informatik.hsnr.de/addons/musicstaves/

References

  1. Bainbridge D, Bell T (1996) An extensible optical music recognition system. Aust Comput Sci Commun 18:308–317

    Google Scholar 

  2. Bellini P and Nesi P (2001) WEDELMUSIC format: An XML music notation format for emerging applications, In Web Delivering of Music, 2001. Proceedings. First International Conference on, pp. 79–86

  3. Blostein D and Baird HS (1992) A critical survey of music image analysis, In Structured Document Image Analysis, Springer, pp. 405–434

  4. Calvo-Zaragoza J and Oncina J (2014) Recognition of Pen-Based Music Notation: the HOMUS dataset, In Pattern Recognition (ICPR), 2014 22nd International Conference on, pp. 3038–3043

  5. Calvo-Zaragoza J et al (2015) Avoiding staff removal stage in optical music recognition: application to scores written in white mensural notation. Pattern Anal Applic 18(4):933–943

    Article  MathSciNet  Google Scholar 

  6. Calvo-Zaragoza J, Mico L, and Oncina J (2016) Music staff removal with supervised pixel classification International Journal on Document Analysis and Recognition (IJDAR), pages 1–9

  7. Calvo-Zaragoza J, Vigliensoni G, and Fujinaga I (2016) STAFF-LINE DETECTION ON GREYSCALE IMAGES WITH PIXEL CLASSIFICATION

  8. Carter NP (1989) Automatic recognition of printed music in the context of electronic publishing.” University of Surrey

  9. Coüasnon B and Camillerapp J, (1994) Using grammars to segment and recognize music scores, In International Association for Pattern Recognition Workshop on Document Analysis Systems, pp. 15–27

  10. Dalitz C, Droettboom M, Pranzas B, Fujinaga I (2008) A comparative study of staff removal algorithms. IEEE Trans Pattern Anal Mach Intell 30(5):753–766

    Article  Google Scholar 

  11. Dutta A, Pal U, Fornes A, and Llados J (2010) An efficient staff removal approach from printed musical documents, In Pattern Recognition (ICPR), 2010 20th International Conference on, pp. 1965–1968

  12. Fornes A, Dutta A, Gordo A, and Llados J (2011) The ICDAR 2011 music scores competition: Staff removal and writer identification. In Document Analysis and Recognition (ICDAR), 2011 International Conference on, pp. 1511–1515. IEEE

  13. Fornés A, Dutta A, Gordo A, Lladós J (2012) CVC-MUSCIMA: a ground truth of handwritten music score images for writer identification and staff removal. Int J Doc Anal Recognit 15(3):243–251

    Article  Google Scholar 

  14. Fornés, A, Kieu VC, Visani M, Journet N, and Dutta A (2013) The ICDAR/GREC 2013 music scores competition: Staff removal. In International Workshop on Graphics Recognition, pp. 207–220. Springer Berlin Heidelberg

  15. Fornés, A, Dutta A, Gordo A, and Lladós J (2013) The 2012 music scores competitions: staff removal and writer identification. In Graphics Recognition. New Trends and Challenges, pp. 173–186. Springer Berlin Heidelberg

  16. Fujinaga I (2004) Staff detection and removal, Vis. Percept. Music Not. on-line off-line Recognit., pp. 1–39

  17. Genfang C, Liyin Z, Wenjun Z, and Qiuqiu W (2010) Detecting the staff-lines of musical score with hough transform and mathematical morphology, In Multimedia Technology (ICMT), 2010 International Conference on, pp. 1–4

  18. Géraud, T (2014) A morphological method for music score staff removal." 2014 I.E. International Conference on Image Processing (ICIP). IEEE

  19. Gonzalez RC, Woods RE, and Eddins SL (2004) Digital image processing using MATLAB. Pearson Education India

  20. Hirata NST (2009) Multilevel training of binary morphological operators. IEEE Trans Pattern Anal Mach Intell 31(4):707–720

    Article  Google Scholar 

  21. Luangnapa N, Silpavarangkura T, Nukoolkit C, and Mongkolnam P (2012) Optical Music Recognition on Android Platform, In Advances in Information Technology, Springer, pp. 106–115

  22. Mahoney JV (1982) Automatic analysis of music score images. Massachusetts Institute of Technology

  23. Miyao H, Okamoto M (2004) Stave Extraction for Printed Music Scores Using DP Matching. JACIII 8(2):208–215

    Article  Google Scholar 

  24. Montagner, IS, Hirata R, and Hirata NST (2014) Learning to remove staff lines from music score images. In Image Processing (ICIP), 2014 I.E. International Conference on, pp. 2614–2618. IEEE

  25. Montagner IS, Hirata NST, Hirata R, and Canu S (2016) NILC: A two level learning algorithm with operator selection. In Image Processing (ICIP), 2016 I.E. International Conference on, pp. 1873–1877. IEEE

  26. Montagner IS, Canu S, Hirata NST, and Hirata R Jr, (2016) Kernel Approximations for W-operator learning,” in to appear in Proceedings of SIBGRAPI 2016 (XXIX Conference on Patterns, Graphics and Images)

  27. Ng K (2004) Optical music analysis for printed music score and handwritten music manuscript, Vis. Percept. Music Not. On-Line Off-Line Recognit., pp. 108–127

  28. Nhat VQ and Lee G (2014) Adaptive line fitting for staff detection in handwritten music score images,” In Proceedings of the 8th International Conference on Ubiquitous Information Management and Communication, p. 99

  29. Prerau DS (1970) Computer pattern recognition of standard engraved music notation,” Massachusetts Institute of Technology. Department of Electrical Engineering. 1970. Ph. D.

  30. Pruslin DH (1966) Automatic recognition of sheet music. Massachusetts Institute of Technology

  31. Pugin L, (2006) Optical Music Recognitoin of Early Typographic Prints using Hidden Markov Models. In ISMIR, pp. 53–56

  32. Ramirez C, Ohya J (2014) Automatic recognition of square notation symbols in western plainchant manuscripts. J New Music Res 43(4):390–399

    Article  Google Scholar 

  33. Randriamahefa R, Cocquerez JP, Fluhr C, Pepin F, and Philipp S (1993) Printed music recognition, In Document Analysis and Recognition, 1993., Proceedings of the Second International Conference on, pp. 898–901

  34. Rebelo A, Capela G, Cardoso JS (2010) Optical recognition of music symbols. Int J Doc Anal Recognit 13(1):19–31

    Article  Google Scholar 

  35. Rebelo A, Paszkiewicz F, Guedes C, Marcal ARS, and Cardoso J (2011) A method for music symbols extraction based on musical rules, In Proceedings of Bridges 2011: Mathematics, Music, Art, Architecture, Culture, pp. 81–88

  36. Rossant F, Bloch I (2007) Robust and adaptive OMR system including fuzzy modeling, fusion of musical rules, and possible error detection. EURASIP J Appl Signal Process 2007(1):160

    MATH  Google Scholar 

  37. dos Santos Cardoso J, Capela A, Rebelo A, Guedes C, Pinto da Costa J (2009) Staff detection with stable paths. IEEE Trans Pattern Anal Mach Intell 31(6):1134–1139

    Article  Google Scholar 

  38. dos Santos Montagner I, Hirata R, and Hirata NST (2014) A machine learning based method for staff removal." Pattern Recognition (ICPR), 2014 22nd International Conference on. IEEE

  39. Soontornwutikul T, Thananart N, Wantanareeyachart A, Nukoolkit C, and Arpnikanondt C (2013) Optical Music Recognition on Windows Phone 7, In The 9th International Conference on Computing and InformationTechnology (IC2IT2013), pp. 239–248

  40. Sotoodeh M and Tajeripour F (2012) Staff detection and removal using derivation and connected component analysis, In Artificial Intelligence and Signal Processing (AISP), 2012 16th CSI International Symposium on, pp. 54–57

  41. Su B, Lu S, Pal U, and Tan CL, (2012) An effective staff detection and removal technique for musical documents, In Document Analysis Systems (DAS), 2012 10th IAPR International Workshop on, pp. 160–164

  42. Szwoch M (2005) A robust detector for distorted music staves, In computer analysis of images and patterns, pp. 701–708

  43. Tajeripour F, Sotoodeh M (2012) A novel staff removal method for printed music image. IEICE Electron Express 9(7):609–615

    Article  Google Scholar 

  44. Timofte, R, and Van Gool L (2012) Automatic stave discovery for musical facsimiles." In Asian Conference on Computer Vision, pp. 510–523. Springer Berlin Heidelberg

  45. Toyama F, Shoji K, and Miyamichi J (2006) Symbol recognition of printed piano scores with touching symbols, In Pattern Recognition, 2006. ICPR 2006. 18th International Conference on, vol. 2, pp. 480–483

  46. Tsandilas T, (2012) Interpreting strokes on paper with a mobile assistant, In Proceedings of the 25th annual ACM symposium on User interface software and technology, pp. 299–308

  47. Visaniy, M, Kieu VC, Fornés A, and Journet N (2013) Icdar 2013 music scores competition: Staff removal." In Document Analysis and Recognition (ICDAR), 2013 12th International Conference on, pp. 1407–1411. IEEE

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mahmood Sotoodeh.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sotoodeh, M., Tajeripour, F., Teimori, S. et al. A music symbols recognition method using pattern matching along with integrated projection and morphological operation techniques. Multimed Tools Appl 77, 16833–16866 (2018). https://doi.org/10.1007/s11042-017-5256-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5256-y

Keywords

Navigation