Abstract
Optical Music Recognition (OMR) can be divided into three main phases: (i) staff line detection and removal. The goal of this phase is to detect and to remove staff lines from sheet music images. (ii) music symbol detection and segmentation. The propose of this phase is to detect the remaining musical symbols such as single symbols and group symbols, then segment the group symbols to single or primitive symbols after removing staff lines. (iii) musical symbols recognition. In this phase, recognition of musical symbols is the main objective. The method presented in this paper, covers all three phases. One advantage of the first phase of the proposed method is that it is robust to staff lines rotation and staff lines which have curvature in sheet music images. Moreover, the staff lines are removed accurately and quickly and also fewer details of the musical symbols are omitted. The proposed method in the first phase focuses on the hand-written documents databases which have been introduced in the CVC-MUSCIMA and ICDAR 2013. It has the lowest error rate among well-known methods and outperforms the state of the art in CVC-MUSCIMA database. In ICDAR 2013, the specificity measure of this method is 99.71% which is the highest specificity among available methods. Also, in terms of accuracy, recall rate and f-measure is only slightly less than the best method. Therefor our method is comparable favorably to the existing methods. In the second phase, the symbols are divided into two categories, single and group. In the recognition phase, we use a pattern matching method to identify single symbols. For recognizing group symbols, a hierarchical method is proposed. The proposed method in the third phase has several advantages over the previous methods. It is quite robust to skewness of musical group symbols. Furthermore, it provides high accuracy in recognition of the symbols.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig1_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig2_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig3_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig4_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig5_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig6_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig7_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig8_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig9_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig10_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig11_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig12_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig13_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig14_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig15_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig16_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig17_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig18_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig19_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig20_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig21_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig22_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig23_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig24_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig25_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-017-5256-y/MediaObjects/11042_2017_5256_Fig26_HTML.gif)
Similar content being viewed by others
References
Bainbridge D, Bell T (1996) An extensible optical music recognition system. Aust Comput Sci Commun 18:308–317
Bellini P and Nesi P (2001) WEDELMUSIC format: An XML music notation format for emerging applications, In Web Delivering of Music, 2001. Proceedings. First International Conference on, pp. 79–86
Blostein D and Baird HS (1992) A critical survey of music image analysis, In Structured Document Image Analysis, Springer, pp. 405–434
Calvo-Zaragoza J and Oncina J (2014) Recognition of Pen-Based Music Notation: the HOMUS dataset, In Pattern Recognition (ICPR), 2014 22nd International Conference on, pp. 3038–3043
Calvo-Zaragoza J et al (2015) Avoiding staff removal stage in optical music recognition: application to scores written in white mensural notation. Pattern Anal Applic 18(4):933–943
Calvo-Zaragoza J, Mico L, and Oncina J (2016) Music staff removal with supervised pixel classification International Journal on Document Analysis and Recognition (IJDAR), pages 1–9
Calvo-Zaragoza J, Vigliensoni G, and Fujinaga I (2016) STAFF-LINE DETECTION ON GREYSCALE IMAGES WITH PIXEL CLASSIFICATION
Carter NP (1989) Automatic recognition of printed music in the context of electronic publishing.” University of Surrey
Coüasnon B and Camillerapp J, (1994) Using grammars to segment and recognize music scores, In International Association for Pattern Recognition Workshop on Document Analysis Systems, pp. 15–27
Dalitz C, Droettboom M, Pranzas B, Fujinaga I (2008) A comparative study of staff removal algorithms. IEEE Trans Pattern Anal Mach Intell 30(5):753–766
Dutta A, Pal U, Fornes A, and Llados J (2010) An efficient staff removal approach from printed musical documents, In Pattern Recognition (ICPR), 2010 20th International Conference on, pp. 1965–1968
Fornes A, Dutta A, Gordo A, and Llados J (2011) The ICDAR 2011 music scores competition: Staff removal and writer identification. In Document Analysis and Recognition (ICDAR), 2011 International Conference on, pp. 1511–1515. IEEE
Fornés A, Dutta A, Gordo A, Lladós J (2012) CVC-MUSCIMA: a ground truth of handwritten music score images for writer identification and staff removal. Int J Doc Anal Recognit 15(3):243–251
Fornés, A, Kieu VC, Visani M, Journet N, and Dutta A (2013) The ICDAR/GREC 2013 music scores competition: Staff removal. In International Workshop on Graphics Recognition, pp. 207–220. Springer Berlin Heidelberg
Fornés, A, Dutta A, Gordo A, and Lladós J (2013) The 2012 music scores competitions: staff removal and writer identification. In Graphics Recognition. New Trends and Challenges, pp. 173–186. Springer Berlin Heidelberg
Fujinaga I (2004) Staff detection and removal, Vis. Percept. Music Not. on-line off-line Recognit., pp. 1–39
Genfang C, Liyin Z, Wenjun Z, and Qiuqiu W (2010) Detecting the staff-lines of musical score with hough transform and mathematical morphology, In Multimedia Technology (ICMT), 2010 International Conference on, pp. 1–4
Géraud, T (2014) A morphological method for music score staff removal." 2014 I.E. International Conference on Image Processing (ICIP). IEEE
Gonzalez RC, Woods RE, and Eddins SL (2004) Digital image processing using MATLAB. Pearson Education India
Hirata NST (2009) Multilevel training of binary morphological operators. IEEE Trans Pattern Anal Mach Intell 31(4):707–720
Luangnapa N, Silpavarangkura T, Nukoolkit C, and Mongkolnam P (2012) Optical Music Recognition on Android Platform, In Advances in Information Technology, Springer, pp. 106–115
Mahoney JV (1982) Automatic analysis of music score images. Massachusetts Institute of Technology
Miyao H, Okamoto M (2004) Stave Extraction for Printed Music Scores Using DP Matching. JACIII 8(2):208–215
Montagner, IS, Hirata R, and Hirata NST (2014) Learning to remove staff lines from music score images. In Image Processing (ICIP), 2014 I.E. International Conference on, pp. 2614–2618. IEEE
Montagner IS, Hirata NST, Hirata R, and Canu S (2016) NILC: A two level learning algorithm with operator selection. In Image Processing (ICIP), 2016 I.E. International Conference on, pp. 1873–1877. IEEE
Montagner IS, Canu S, Hirata NST, and Hirata R Jr, (2016) Kernel Approximations for W-operator learning,” in to appear in Proceedings of SIBGRAPI 2016 (XXIX Conference on Patterns, Graphics and Images)
Ng K (2004) Optical music analysis for printed music score and handwritten music manuscript, Vis. Percept. Music Not. On-Line Off-Line Recognit., pp. 108–127
Nhat VQ and Lee G (2014) Adaptive line fitting for staff detection in handwritten music score images,” In Proceedings of the 8th International Conference on Ubiquitous Information Management and Communication, p. 99
Prerau DS (1970) Computer pattern recognition of standard engraved music notation,” Massachusetts Institute of Technology. Department of Electrical Engineering. 1970. Ph. D.
Pruslin DH (1966) Automatic recognition of sheet music. Massachusetts Institute of Technology
Pugin L, (2006) Optical Music Recognitoin of Early Typographic Prints using Hidden Markov Models. In ISMIR, pp. 53–56
Ramirez C, Ohya J (2014) Automatic recognition of square notation symbols in western plainchant manuscripts. J New Music Res 43(4):390–399
Randriamahefa R, Cocquerez JP, Fluhr C, Pepin F, and Philipp S (1993) Printed music recognition, In Document Analysis and Recognition, 1993., Proceedings of the Second International Conference on, pp. 898–901
Rebelo A, Capela G, Cardoso JS (2010) Optical recognition of music symbols. Int J Doc Anal Recognit 13(1):19–31
Rebelo A, Paszkiewicz F, Guedes C, Marcal ARS, and Cardoso J (2011) A method for music symbols extraction based on musical rules, In Proceedings of Bridges 2011: Mathematics, Music, Art, Architecture, Culture, pp. 81–88
Rossant F, Bloch I (2007) Robust and adaptive OMR system including fuzzy modeling, fusion of musical rules, and possible error detection. EURASIP J Appl Signal Process 2007(1):160
dos Santos Cardoso J, Capela A, Rebelo A, Guedes C, Pinto da Costa J (2009) Staff detection with stable paths. IEEE Trans Pattern Anal Mach Intell 31(6):1134–1139
dos Santos Montagner I, Hirata R, and Hirata NST (2014) A machine learning based method for staff removal." Pattern Recognition (ICPR), 2014 22nd International Conference on. IEEE
Soontornwutikul T, Thananart N, Wantanareeyachart A, Nukoolkit C, and Arpnikanondt C (2013) Optical Music Recognition on Windows Phone 7, In The 9th International Conference on Computing and InformationTechnology (IC2IT2013), pp. 239–248
Sotoodeh M and Tajeripour F (2012) Staff detection and removal using derivation and connected component analysis, In Artificial Intelligence and Signal Processing (AISP), 2012 16th CSI International Symposium on, pp. 54–57
Su B, Lu S, Pal U, and Tan CL, (2012) An effective staff detection and removal technique for musical documents, In Document Analysis Systems (DAS), 2012 10th IAPR International Workshop on, pp. 160–164
Szwoch M (2005) A robust detector for distorted music staves, In computer analysis of images and patterns, pp. 701–708
Tajeripour F, Sotoodeh M (2012) A novel staff removal method for printed music image. IEICE Electron Express 9(7):609–615
Timofte, R, and Van Gool L (2012) Automatic stave discovery for musical facsimiles." In Asian Conference on Computer Vision, pp. 510–523. Springer Berlin Heidelberg
Toyama F, Shoji K, and Miyamichi J (2006) Symbol recognition of printed piano scores with touching symbols, In Pattern Recognition, 2006. ICPR 2006. 18th International Conference on, vol. 2, pp. 480–483
Tsandilas T, (2012) Interpreting strokes on paper with a mobile assistant, In Proceedings of the 25th annual ACM symposium on User interface software and technology, pp. 299–308
Visaniy, M, Kieu VC, Fornés A, and Journet N (2013) Icdar 2013 music scores competition: Staff removal." In Document Analysis and Recognition (ICDAR), 2013 12th International Conference on, pp. 1407–1411. IEEE
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sotoodeh, M., Tajeripour, F., Teimori, S. et al. A music symbols recognition method using pattern matching along with integrated projection and morphological operation techniques. Multimed Tools Appl 77, 16833–16866 (2018). https://doi.org/10.1007/s11042-017-5256-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-017-5256-y