Abstract
Most of optical music recognition (OMR) systems work under the assumption that the input image is scanner-based. However, we propose in this paper, camera based OMR system. Camera based OMR has a challengeable work in un-controlled environment such as a light, perspective, curved, transparency distortions and uneven staff-lines which tend to incur more frequently. In addition, the loss in performance of binarization methods, line thickness variation and space variation between lines are inevitable. In order to solve these problems, we propose a novel and effective staff-line removal method based on following three main ideas. First, a state-of-the-art staff-line detection method, Stable Path, is used to extract staff-line skeletons of the music score. Second, a line adjacency graph (LAG) model is exploited in a different manner over segmentation to cluster pixel runs generated from the run-length encoding (RLE) of an music score image. Third, a two-pass staff-line removal pipeline called filament filtering is applied to remove clusters lying on the staff-line. A music symbol is comprised of several parts so-called primitives, but the combination of these parts to form music symbol is unlimited. It causes difficulty applying the state-of-the-art method for music symbol recognition. To overcome these challenges and deal with primitive parts separately, we proposed a combination model which consists of LAG model, Graph model, and Set model as a framework for music symbol recognition. Our method shows impressive results on music score images captured from cameras, and gives high performance when applied to the ICDAR/GREC 2013 database, and a Gamera synthetic database. We have compared to some commercial software and proved the expediency and efficiency of the proposed method.
Similar content being viewed by others
References
Bainbridge D, Bell T (2001) The challenge of optical music recognition. Comput Hum 35(2):95–121
Bellini P, Bruno I, Nesi P (2007) Assessing optical music recognition tools. Comput Music J 31:68–93
Bellini P, Ivan B, Paolo N (2008) Optical music recognition: Architecture and algorithms. In: Kia N, Paolo N (eds) Interactive Multimedia Music Technologies, IGI Global, 5, 80–110
Bui HN, Na IS, Lee GS, Yang HJ, Kim SH (2014a) Boosted stable path for staff-line detection using order statistic downscaling and coarse-to-fine technique. Proc. 22nd International Conference on Pattern Recognition, Stockholm, Sweden, pp 522–526
Bui HN, Na IS, Kim SH (2014b) Staff line removal using line adjacency graph and staff line skeleton for camera based printed music scores. In: Proceedings 22nd International Conference on Pattern Recognition, Stockholm, Sweden, pp 2787–2789
Cardoso JF, Capela A, Rebelo A, Guedes C, Pinto da Costa JF (2009) Staff detection with stable paths. IEEE Trans Pattern Anal Mach Intell 31(6):1134–1139
Carter NP, Bacon RA (1992) Automatic recognition of printed music. In: Baird H, Bunke, Yamamoto K (eds) Structured document image analysis, Springer Berlin, pp 456–465. [Online]. Available: doi:10.1007/978-3-642-77281-8_21
Dalitz C The Synthetic Score Database. [Online]. Available: http://gamera.informatik.hsnr.de/addons/musicstaves/
Dalitz C, Droettboom M, Pranzas B, Fujinaga I (2008) A comparative study of staff removal algorithms. IEEE Trans Pattern Anal Mach Intell 30(5):753–766
Fujinaga I (2004) Staff detection and removal. Visual perception of music notation: on-line and off-line recognition, 1–39
Gamera Project, [Online]. Available: http://gamera.informatik.hsnr.de/addons/ocr4gamera/
Homenda W (1995) Optical pattern recognition for printed music notation. 2490, 230–239. [Online]. Available: doi:10.1117/12.205779
Homenda W (2005) Optical music recognition: the case studyof pattern recognition. In: Kurzyski M, Puchaa E, Woniak M, Onierek A (eds) Computer recognition Systems,ser. Advances in soft computing, vol 30. Springer, Berlin, pp. 835–842
ICDAR/GREC 2013 Database. [Online]. Available: http://dag.cvc.uab.es/muscima/?pageid=6
Kato H, Inokuchi S (1992) A recognition system for printed piano music using musical knowledge and constraints. In: Baird H, Bunke H, Yamamoto K (eds) Structured document image analysis. Springer, Heidelberg, pp. 435–455
Kim SH, Son HJ, Oh SR, Lee CW, Oh IS (2007) Staff-line detection and removal algorithm for a camera based recognition of music score images. In: Pattern Recognition, 2007, 2nd Korea-Japan Joint Workshop on, pp 131–138
Modayur BR, Ramesh V, Haralick RM, Shapiro LG (1993) Muser: a prototype musical score recognition system using mathematical morphology. Mach Vis Appl 6(2–3):140–150
Ng K, Boyle R (1996) Recognition and reconstruction of primitives in music scores. Image Vis Comput 14(1):39–46 Image and Vision Computing Journal on Vision-Based Aids for the Disabled. [Online]. Available: http://www.sciencedirect.com/science/article/pii/026288569 5010386
Pavlidis T (1982) Data structures. In: Algorithms for graphics and image processing. Springer, Berlin, pp. 99–127. doi:10.1007/978-3-642-93208-3_6
Prerau D (1970) Computer pattern recognition of standard engraved music notation. In: Baird H, Bunke H, Yamamoto K (eds) Structured document image analysis. Springer, Berlin. doi:10.1007/3-540-32390-2_98
Pruslin D (1966) Automatic recognition of sheet music. In: Baird H, Bunke H, Yamamoto K (eds) Structured document image analysis, Springer, Berlin
Randriamahefa R, Cocquerez JP, Fluhr C, Pepin F, Philipp S (1993) “Printed music recognition,” in Document Analysis and Recognition. In: Proceedings of the Second International Conference on, pp 898–901
Rebelo A, Capela G, Cardoso JS (2010) Optical recognition of music symbols: a comparative study. Int J Doc Anal Recognit 13(1):19–31. doi:10.1007/s10032-009-0100-1
Acknowledgements
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (2015-018993) and Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (NRF-2015R1C1A1A02036495) and This Research was supported by Korea Institute of Planning and Evaluation for Technology in Food, Agriculture, Forestry and Fisheries(IPET) through Agriculture, Food and Rural Affairs Research Center Support Program(Project No.: 714002-07) funded by Ministry of Agriculture, Food and Rural Affairs(MAFRA).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Na, I.S., Kim, S.H. Music symbol recognition by a LAG-based combination model. Multimed Tools Appl 76, 25563–25579 (2017). https://doi.org/10.1007/s11042-016-4170-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-016-4170-z