Skip to main content
SpringerLink
Log in

Multi-objective evolutionary feature selection for instrument recognition in polyphonic audio mixtures

  • Original Paper
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Instrument recognition is one of the music information retrieval research topics. This task becomes very challenging if several instruments are played simultaneously because of their varying physical characteristics: inharmonic attack noise, energy development during attack–decay–sustain–release envelope or overtone distribution. In our framework, we treat instrument detection as a machine-learning task based on a large amount of preprocessed audio features with target to build classification models. Since classification algorithms are very sensitive to feature input and the optimal feature set differs from instrument to instrument, we propose to run a multi-objective feature selection procedure before building of classification models. Two objectives are considered for evaluation: classification mean-squared error and feature rate (smaller amount of features stands for reduced costs and decreased risk of overfitting). The analysis of the extensive experimental study confirms that application of an evolutionary multi-objective algorithm is a good choice to optimize feature selection for music instrument identification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. http://www.music.mcgill.ca/resources/mums/html.

  2. http://www.theremin.music.uiowa.edu.

References

  • Ahrendt P (2006) Music genre classification systems: Ph.D. thesis. Informatics and mathematical modelling, Technical University of Denmark

  • Beume N, Naujoks B, Emmerich M (2007) SMS-EMOA: multiobjective selection based on dominated hypervolume. Eur J Oper Res 181(3):1653–1669

    Article  MATH  Google Scholar 

  • Bischl B, Mersmann O, Trautmann H, Weihs C (2012) Resampling methods for meta-model validation with recommendations for evolutionary computation. Evol Comput 20(2):249–275

    Google Scholar 

  • Bischl B, Vatolkin I, Preuß M (2010) Selecting small audio feature sets in music classification by means of asymmetric mutation. In: Proceedings of the 11th international conference on parallel problem solving from nature (PPSN). Springer, Berlin, pp 314–323

  • Blume H, Bischl B, Botteck M, Igel C, Martin R, Rötter G, Rudolph G, Theimer W, Vatolkin I, Weihs C (2011) Huge music archives on mobile devices. IEEE Signal Process Mag 28(4):24–33

    Article  Google Scholar 

  • Brown JC, Houix O, McAdams S (2001) Feature dependence in the automatic identification of musical woodwind instruments. J Acoust Soc Am 109(3):1064–1072

    Article  Google Scholar 

  • Coello CAC, Van Veldhuizen DA, Lamont GB (eds) (2006) Evolutionary algorithms for solving multi-objective problems. Kluwer Academic Publishers, New York

  • Deb K (2001) Multi-objective optimization using evolutionary algorithms: Wiley-Interscience Series in systems and optimization.. Wiley, Chichester

    Google Scholar 

  • Eichhoff M, Weihs C (2012) Musical instrument recognition by high-level features. In: Gaul W, Geyer-Schulz A, Schmidt-Thieme L, Kunze J (eds) Challenges at the interface of data analysis, computer science, and optimization. Proceedings of the 34th annual conference of the Gesellschaft für Klassifikation e. V. Springer, Berlin, pp 373–381

  • Emmanouilidis C, Hunter A, MacIntyre J (2000) A multiobjective evolutionary setting for feature selection and a commonality-based crossover operator. In: Proceedings of the 2000 Congress on Evolutionary Computation (CEC), vol 1. IEEE, pp 309–316

  • Eronen A (2003) Musical instrument recognition using ica-based transform of features and discriminatively trained hmms. In: Proceedings of the 7th international symposium on signal processing and its applications (ISSPA), pp 133–136

  • Eronen AJ (2009) Signal processing methods for audio classification and music content analysis: Ph.D. thesis. Department of Signal Processing, Tampere University of Technology, Finland

  • Essid S, Richard G, David B (2006) Instrument recognition in polyphonic music based on automatic taxonomies. IEEE Trans Audio Speech Lang Process 14(1):68–80

    Article  Google Scholar 

  • Fiebrink R, Fujinaga I (2006) Feature selection pitfalls and music classification. In: Proceedings of the 7th international conference on music information retrieval (ISMIR), pp 340–341

  • Fu Z, Lu G, Ting K, Zhang D (2011) A survey of audio-based music classification and annotation. IEEE Trans Multimedia 13(2):303–319

    Article  Google Scholar 

  • Fujinaga I (1998) Machine recognition of timbre using steady-state tone of acoustic musical instruments. In: Proceedings of the international computer music conference (ICMC), ICMA, pp 207–210

  • Goto M, Hashiguchi H, Nishimura T, Oka R (2003) Rwc music database: music genre database and musical instrument sound database. In: Proceedings of the 4th international conference on music information retrieval (ISMIR), pp 229–230

  • Guyon I, Gunn S, Nikravesh M, Zadeh L (eds) (2006) Feature extraction, foundations and applications. Springer, Berlin

  • Hall M (1999) Correlation-based feature selection for machine learning: Ph.D thesis. Department of Computer Science, Waikato University, New Zealand

  • Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The weka data mining software: an update. SIGKDD Explor 11:10–18

    Article  Google Scholar 

  • Heittola T, Klapuri A, Virtanen T (2009) Musical instrument recognition in polyphonic audio using source-filter model for sound separation. In: Proceedings of the 10th international society for music information retrieval conference (ISMIR), pp 327–332

  • Kitahara T, Goto M, Komatani K, Ogata T, Okuno HG (2007) Instrument identification in polyphonic music: Feature weighting to minimize influence of sound overlaps. EURASIP J Adv Signal Process, vol 2007

  • Kobayashi Y (2009) Automatic generation of musical instrument detector by using evolutionary learning method. In: Proceedings of the 10th international society for music information retrieval conference (ISMIR), pp 93–98

  • Lartillot O, Toiviainen P (2007) Mir in Matlab (ii): a toolbox for musical feature extraction from audio. In: Proceedings of the 8th international conference on music information retrieval (ISMIR), pp 127–130

  • Li T, Ogihara M, Tzanetakis G (2011) Music data mining. CRC Press, USA

  • Liu J, Hu X (2010) User-centered music information retrieval evaluation. In: Proceedings of the joint conference on digital libraries (JCDL) workshop: music information retrieval for the masses

  • Livshin A, Rodet X (2006) The significance of the non-harmonic “noise” versus the harmonic series for musical instrument recognition. In: Proceedings of the 7th international conference on music information retrieval (ISMIR), pp 95–100

  • Loughrey J, Cunningham P (2004) Overfitting in wrapper-based feature subset selection: the harder you try the worse it gets. In: M. Bramer, F. Coenen, T. Allen (eds) Proceedings of the 24th SGAI international conference on innovative techniques and applications of artificial intelligence (AI-2004), pp. 33–43

  • Lukashevich H (2008) Towards quantitative measures of evaluating song segmentation. In: Proceedings of the 9th international conference on music information retrival (ISMIR), pp 375–380

  • McEnnis D, McKay C, Fujinaga I (2006) jAudio: additions and improvements. In: Proceedigs of the 7th international conference on music information retrieval (ISMIR), pp 385–386

  • Mierswa I, Morik K (2005) Automatic feature extraction for classifying audio data. Mach Learn J 58(2–3):127–149

    Article  MATH  Google Scholar 

  • Mierswa I, Wurst M, Klinkenberg R, Scholz M, Euler T (2006) Yale rapid prototyping for complex data mining tasks. In:Ungar L, Craven M, Gunopulos D, Eliassi-Rad T (eds) Proceedings of the 12th ACM SIGKDD international conference on knowledge discovery and data mining (KDD). ACM Press, New York, pp 935–940

  • Müller M, Ewert S (2010) Towards timbre-invariant audio features for harmony-based music. IEEE Trans Audio Speech Lang Process 18(3):649–662

    Article  Google Scholar 

  • Müller M, Ewert S (2011) Chroma toolbox: Matlab implementations for extracting variants of chroma-based audio features. In: Proceedings of the 12th international conference on music information retrieval (ISMIR), pp 215–220

  • Nagathil A, Göttel P, Martin R (2011) Hierarchical audio classification using cepstral modulation ratio regressions based on Legendre polynomials. In: Proceedings of the international conference on acoustics, speech and signal processing (ICASSP), Prague, Czech Republic, pp 2216–2219

  • Park TH (2010) Introduction to digital signal processing: computer musically speaking. World Scientific, USA

  • Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann, Menlo Park

  • Ras ZW, Wieczorkowska A (2010) Advances in music information retrieval: studies in computational intelligence, vol 274. Springer, Berlin

  • Reynolds A, Corne D, Chantler M (2010) Feature selection for multi-purpose predictive models: a many-objective task. In: Proceedings of the 11th international conference on parallel problem solving from nature (PPSN). Springer, Berlin, pp 384–393

  • Rudolph G (2012) Evolutionary strategies. In: Rozenberg G, Bäck T, Kok J (eds) Handbook of natural computing. Springer, Berlin

  • Snyman J (2005) Practical mathematical optimization: an introduction to basic optimization theory and classical and new gradient-based algorithms. Springer, Berlin

  • Theimer W, Vatolkin I, Eronen A (2008) Definitions of audio features for music content description: technical report TR08-2-001. Chair of Algorithm Engineering, University of Dortmund, Germany

  • Vatolkin I (2012) Multi-objective evaluation of music classification. In: Gaul W, Geyer-Schulz A, Schmidt-Thieme L, Kunze L (eds) Challenges at the interface of data analysis, computer science, and optimization. Proceedings of the 34th annual conference of the Gesellschaft für Klassifikation e. V. Springer, Berlin, pp 401–410

  • Vatolkin I, Preuß M, Rudolph G (2011) Multi-objective feature selection in music genre and style recognition tasks. In: Krasnogor N, Lanzi PN (eds) Proceedings of the 2011 genetic and evolutionary computation conference (GECCO). ACM Press, New York, pp 411–418

  • Vatolkin I, Theimer W, Botteck M (2010) Amuse (advanced music explorer): a multitool framework for music data analysis. In: Downie JS, Veltkamp RC (eds) Proceedings of the 11th international society on music information retrieval conference (ISMIR), pp 33–38

  • Vatolkin I, Theimer W, Rudolph G (2009) Design and comparison of different evolution strategies for feature selection and consolidation in music classification. In: Proceedings of the 2009 IEEE Congress on Evolutionary Computation (CEC). IEEE Press, pp 174–181

  • Weihs C, Ligges U, Mörchen F, Müllensiefen D (2007) Classification in music research. Adv Data Anal Classif 1(3):255–291

    Article  MathSciNet  MATH  Google Scholar 

  • Zhu Z, Jia S, Ji Z (2010) Towards a memetic feature selection algorithm. IEEE Comput Intell Mag 5(2):41–53

    Article  Google Scholar 

  • Zitzler E, Brockhoff D, Thiele L (2007) The hypervolume indicator revisited: on the design of Pareto-compliant indicators via weighted integration. In: Proceedings of the conference on evolutionary multi-criterion optimization (EMO), vol 4403. Springer, Berlin, pp. 862–876

Download references

Acknowledgements

This work has been supported by the Klaus Tschira Foundation within the project 00.146.2008, “Multi-objective optimization of music classification based on high-level features with Computational Intelligence methods”.

Author information

Authors and Affiliations

  1. Fakultät für Informatik, Technische Universität Dortmund, Otto-Hahn-Str. 14, 44227, Dortmund, Germany

    Igor Vatolkin, Mike Preuß & Günter Rudolph

  2. Fakultät für Statistik, Technische Universität Dortmund, 44221, Dortmund, Germany

    Markus Eichhoff & Claus Weihs

Authors
  1. Igor Vatolkin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mike Preuß
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Günter Rudolph
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Markus Eichhoff
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Claus Weihs
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Igor Vatolkin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vatolkin, I., Preuß, M., Rudolph, G. et al. Multi-objective evolutionary feature selection for instrument recognition in polyphonic audio mixtures. Soft Comput 16, 2027–2047 (2012). https://doi.org/10.1007/s00500-012-0874-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-012-0874-9

Keywords

Search

Navigation