Several real problems involve the classification of data into categories or classes. Given a dataset containing data whose classes are known, Machine Learning algorithms can be employed for the induction of a classifier able to predict the class of new data from the same domain, performing the desired discrimination. Several machine learning techniques are originally conceived for the solution of problems with only two classes. In multiclass applications, an alternative frequently employed is to divide the original problem into binary subtasks, whose results are then combined. The decomposition can be generally represented by a code-matrix, where each row corresponds to a codeword assigned for one class and the columns represent the binary classifiers employed. This chapter presents a survey on techniques for multiclass problems code-matrix design. It also shows how evolutionary techniques can be employed to solve this problem.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alba, E., Cotta, C., Chicano, F., Nebro, A.J., (2002), Parallel evolutionary algorithms in telecommunications: two case studies. In: Proceedings of Congresso Argentino de Ciências de la Computación.
Alba, E., Chicano, J.F., (2004), Solving the error correcting code problem with parallel hybrid heuristics. In: Proceedings of 2004 ACM Symposium on Applied Computing. Volume 2. 985-989.
Allwein, E.L., Shapire, R.E., Singer, Y., (2000), Reducing multiclass to binary: a unifying approach for magin classifiers. In: Proceedings of the 17th International Conference on Machine Learning, Morgan Kaufmann 9-16.
Alpaydin, E., Mayoraz, E., (1999), Learning error-correcting output codes from data. In: Proceedings of the 9th International Conference on Neural Networks. 743-748.
Beasley, D. (2000), (Bäck et al., 2000) 4-18
Berger, A., (1999), Error-correcting output coding for text classification.
Blake, C.L., Merz, C.J., (1998), UCI repository of machine learning databases. Available at: http://www.ics.uci.edu/∼mlearn/MLRepository.html.
Boser, R.C., Ray-Chaudhuri, D.K., 1960, On a class of error-correcting binary group codes. Information and Control 3 68-79.
Bäck, T., Fogel, D.B., Michalewicz, T., (2000), Evolutionary Computation 1: Basic Algorithms and Operators. Institute of Physics Publishing.
Bäck, T. (2000), (Bäck et al., 2000) 132-135
Collins, M., Shapire, R.E., Singer, Y., 2002, Logistic regression, adaboost and bregman distances. Machine Learning 47(2/3) 253-285.
Crammer, K., Singer, Y., 2002, On the learnability and design of output codes for multiclass problems. Machine Learning 47(2-3) 201-233.
Cristianini, N., Shawe-Taylor, J., (2000), An introduction to Support Vector Machines and other kernel-based learning methods. Cambridge University Press.
Darwin, C., 1859, On the origin of species by means of natural selection. John Murray, London.
Deb, K., 2000, An efficient constraint handling method for genetic algorithms. Computer Methods in Applied Mechanics and Engineering 186 311-338.
Dekel, O., Singer, Y., (2003), Multiclass learning by probabilistic embeddings. In: Advances in Neural Information Processing Systems. Volume 15., MIT Press 945-952.
Dietterich, T.G., Bariki, G., 1995, Solving multiclass learning problems via errorcorrecting output codes. Journal of Artificial Intelligence Research 2 263-286.
Dontas, K., Jong, K.D., (1990), Discovery of maximal distance codes using genetic algorithms. In: Proceedings of the 2nd International IEEE Conference on Tools for Artificial Intelligence, IEEE Computer Society Press 905-811.
Eiben, A.E., Smith, J.E., (2003), Introduction to Evolutionary Computing. Springer.
Escalera, S., Pujol, O., Radeva, R., (2006), Decoding of ternary error correcting output codes. In: Proceedings of the 11th Iberoamerican Congress on Pattern Recognition. Volume 4225 of Lecture Notes in Computer Science., SpringerVerlag 753-763.
Freund, Y., Schapire, R.E., 1997, A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 1(55) 119-139.
Fürnkranz, J., 2002, Round robin classification. Journal of Machine Learning Research 2 721-747.
Ghani, R., (2000), Using error correcting output codes for text classification. In: Proceedings of the 17th International Conference on Machine Learning, Morgan Kaufmann 303-310.
Hastie, T., Tibshirani, R., 1998, Classification by pairwise coupling. The Annals of Statistics 2 451-471.
Haykin, S., 1999, Neural Networks - A Compreensive Foundation. 2nd edn. Prentice-Hall, New Jersey.
Holland, J.H., (1975), Adaptation in Natural and Artificial Systems. University of Michigan Press.
Hsu, C.W., Lin, C.J., 2002, A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks 13(2) 415-425.
Klautau, A., Jevtić, N., Orlistky, A., 2003, On nearest-neighbor error-correcting output codes with application to all-pairs multiclass support vector machines. Journal of Machine Learning Research 4 1-15.
Knerr, S., Personnaz, L., Dreyfus, G., 1992, Handwritten digit recognition by neural networks with single-layer training. IEEE Transactions on Neural Networks 3(6) 962-968.
Knerr, S., Personnaz, L., Dreyfus, G., (1990), In: Single-layer learning revisited: a stepwise procedure for building and training a neural network. Springer-Verlag, pp. 41-50
Kreβel, U., (1999), Pairwise classification and support vector machines. In Schölkopf, B., Burges, C.J.C., Smola, A.J., eds.: Advances in Kernel Methods - Support Vector Learning, MIT Press 185-208.
Kuncheva, L.I., 2005, Using diversity measures for generating error-correcting output codes in classifier ensembles. Pattern Recognition Letters 26 83-90.
Lorena, A.C., Carvalho, A.C.P.L.F., (2006), Evolutionary design of multiclass support vector machines. Journal of Intelligent and Fuzzy Systems . Accepted, to be published..
Lorena, A.C., (2006), Investigação de estratégias para a geração de máquinas de vetores de suporte multiclasses [in portuguese], Ph.D. thesis, Departamento de Ciências de Computação, Instituto de Ciências Matemáticas e de Computação, Universidade de São Paulo, São Carlos, Brazil, http://www.teses.usp.br/ teses/disponiveis/55/55134/tde- 26052006- 111406.
ı, R., Laguna, M., Campos, V., (2005), Scatter search vs. genetic algorithms: An experimental evaluation with permutation problems. In Rego, C., Alidaee, B., eds.: Metaheuristic Optimization Via Adaptive Memory and Evolution: Tabu Search and Scatter Search. Kluwer Academic Publishers 263-282.
Masulli, F., Valentini, G., (2000), Effectiveness of error correcting output codes in multiclass learning problems. In: Proceedings of the 1st International Workshop on Multiple Classifier Systems. Volume 1857 of Lecture Notes in Computer Science., Springer-Verlag 107-116.
Mayoraz, E., Alpaydim, E., 1998, Support vector machines for multi-class classification. Research Report IDIAP-RR-98-06, Dalle Molle Institute for Perceptual Artificial Intelligence, Martigny, Switzerland.
Mayoraz, E., Moreira, M., 1996, On the decomposition of polychotomies into dichotomies. Research Report 96-08, IDIAP, Dalle Molle Institute for Perceptive Artificial Intelligence, Martigny, Valais, Switzerland.
Michalewicz, Z., Fogel, D.B., (2004), How to solve it: modern heuristics. Springer. Mitchell, T., (1997), Machine Learning. McGraw Hill.
Mitchell, M., (1999), An introduction to Genetic Algorithms. MIT Press.
Passerini, A., Pontil, M., Frasconi, P., 2004, New results on error correcting output codes of kernel machines. IEEE Transactions on Neural Networks 15 45-54.
Pimenta, E., Gama, J., (2005), A study on error correcting output codes. In: Proceedings of the 2005 Portuguese Conference on Artificial Intelligence, IEEE Computer Society Press 218-223.
Pimenta, E.M.C., 2005, Abordagens para decomposição de problemas multiıda (in portuguese). Master’s thesis, Departamento de Ciências de Computadores, Faculdade de Ciências da Universidade do Porto, Portugal.
Pujol, O., Tadeva, P., Vitrià, J., 2006, Discriminant ECOC: a heuristic method for application dependetn design of error correcting output codes. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(6) 1007-1012.
ıguez, A., (2002), Puncturing multi-class support vector machines. In: Proceedings of the 12th International Conference on Neural Networks (ICANN). Volume 2415 of Lecture Notes in Computer Science., Springer-Verlag 751-756.
Quinlan, J.R., 1986, Induction of decision trees. Machine Learning 1(1) 81-106. Rifkin, R., Klautau, A., (2004), In defense of one-vs-all classification. Journal of Machine Learning Research 5 1533-7928.
Rätsch, G., Smola, A.J., Mika, S., (2003), Adapting codes and embeddings for polychotomies. In: Advances in Neural Information Processing Systems. Volume 15., MIT Press 513-520.
Shen, L., Tan, E.C., (2005), Seeking better output-codes with genetic algorithm for multiclass cancer classification. Submitted to Bioinformatics.
Simn, M.D.J., Pulido, J.A.G., Rodrguez, M.A.V., (2006), Prez, J.M.S., Criado, J.M.G., A genetic algorithm to design error correcting codes. In: Proceedings of the 13th IEEE Mediterranean Eletrotechnical Conference 2006, IEEE Computer Society Press 807-810.
Statnikov, A., Aliferis, C.F., Tsamardinos, I., 2005, Hardin, D., Levy, S., A comprehensive evaluation of multicategory methods for microarray gene expression cancer diagnosis. Bioinformatics 21(5) 631-643.
ıa-Villalba, J., Villena, J., (2001), Recursive adaptive ECOC models. In: Proceedings of the 10th Portuguese Conference on Artificial Intelligence. Volume 2258 of Lecture Notes in Artificial Intelligence., Springer-Verlag 96-103.
ıa-Villalba, J., (2003), Good error correcting output codes for adaptive multiclass learning. In: Proceedings of the 4th International Workshop on Multiple Classifier Systems 2003. Volume 2709 of Lecture Notes in Computer Science., Springer-Verlag 156-165.
Wallet, B.C., Marchette, D.J., Solka, J.L., (1996), A matrix representation for genetic algorithms. In: Automatic object recognition VI, Proceedings of the International Society for Optical Engineering. 206-214.
Wallis, J.L., Houghten, S.K., (2002), A comparative study of search techniques applied to the minimum distance problem of BCH codes. Technical Report CS-02-08, Department of Computer Science, Brock University.
Windeatt, T., Ghaderi, R., 2003, Coding and decoding strategies for multi-class learning problems. Information Fusion 4(1) 11-21.
Zhang, A., Wu, Z.L., Li, C.H., Fang, K.T., (2003), On hadamard-type output coding in multiclass learning. In: Proceedings of IDEAL. Volume 2690 of Lecture Notes in Computer Science., Springer-Verlag 397-404.
Zitzler, E., Laumanns, M., Thiele, L., (2002), SPEA2: Improving the strength pareto evolutionary algorithm. In: Evolutionary Methods for Design, Optimisation, and Control, CIMNE, Barcelona, Spain. 95-100.
Zitzler, E., Laumanns, M., Bleuler, S., (2004), A tutorial on evolutionary multiobjective optimization. In Gandibleux, X., Sevaux, M., Srensen, K., T’kindt, V., eds.: Metaheuristics for Multiobjective Optimisation. Volume 535 of Lecture Notes in Economics and Mathematical Systems., Springer-Verlag 3-37.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Lorena, A.C., de Carvalho, A.C.P.L.F. (2008). evolutionary Design of Code-matrices for Multiclass Problems. In: Maimon, O., Rokach, L. (eds) Soft Computing for Knowledge Discovery and Data Mining. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-69935-6_7
Download citation
DOI: https://doi.org/10.1007/978-0-387-69935-6_7
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-69934-9
Online ISBN: 978-0-387-69935-6
eBook Packages: Computer ScienceComputer Science (R0)