Abstract
The supervised learning algorithms assume that the training data has a fixed set of predicting attributes and a single-dimensional class which contains the class label of each training example. However, many real-world domains may contain several objectives each characterized by its own set of labels. Though one may induce a separate model for each objective, there are several reasons to prefer a shared multi-objective model over a collection of single-objective models. We present a novel, greedy algorithm, which builds a shared classification model in the form of an ordered (oblivious) decision tree called Multi-Objective Info-Fuzzy Network (M-IFN). We compare the M-IFN structure to Shared Binary Decision Diagrams and bloomy decision trees and study the information-theoretic properties of the proposed algorithm. These properties are further supported by the results of empirical experiments, where we evaluate M-IFN performance in terms of accuracy and readability on real-world multi-objective tasks from several domains.
Chapter PDF
Similar content being viewed by others
Keywords
References
Babu, H., Sasao, T.: Shared Multi-Terminal Binary Decision Diagrams for Multiple-Output Functions. IEICE Trans. Fundamentals of Electronics, Communications and Computer Sciences E81-A(12), 2545–2553 (1998)
Babu, H., Sasao, T.: Representations of Multiple-Output Functions Using Binary Decision Diagrams for Characteristic Functions. IEICE Trans. Fundamentals E82-A(11), 2398–2406 (1999)
Blake, C., Merz, C.J.: UCI Repository of Machine Learning Databases. Machinereadable data repository, Department of Information and Computer Science, University of California at Irvine, Irvine, CA. (2000), Available at http://www.ics.uci.edu/~mlearn/MLRepository.html
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, P.J.: Classification and Regression Trees. Wadsworth, Belmont (1984)
Bryant, R.E.: Graph-Based Algorithms for Boolean Function Manipulation. IEEE Transactions on Computers C-35-8, 677–691 (1986)
Caruana, R.: Multitask Learning: A Knowledge-Based Source of Inductive Bias. In: Proceedings of the 10th International Conference on Machine Learning, ML 1993, University of Massachusetts, Amherst, pp. 41–48 (1993)
Caruana, R.: Multitask Learning. Machine Learning 28, 41–75 (1997)
Chan, R.: Protecting Rivers & Streams by Monitoring Chemical Concentrations and Algae Communities. In: The 3rd International Competition of Data Analysis by Intelligent Techniques (1999), http://www.erudit.de/erudit/competitions/ic-99/
Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, Chichester (1991)
Dietterich, T.G., Hild, H., Bakiri, G.: A Comparison of ID3 and Backpropagation for English Text-to speech Mapping. Machine Learning 18(1), 51–80 (1995)
Fayyad, U., Irani, K.: Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning. In: Proc. Thirteenth Int’l Joint Conference on Artificial Intelligence, San Mateo, CA, pp. 1022–1027 (1993)
Fayyad, U., Piatetsky-Shapiro, G., Smyth, P.: From Data Mining to Knowledge Discovery: An Overview. In: Piatetsky-Shapiro, U.G., Smyth, P., Uthurusamy, R. (eds.) Advances in Knowledge Discovery and Data Mining, Fayyad, AAAI/MIT Press (1996)
GVU’s WWW User Survey. Georgia Tech Research Corporation (1998), www.gvu.gatech.edu/user_surveys
Han, T.S.: Nonnegative Entropy Measures of Multivariate Symmetric Correlations. Information and Control 36(2), 133–156 (1978)
Han, J., Kamber, M.: Data Mining: Concepts and Techniques. Morgan Kaufmann, San Francisco (2001)
Hettich, S., Bay, S.D.: The UCI KDD Archive. Irvine, CA: University of California, Department of Information and Computer Science (1999), http://kdd.ics.uci.edu
Kohavi, R.: Bottom-Up Induction of Oblivious Read-Once Decision Graphs. In: Proceedings of the European Conference on Machine Learning (1994)
Kohavi, R., Li, C.-H.: Oblivious Decision Trees, Graphs, and Top-Down Pruning. In: Proc. of International Joint Conference on Artificial Intelligence (IJCAI), pp. 1071–1077 (1995)
Last, M., Klein, Y., Kandel, A.: Knowledge Discovery in Time Series Databases. IEEE Transactions on Systems, Man, and Cybernetics 31 Part B(1), 160–169 (2001)
Last, M.: Online Classification of Nonstationary Data Streams. Intelligent Data Analysis 6(2), 129–147 (2002)
Last, M., Maimon, O.: A Compact and Accurate Model for Classification. IEEE Transactions on Knowledge and Data Engineering 16(2), 203–215 (2004)
Last, M., Friedman, M., Kandel, A.: The Data Mining Approach to Automated Software Testing. In: Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2003), pp. 388–396 (2003)
Maimon, O., Last, M.: Knowledge Discovery and Data Mining – The Info-Fuzzy Network (IFN) Methodology. Kluwer Academic Publishers, Massive Computing (2000)
Minato, S.: Graph-Based Representations of Discrete Functions. In: Sasao, T., Fujita, M. (eds.) Representations of Discrete Functions, pp. 1–28. Kluwer Academic Publishers, Dordrecht (1996)
Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997)
Provost, F., Domingos, P.: Tree Induction for Probability-Based Ranking. Machine Learning 52, 199–215 (2003)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
Rao, C.R., Toutenburg, H.: Linear Models: Least Squares and Alternatives. Springer, Heidelberg (1995)
Suzuki, E., Gotoh, M., Choki, Y.: Bloomy Decision Tree for Multi-objective Classification. In: Siebes, A., De Raedt, L. (eds.) PKDD 2001. LNCS (LNAI), vol. 2168, pp. 436–447. Springer, Heidelberg (2001)
Last, M., Friedman, M.: Black-Box Testing with Info-Fuzzy Networks. In: Last, M., Kandel, A., Bunke, H. (eds.) Artificial Intelligence Methods in Software Testing, pp. 21–50. World Scientific, Singapore (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Last, M. (2004). Multi-objective Classification with Info-Fuzzy Networks. In: Boulicaut, JF., Esposito, F., Giannotti, F., Pedreschi, D. (eds) Machine Learning: ECML 2004. ECML 2004. Lecture Notes in Computer Science(), vol 3201. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30115-8_24
Download citation
DOI: https://doi.org/10.1007/978-3-540-30115-8_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23105-9
Online ISBN: 978-3-540-30115-8
eBook Packages: Springer Book Archive