Abstract
We introduce a new notion called Fourier-accessibility that allows us to precisely characterize the class of Boolean functions for which a standard greedy learning algorithm successfully learns all relevant attributes. If the target function is Fourier-accessible, then the success probability of the greedy algorithm can be made arbitrarily close to one. On the other hand, if the target function is not Fourier-accessible, then the error probability tends to one. Finally, we extend these results to the situation where the input data are corrupted by random attribute and classification noise and prove that greedy learning is quite robust against such errors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Akutsu, T., Bao, F.: Approximating Minimum Keys and Optimal Substructure Screens. In: Cai, J.-Y., Wong, C.K. (eds.) COCOON 1996. LNCS, vol. 1090, pp. 290–299. Springer, Heidelberg (1996)
Akutsu, T., Miyano, S., Kuhara, S.: Algorithms for Identifying Boolean Networks and Related Biological Networks Based on Matrix Multiplication and Fingerprint Function. J. Comput. Biology 7(3-4), 331–343 (2000)
Akutsu, T., Miyano, S., Kuhara, S.: A Simple Greedy Algorithm for Finding Functional Relations: Efficient Implementation and Average Case Analysis. Theoret. Comput. Sci. 292(2), 481–495 (2003)
Almuallim, H., Dietterich, T.G.: Learning Boolean Concepts in the Presence of Many Irrelevant Features. Artificial Intelligence 69(1-2), 279–305 (1994)
Alon, N., Spencer, J.: The Probabilistic Method. Wiley-Intersci. Ser. Discrete Math. Optim. John Wiley and Sons, Chichester (1992)
Arpe, J.: Learning Concepts with Few Unknown Relevant Attributes from Noisy Data. PhD thesis, Institut für Theoretische Informatik, Universität zu Lübeck (2006)
Arpe, J., Reischuk, R.: Robust Inference of Relevant Attributes. In: Gavaldá, R., Jantke, K.P., Takimoto, E. (eds.) ALT 2003. LNCS (LNAI), vol. 2842, pp. 99–113. Springer, Heidelberg (2003)
Arpe, J., Reischuk, R.: Learning Juntas in the Presence of Noise. In: Cai, J.-Y., Cooper, S.B., Li, A. (eds.) TAMC 2006. LNCS, vol. 3959, pp. 387–398. Springer, Heidelberg, Invited to appear in special issue of TAMC 2006 in Theoret. Comput. Sci., Series A (2006)
Arpe, J., Reischuk, R.: When Does Greedy Learning of Relevant Attributes Succeed?—A Fourier-based Characterization. Technical Report ECCC TR06-065, Electronic Colloquium on Computational Complexity (2006)
Bahadur, R.R.: A Representation of the Joint Distribution of Responses to n Dichotomous Items. In: Solomon, H. (ed.) Studies in Item Analysis and Prediction, pp. 158–168. Stanford University Press, Stanford (1961)
Bernasconi, A.: Mathematical Techniques for the Analysis of Boolean Functions. PhD thesis, Università degli Studi di Pisa, Dipartimento di Ricerca in Informatica (1998)
Blum, A., Furst, M., Jackson, J.C., Kearns, M., Mansour, Y., Rudich, S.: Weakly Learning DNF and Characterizing Statistical Query Learning Using Fourier Analysis. In: Proc. 26th STOC 1994, pp. 253–262 (1994)
Blum, A., Langley, P.: Selection of Relevant Features and Examples in Machine Learning. Artificial Intelligence 97(1-2), 245–271 (1997)
Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.K.: Occam’s Razor. Inform. Process. Lett. 24(6), 377–380 (1987)
Boros, E., Horiyama, T., Ibaraki, T., Makino, K., Yagiura, M.: Finding Essential Attributes from Binary Data. Ann. Math. Artif. Intell. 39(3), 223–257 (2003)
Chvátal, V.: A Greedy Heuristic for the Set Covering Problem. Math. Oper. Res. 4(3), 233–235 (1979)
Feige, U.: A Threshold of ln n for Approximating Set Cover. J. ACM 45(4), 634–652 (1998)
Fukagawa, D., Akutsu, T.: Performance Analysis of a Greedy Algorithm for Inferring Boolean Functions. Inform. Process. Lett. 93(1), 7–12 (2005)
Furst, M.L., Jackson, J.C., Smith, S.W.: Improved Learning of AC 0 Functions. In: Proc. 4th COLT 1991, pp. 317–325
Johnson, D.S.: Approximation Algorithms for Combinatorial Problems. J. Comput. System Sci. 9(3), 256–278 (1974)
Kleinberg, J., Tardos, É.: Algorithm Design. Addison-Wesley, Reading (2005)
Linial, N., Mansour, Y., Nisan, N.: Constant Depth Circuits, Fourier Transform, and Learnability. J. ACM 40(3), 607–620 (1993)
Mansour, Y.: Learning Boolean Functions via the Fourier Transform. In: Roychodhury, V., Siu, K.-Y., Orlitsky, A. (eds.) Theoretical Advances in Neural Computation and Learning, pp. 391–424. Kluwer Academic Publishers, Dordrecht (1994)
Mossel, E., O’Donnell, R.W., Servedio, R.A.: Learning functions of k relevant variables. J. Comput. System Sci. 69(3), 421–434 (2004)
Reischuk, R.: Can Large Fanin Circuits Perform Reliable Computations in the Presence of Noise? Theoretical Comput. Sci. 240(4), 319–335 (2000)
Shamir, R., Dietrich, B.: Characterization and Algorithms for Greedily Solvable Transportation Problems. In: Proc. 1st SODA 1990, pp. 358–366 (1990)
Slavík, P.: A Tight Analysis of the Greedy Algorithm for Set Cover. In: Proc. 28th STOC 1996, pp. 435–441 (1996)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Arpe, J., Reischuk, R. (2007). When Does Greedy Learning of Relevant Attributes Succeed?. In: Lin, G. (eds) Computing and Combinatorics. COCOON 2007. Lecture Notes in Computer Science, vol 4598. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73545-8_30
Download citation
DOI: https://doi.org/10.1007/978-3-540-73545-8_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73544-1
Online ISBN: 978-3-540-73545-8
eBook Packages: Computer ScienceComputer Science (R0)