Abstract
Multi-instance learning is regarded as a new learning framework where the training examples are bags composed of instances without labels, and the task is to predict the labels of unseen bags through analyzing the training bags with known labels. Recently, a multi-instance neural network BP-MIP was proposed. In this paper, BP-MIP is improved through adopting two different feature selection techniques, i.e. feature scaling with Diverse Density and feature reduction with principal component analysis. In detail, before feature vectors are fed to a BP-MIP neural network, they are scaled by the feature weights found by running Diverse Density on the training data, or projected by a linear transformation matrix formed by principal component analysis. Experiments show that these feature selection mechanisms can significantly improve the performance of BP-MIP.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Amar, R. A., Dooly, D. R., Goldman, D. R. and Zhang, Q.:Multiple-instance learning of real-valued data, In:Proceedings of the 18th International Conference on Machine Learning, pp. 3–10, Williamstown, MA, 2001.
Auer, P.:On learning from multi-instance examples:empirical evaluation of a theoretical approach, In:Proceedings of the 14th International Conference on Machine Learning, pp. 21–29, Nashville, TN, 1997.
Auer, P., Long, P. M. and Srinivasan, A.:Approximating hyper-rectangles learning and pseudo-random sets, J. Comput. Syst. Sci. 57 (3) (1998), 376–388.
Blake, C., Keogh, E. and Merz, C. J.:UCI repository of machine learning databases, Department of Information and Computer Science, University of California, Irvine, CA, 1998. [http://www.ics.uci.edu/mlearn/MLRepository.html ]
Blum, A. and Kalai, A.:A note on learning from multiple-instance examples, Mach. Learn. 30 (1) (1998), 23–29.
Chevaleyre, Y. and Zucker, J.-D.:Solving multiple-instance and multiple-part learning problems with decision trees and decision rules. Application to the mutagenesis problem, In:E. Stroulia and S. Matwin (eds), Lecture Notes in Artificial Intelligence, Vol. 2056, Berlin, Springer, pp. 204–214, 2001.
Dietterich, T. G., Lathrop, R. H. and Lozano-Pérez, T.:Solving the multiple-instance problem with axis-parallel rectangles, Artif. Intell. 89 (1–2) (1997), 31–71.
Dooly, D. R., Goldman, S. A. and Kwek, S. S.:Real-valued multiple-instance learning with queries, In:N. Abe, R. Khardon and R. Zeugmann (eds), Lecture Notes in Artificial Intelligence, Vol. 2225, Berlin, Springer, pp. 167–180, 2001.
Jollife, I. T.:Principal Component Analysis, Springer-Verlag, New York, 1986.
Long, P. M. and Tan, L.:PAC learning axis-aligned rectangles with respect to product distributions from multiple-instance examples, Mach. Learn. 30 (1) (1998), 7–21.
Maron, O.:Learning from Ambiguity. Ph D thesis, Department of Electrical Engineering and Computer Science, MIT, June 1998.
Maron, O. and Lozano-Pérez, T.:A framework for multiple-instance learning, In: M. I. Jordan and M. J. Kearns (eds), Advances in Neural Information Processing Systems 10, Cambridge, MA, MIT Press, pp. 570–576, 1998.
Maron, O. and Ratan, A. L.:Multiple-instance learning for natural scene classification, In:Proceedings of the 15th International Conference on Machine Learning, pp. 341–349, Madison, WI, 1998.
Pearl, J.:Probabilistic Reasoning in Intelligent Systems:Networks of Plausible Inference, Morgan Kaufmann, San Mateo, CA, 1988.
Ray, S. and Page, D.:Multiple instance regression, In:Proceedings of the 18th Interna-tional Conference on Machine Learning, pp. 425–432, Williamstown, MA, 2001.
Ruffo, G.:Learning Single and Multiple Instance Decision Tree for Computer Security Applications. Ph D thesis, Department of Computer Science, University of Turin, Torino, Italy, 2000.
Rumelhart, D. E., Hinton, G. E. and Williams, R. J.:Learning internal representations by error propagation, In:D. E. Rumelhart and J. L. McClelland (eds), Parallel Distributed Processing:Explorations in the Microstructure of Cognition, Vol. 1, pp. 318–362, Cambridge, MA, MIT Press, 1986.
Wang, J. and Zucker, J.-D.:Solving the multiple-instance problem:a lazy learning approach, In:Proceedings of the 17th International Conference on Machine Learning, pp. 1119–1125, San Francisco, CA, 2000.
Yang, C. and Lozano-Pérez, T.:Image database retrieval with multiple-instance learning techniques, In:Proceedings of the 16th International Conference on Data Engineering, pp. 233–243, San Diego, CA, 2000.
Zhang, Q. and Goldman, S. A.:EM-DD:an improved multi-instance learning technique, In:T. G. Dietterich, S. Becker and Z. Ghahramani (eds), Advances in Neural Information Processing Systems 14, Cambridge, MA, MIT Press, pp. 1073–1080, 2002.
Zhou, Z.-H. and Zhang, M.-L.:Neural networks for multi-instance learning, Technical Report, AI Lab, Computer Science & Technology Department, Nanjing University, China, Aug. 2002.
Zhou, Z.-H. and Zhang, M.-L.:Ensembles of multi-instance learners. In:N. Lavrac, D. Gamberger, H. Blockeel and L. Todorovski, (eds), Lecture Notes in Arti cial Intelligence, Vol. 2837, Berlin, Springer, pp. 492–502, 2003.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, ML., Zhou, ZH. Improve Multi-Instance Neural Networks through Feature Selection. Neural Processing Letters 19, 1–10 (2004). https://doi.org/10.1023/B:NEPL.0000016836.03614.9f
Issue Date:
DOI: https://doi.org/10.1023/B:NEPL.0000016836.03614.9f