Skip to main content
SpringerLink
Log in

Multi-Instance Learning from Supervised View

  • Applications
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

In multi-instance learning, the training set comprises labeled bags that are composed of unlabeled instances, and the task is to predict the labels of unseen bags. This paper studies multi-instance learning from the view of supervised learning. First, by analyzing some representative learning algorithms, this paper shows that multi-instance learners can be derived from supervised learners by shifting their focuses from the discrimination on the instances to the discrimination on the bags. Second, considering that ensemble learning paradigms can effectively enhance supervised learners, this paper proposes to build multi-instance ensembles to solve multi-instance problems. Experiments on a real-world benchmark test show that ensemble learning paradigms can significantly enhance multi-instance learners.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Maron O. Learning from ambiguity [Dissertation]. Department of Electrical Engineering and Computer Science, MIT, Jun. 1998.

  2. Dietterich T G, Lathrop R H, Lozano-Pérez T. Solving the multiple-instance problem with axis-parallel rectangles. Artificial Intelligence, 1997, 89(1-2): 31–71.

    Article  MATH  Google Scholar 

  3. Dietterich T G. Machine learning research: Four current directions. AI Magazine, 1997, 18(4): 97–136.

    Google Scholar 

  4. Lindsay R, Buchanan B, Feigenbaum E, Lederberg J. Applications of Artificial Intelligence to Organic Chemistry: The DENDRAL Project. New York: McGraw-Hill, 1980.

    Google Scholar 

  5. Zucker J-D, Ganascia J-G. Changes of representation for efficient learning in structural domains. In Proc. the 13th Int. Conf. Machine Learning, Bary, Italy, 1996, pp. 543–551.

  6. Long P M, Tan L. PAC learning axis-aligned rectangles with respect to product distributions from multiple-instance examples. Machine Learning, 1998, 30(1): 7–21.

    Article  MATH  Google Scholar 

  7. Auer P, Long P M, Srinivasan A. Approximating hyper-rectangles: Learning and pseudo-random sets. Journal of Computer and System Sciences, 1998, 57(3): 376–388.

    Article  MATH  MathSciNet  Google Scholar 

  8. Auer P. On learning from multi-instance examples: Empirical evaluation of a theoretical approach. In Proc. the 14th Int. Conf. Machine Learning, Nashville, TN, 1997, pp. 21–29.

  9. Blum A, Kalai A. A note on learning from multiple-instance examples. Machine Learning, 1998, 30(1): 23–29.

    Article  MATH  Google Scholar 

  10. Maron O, Lozano-Pérez T. A Framework for Multiple-Instance Learning. Advances in Neural Information Processing Systems 10, Jordan M I, Kearns M J, Solla S A (eds.), Cambridge, MA: MIT Press, 1998, pp. 570–576.

  11. Maron O, Ratan A L. Multiple-instance learning for natural scene classification. In Proc. the 15th Int. Conf. Machine Learning, Madison, WI, 1998, pp. 341–349.

  12. McGovern A, Barto A G. Automatic discovery of subgoals in reinforcement learning using diverse density. In Proc. the 18th Int. Conf. Machine Learning, Williamstown, MA, 2001, pp. 361–368.

  13. Yang C, Lozano-Pérez T. Image database retrieval with multiple-instance learning techniques. In Proc. the 16th Int. Conf. Data Engineering, San Diego, CA, 2000, pp. 233–243.

  14. Zhou Z-H, Zhang M-L, Chen K-J. A novel bag generator for image database retrieval with multi-instance learning techniques. In Proc. the 15th IEEE Int. Conf. Tools with Artificial Intelligence, Sacramento, CA, 2003, pp. 565–569.

  15. Chen Y, Wang J Z. Image categorization by learning and reasoning with regions. Journal of Machine Learning Research, 2004, 5: 913–939.

    Google Scholar 

  16. Dempster A P, Laird N M, Rubin D B. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistics Society — B, 1977, 39(1): 1–38.

    MATH  MathSciNet  Google Scholar 

  17. Zhang Q, Goldman S A. EM-DD: An Improved Multi-Instance Learning Technique. Advances in Neural Information Processing Systems 14, Dietterich T G, Becker S, Ghahramani Z (eds.), Cambridge, MA: MIT Press, 2002, pp. 1073–1080.

  18. Zhang Q, Yu W, Goldman S A, Fritts J E. Content-based image retrieval using multiple-instance learning. In Proc. the 19th Int. Conf. Machine Learning, Sydney, Australia, 2002, pp. 682–689.

  19. Wang J, Zucker J-D. Solving the multiple-instance problem: A lazy learning approach. In Proc. the 17th Int. Conf. Machine Learning, San Francisco, CA, 2000, pp. 1119–1125.

  20. Zhou Z-H, Jiang K, Li M. Multi-instance learning based web mining. Applied Intelligence, 2005, 22(2): 135–147.

    Article  Google Scholar 

  21. Zhou Z-H, Zhang M-L. Neural networks for multi-instance learning. Technical Report, AI Lab, Computer Science & Technology Department, Nanjing University, Nanjing, China, Aug. 2002.

  22. Zhang M-L, Zhou Z-H. Improve multi-instance neural networks through feature selection. Neural Processing Letters, 2004, 19(1): 1–10.

    Google Scholar 

  23. Ruffo G. Learning single and multiple instance decision tree for computer security applications [Dissertation]. Department of Computer Science, University of Turin, Torino, Italy, Feb. 2000.

  24. Chevaleyre Y, Zucker J-D. Solving multiple-instance and multiple-part learning problems with decision trees and decision rules. Application to the mutagenesis problem. Lecture Notes in Artificial Intelligence 2056, Stroulia E, Matwin S (eds.), Berlin: Springer, 2001, pp. 204–214.

  25. Gärtner T, Flach P A, Kowalczyk A, Smola A J. Multi-instance kernels. In Proc. the 19th Int. Conf. Machine Learning, Sydney, Australia, 2002, pp. 179–186.

  26. Andrews S, Tsochantaridis I, Hofmann T. Support Vector Machines for Multiple-Instance Learning. Advances in Neural Information Processing Systems 15, Becker S, Thrun S, Obermayer K (eds.), Cambridge, MA: MIT Press, 2003, pp. 561–568.

  27. Zhou Z-H, Zhang M-L. Ensembles of multi-instance learners. Lecture Notes in Artificial Intelligence 2837, Lavrač N, Gamberger D, Blockeel H, Todorovski L (eds.), Berlin: Springer, 2003, pp. 492–502.

  28. Xu X, Frank E. Logistic regression and boosting for labeled bags of instances. Lecture Notes in Artificial Intelligence 3056, Dai H, Srikant R, Zhang C (eds.), Berlin: Springer, 2004, pp. 272–281.

  29. Zhang M-L, Zhou Z-H. Adapting RBF neural networks to multi-instance learning. Neural Processing Letters, 2006, 23(1): 1–26.

    Google Scholar 

  30. Amar R A, Dooly D R, Goldman S A, Zhang Q. Multiple-instance learning of real-valued data. In Proc. the 18th Int. Conf. Machine Learning, Williamstown, MA, 2001, pp. 3–10.

  31. Ray S, Page D. Multiple instance regression. In Proc. the 18th Int. Conf. Machine Learning, Williamstown, MA, 2001, pp. 425–432.

  32. Dooly D R, Zhang Q, Goldman S A, Amar R A. Multiple-instance learning of real-valued data. Journal of Machine Learning Research, 2002, 3: 651–678.

    Article  Google Scholar 

  33. Goldman S A, Scott S D. Multiple-instance learning of real-valued geometric patterns. Annals of Mathematics and Artificial Intelligence, 2003, 39(3): 259–290.

    Article  MATH  MathSciNet  Google Scholar 

  34. Weidmann N, Frank E, Pfahringer B. A two-level learning method for generalized multi-instance problem. Lecture Notes in Artificial Intelligence 2837, Lavrač N, Gamberger D, Blockeel H, Todorovski L (eds.), Berlin: Springer, 2003, pp. 468–479.

  35. Scott S D, Zhang J, Brown J. On generalized multiple-instance learning. Technical Report UNL-CSE-2003-5, Department of Computer Science, University of Nebraska, Lincoln, NE, 2003.

  36. Tao Q, Scott S, Vinodchandran N V, Osugi T T. SVM-based generalized multiple-instance learning via approximate box counting. In Proc. the 21st Int. Conf. Machine Learning, Banff, Canada, 2004, pp. 779–806.

  37. Tao Q, Scott S, Vinodchandran N V, Osugi T T, Mueller B. An extended kernel for generalized multiple-instance learning. In Proc. the 16th IEEE Int. Conf. Tools with Artificial Intelligence, Boca Raton, FL, 2004, pp. 272–277.

  38. De Raedt L. Attribute-value learning versus inductive logic programming: The missing links. Lecture Notes in Artificial Intelligence 1446, Page D (ed.), Berlin: Springer, 1998, pp. 1–8.

  39. Alphonse É, Matwin S. Filtering multi-instance problems to reduce dimensionality in relational learning. Journal of Intelligent Information Systems, 2004, 22(1): 23–40.

    Article  Google Scholar 

  40. McGovern A, Jensen D. Identifying predictive structures in relational data using multiple instance learning. In Proc. the 20th Int. Conf. Machine Learning, Washington DC, 2003, pp. 528–535.

  41. Freund Y, Schapire R E. A decision-theoretic generalization of on-line learning and an application to boosting. Lecture Notes in Computer Science 904, Vitányi P M B (ed.), Berlin: Springer, 1995, pp. 23–37.

  42. Breiman L. Bagging predictors. Machine Learning, 1996, 24(2): 123–140.

    MATH  MathSciNet  Google Scholar 

  43. Zhou Z-H, Wu J, Tang W. Ensembling neural networks: Many could be better than all. Artificial Intelligence, 2002, 137(1-2): 239–263.

    Article  MATH  MathSciNet  Google Scholar 

  44. Efron B, Tibshirani R. An Introduction to the Bootstrap. New York: Chapman & Hall, 1993.

    MATH  Google Scholar 

  45. Blake C, Keogh E, Merz C J. UCI repository of machine learning databases. Department of Information and Computer Science, University of California, Irvine, CA, 1998, http://www.ics.uci.edu/~mlearn/MLRepository.html.

  46. Zhou Z-H, Zhang M-L. Solving multi-instance problems with classifier ensemble based on constructive clustering. Knowledge and Information Systems, 2006, in press.

  47. Blockeel H, Page D, Srinivasan A. Multi-instance tree learning. In Proc. the 22nd Int. Conf. Machine Learning, Bonn, Germany, 2005.

  48. Zhou Z-H, Xue X-B, Jiang Y. Locating regions of interest in CBIR with multi-instance learning techniques. Lecture Notes in Artificial Intelligence 3809, Zhang S, Jarvis R (eds.), Berlin: Springer, 2005, pp. 92–101.

Download references

Author information

Authors and Affiliations

  1. National Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210093, P.R. China

    Zhi-Hua Zhou

Authors
  1. Zhi-Hua Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhi-Hua Zhou.

Additional information

Regular Paper: Supported by the National Natural Science Foundation of China under Grant Nos. 60105004 and 60325207.

Zhi-Hua Zhou received the B.Sc., M.Sc. and Ph.D. degrees in computer science from Nanjing University, China, in 1996, 1998 and 2000, respectively, all with the highest honor. He joined the Department of Computer Science & Technology of Nanjing University as a lecturer in 2001, and at present he is a professor and head of the LAMDA group. His research interests are in artificial intelligence, machine learning, data mining, information retrieval, pattern recognition, neural computing, and evolutionary computing. In these areas he has published over 70 technical papers in refereed international journals or conference proceedings. He has won various awards. He is an associate editor of Knowledge and Information Systems, on the editorial boards of Artificial Intelligence in Medicine, International Journal of Data Warehousing and Mining, Journal of Computer Science and Technology and Journal of Software, and guest editor/co-editor of ACM/Springer Multimedia Systems, The Computer Journal, etc. He served as program committee member for various international conferences and chaired a number of native conferences. He is a senior member of China Computer Federation (CCF) and the vice chair of CCF Artificial Intelligence & Pattern Recognition Society, an executive committee member of Chinese Association of Artificial Intelligence (CAAI), the vice chair and chief secretary of CAAI Machine Learning Society, a member of AAAI and ACM, and a senior member of IEEE and IEEE Computer Society.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhou, ZH. Multi-Instance Learning from Supervised View. J Comput Sci Technol 21, 800–809 (2006). https://doi.org/10.1007/s11390-006-0800-7

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-006-0800-7

Keywords

Search

Navigation