Abstract
Multiple instance learning (MIL) has been studied actively in recent years. However, it is facing a computational challenge due to the large scale of data volume. Parallel computing is a good way of overcoming the computational challenge. In this paper, we propose a new MIL method based on a MIL back-propagation neural network (MIBP), which is an extension of the standard back-propagation neural network (BPNN) that uses labeled bags of instances as training data. We use parallel computing to speed up the learning process. The proposed method finds a concept point t in the feature space which is close to instances from positive bags and far from instances in negative bags. The description of our method is as follows: First, train MIBP with positive and negative bags. Second, extract t from the trained MIBP. This is achieved by, for each positive bag, presenting all the instances to the trained MIBP and selecting the one with maximal output value. The t is then obtained by averaging all the extracted instances. Finally, a sensitivity analysis of the trained MIBP is performed to obtain feature relevance/weighting information. Parallel computing is performed during the training of the MIBP. We conduct experiments to measure the performance of the obtained t when used for classification purposes and evaluate the parallel computing method. The experimental results on the MUSK data set show that our method has better classification performance and is more computationally efficient than other well-established MIL methods.
Similar content being viewed by others
References
Maron O, Ratan AL (1998) Multiple-instance learning for natural scene classification. In: Proceedings of the 15th international conference on machine learning, Madison, WI, pp 341–349
Maron O, Lozano-Pérez T (1998) A framework for multiple-instance learning. In: Jordan MI, Kearns MJ, Solla SA (eds) Advances in neural information processing systems 10. MIT Press, Cambridge, pp 570–576
Dietterich TG, Lathrop RH, Lozano-Perez T (1997) Solving the multiple-instance problem with axis-parallel rectangles. Artif Intell J 89:31–37
Zhang Q, Goldman SA (2001) EM-DD: An improved multiple-instance learning technique. Adv Neural Inf Process Syst 14:1073–1080
Zhou ZH, Zhang ML (2003) Ensembles of multi-instance learners. In: Proc of the 14th European conf on machine learning. Springer, Berlin, pp 492–501
Wang J, Zucker J-D (2000) Solving the multiple-instance problem: a lazy learning approach. In: Proc 17th int’l conf on machine learning, pp 1119–1125
Zhang Q, Goldman SA, Yu W, Fritts JE (2002) Content-based image retrieval using multiple-instance learning. In: Proc of the19th int conf on machine learning, pp 682–689
Yang C, Lozano-Perez T (2000) Image database retrieval with multiple-instance learning techniques. In: Proc of the 16th int conf on data engineering, pp 233–243
Ramon J, De Raedt L (2000) Multi-instance neural networks. In: Proceedings of ICML2000 workshop on attribute, value and relational learning, pp 53–60
Zhou ZH, Jiang K, Li M (2005) Multi-instance learning based Web mining. Appl Intell 22(2):135–147
Andrews S, Tsochantaridis I, Hofmann T (2003) Support vector machines for multiple-instance learning. Adv Neural Inf Process Syst 15:561–568
Qi G-J, Hua X-S, Rui Y, Mei T, Tang J, Zhang H-J (2007) Concurrent multiple instance learning for image categorization. In: IEEE conference on computer vision and pattern recognition (CVPR 2007), Minneapolis, Minnesota
Chen Y, Wang JZ (2004) Image categorization by learning and reasoning with regions. J Mach Learn Res 5:913–939
Saltelli A, Chan K, Scott EM (2000) Sensitivity Analysis. Wiley, New York
Goh TH, Wong F (1991) Semantic extraction using neural network modeling and sensitivity analysis. In: Proceedings of IEEE international joint conference on neural networks, pp 18–21
Yu L, Wang S, Lai KK (2008) Neural network metalearning for parallel textual information retrieval. Int J Artif Intell 1:A08
Kontár S (2006) Parallel training of neural network for speech recognition. In: Proc 12th international conference on soft computing MENDEL
Andrews S, Hofmann T (2004) Multiple instance learning via disjunctive programming boosting. Adv Neural Inf Process Syst 16:65–72
Gondra I, Xu T (2010) A multiple instance learning based framework for semantic image segmentation. Multimed Tools Appl 48(2):339–365
Chevaleyre Y, Zucker J-D (2001) Solving multiple-instance and multiple-part learning problems with decision trees and decision rules. Application to the mutagenesis Problem. In: Stroulia E, Matwin S (eds) Lecture notes in artificial intelligence, vol 2056. Springer, Berlin, pp 204–214
Ruffo G (2000) Learning single and multiple instance decision tree for computer security applications. PhD dissertation, Department of Computer Science, University of Turin, Torino, Italy
Amar RA, Dooly DR, Goldman SA, Zhang Q (2001) Multiple-instance learning of real-valued data. In: Proceedings of the 18th international conference on machine learning, Williamstown, MA, pp 3–10
Dempster AP, Laird NM, Rubin DB (1997) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc, Ser B 39(1):1–38
Yang J A toolkit for multiple-instance learning and its experiments with information retrieval. http://www.cs.cmu.edu/~juny/MILL/MIL-experiments.htm
Li CH, Gondra I (2010) A novel neural network-based instance selection approach for multiple instance learning. In: 10th IEEE international conference on computer and information technology (CIT’10), United Kingdom, pp 451–456
MUSK data set is available at http://www.cs.sfu.ca/~wangk/ucidata/dataset/musk/
Dahl G, McAvinney A, Newhall T (2008) Parallelizing neural network training for cluster systems. In: Proceedings of the IASTED international conference on parallel and distributed computing and networks, Innsbruck, Austria, pp 220–225
Wu W, Feng G, Li Z, Xu Y (2005) Deterministic convergence of an online gradient method for BP neural networks. IEEE Trans Neural Netw 16(3):533–540
Liming Zhang (1993) Models and applications of artificial neural networks. Fudan University. Shanghai, p 50
Faerber P, Asanovi K (1997) Parallel neural network training on Multi-Spert. In: IEEE 3rd international conference on algorithms and architectures for parallel processing
Acknowledgements
This work was supported by NSERC (Natural Sciences and Engineering Research Council).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Li, C.H., Gondra, I. & Liu, L. An efficient parallel neural network-based multi-instance learning algorithm. J Supercomput 62, 724–740 (2012). https://doi.org/10.1007/s11227-012-0746-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-012-0746-1