Abstract
We consider the problem of learning classifiers from samples which have additional features that are absent due to noise or corruption of measurement. The common approach for handling missing features in discriminative models is first to complete their unknown values, and then a standard classification algorithm is employed over the completed data. In this paper, an algorithm which aims to maximize the margin of each sample in its own relevant subspace is proposed. We show how incomplete data can be classified directly without completing any missing features in a large-margin learning framework. Moreover, according to the theory of optimal kernel function, we proposed an optimal kernel function which is a convex composition of a set of linear kernel function to measure the similarity between additional features of each two samples. Based on the geometric interpretation of the margin, we formulate an objective function to maximize the margin of each sample in its own relevant subspace. In this formulation, we make use of the structural parameters trained from existing features and optimize the structural parameters trained from additional features only. A two-step iterative procedure for solving the objective function is proposed. By avoiding the pre-processing phase in which the data is completed, our algorithm could offer considerable computational saving. We demonstrate our results on a large number of standard benchmarks from UCI and the results show that our algorithm can achieve better or comparable classification accuracy compared to the existing algorithms.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Little, R.J.A., Rubin, D.B.: Statistical Analysis with Missing Data. Wiley, New York (1987)
Roth, P.: Missing data: A conceptual review for applied psychologists. Personnel Psychology 47(3), 537–560 (1994)
Ghahramani, Z., Jordan, M.I.: Supervised learning from incomplete data via an EM approach. In: Cowan, J.D., Tesauro, G., Alspector, J. (eds.) Advances in Neural Information Processing Systems, vol. 6, pp. 120–127. Morgan Kaufmann Publishers, Inc., San Francisco (1994)
Kapoor, A.: Learning Discriminative Models with Incomplete Data. PhD thesis, MIT Media Lab, Feburary (2006)
Chechik, G., Heitz, G., Elidan, G., Abbeel, P., Koller, D.: Max-margin Classification of Data with Absent Features. Journal of Machine Learning Research 9, 1–21 (2008)
Ong, C.S., Smola, A.J., Williamson, R.C.: Learning the Kernel with Hyperkernels. Journal of Machine Learning Research 6, 1043–1071 (2005)
Crammer, K., Keshet, J., Singer, Y.: Kernel design using boosting. In: Advances in Neural Information Processing Systems, vol. 15, pp. 537–544 (2002)
Ong, C.S., Smola, A.J.: Machine learning using hyperkernels. In: Proceedings of the International Conference on Machine Learning, pp. 568–575 (2003)
Liu, X., Zhang, G.: An Incremental Feature Learning Algorithm Based on Least Square Support Vector Machine. In: Preparata, F.P., Wu, X., Yin, J. (eds.) FAW 2008. LNCS, vol. 5059, pp. 330–338. Springer, Heidelberg (2008)
Kazushi, I., Takemasa, Y.: Incremental support vector machines and their geometrical analyses. Neuro-computing, 2528–2533 (2007)
Wang, L., Yang, C., Feng, J.: On learning with dissimilarity functions. In: Proceedings of the 24th international conference on machine learning, pp. 991–998 (2007)
Dick, U., Haider, P., Scheffer, T.: Learning from Incomplete Data with Infinite Imputations. In: Proceedings of the 25th international conference on machine learning, pp. 232–239 (2008)
Williams, D., Carin, L.: Analytical kernel ma-trix completion with incomplete multi-view data. In: Proceedings of the ICML Workshop on Learning With Multiple Views (2005)
Williams, D., Liao, X., Xue, Y., Carin, L.: Incomplete-data classification using logistic regression. In: Proceedings of the 22nd International Conference on Machine learning (2005)
Chen, Z., Li, J.: Least Square Support Vector Machine. In: Proceedings of the 2006 International Conference on Computational Intelligence and Security, pp. 176–179 (2006)
Asuncion, A., Newman, D.J.: UCI machine learning repository [OL] (2009), http://archive.ics.uci.edu/ml/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, X., Yin, J., Zhu, E., Zhang, G., Zhan, Y., Li, M. (2009). A Large Margin Classifier with Additional Features. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2009. Lecture Notes in Computer Science(), vol 5632. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03070-3_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-03070-3_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03069-7
Online ISBN: 978-3-642-03070-3
eBook Packages: Computer ScienceComputer Science (R0)