Abstract
In this paper we propose a new classification criterion based on maximum a posteriori (MAP) estimation for a binary problem. In our method, we do not estimate the posteriori probability; instead we construct a discriminant function that provides the same result. The criterion consists of the maximization of an expected cost function and a quadratic constraint of the discriminant function with a weighting function. By selecting different weighting functions we show that the least squares regression and the support vector machine can be derived from the criterion. Furthermore, we propose a novel classifier based on the criterion and conduct experiments to demonstrate its advantages.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Arribas, J., Sueiro, J., Lopez, C.: Estimation of Posterior Probabilities with Neural Networks: Application to Microcalcification Detection in Breast Cancer Diagnosis. John Wiley & Sons Inc., Chichester (2007)
Benson, H., Vanderbei, R.: Solving problems with semidefinite and related constraints using interior-point methods for nonlinear programming (2002)
Cawley, G.C., Talbot, N.L.C.: Preventing over-fitting during model selection via Bayesian regularisation of the hyper-parameters. J. Mach. Learn. Res. 8, 841–861 (2007), http://portal.acm.org/citation.cfm?id=1248659.1248690
Huy, T., Takeda, K., Itakura, F.: Maximum a posterior probability and cumulative distribution function equalization methods for speech spectral estimation with application in noise suppression filtering. In: Faundez-Zanuy, M., Janer, L., Esposito, A., Satue-Villar, A., Roure, J., Espinosa-Duro, V. (eds.) NOLISP 2005. LNCS (LNAI), vol. 3817, pp. 328–337. Springer, Heidelberg (2006)
IBM ILOG CPLEX Optimizer, http://www.ibm.com/software/integration/optimization/cplex-optimizer/
Igual, J., Camachoa, A., Bernabeua, P., Vergarab, L.: A maximum a posteriori estimate for the source separation problem with statistical knowledge about the mixing matrix. Science Direct. Pattern Recognition Letters 24, 2519–2523 (2003)
Jaroudi, A., Makhoul, J.: A new error criterion for posterior probability estimation with neural nets. In: International Joint Conference on Neural Networks, pp. 185–192 (1990)
Lobo, M., Vandenberghe, L., Boyd, S., Lebret, H.: Applications of second-order cone programming (November 1998)
Loewenster, D., Berman, H., Hirsh, H.: Maximum a posteriori classification of DNA structure from sequence information. In: Proceedings of PSB 1998 (1998)
Lustig, I.J.: Feasibility issues in a primal-dual interior-point method for linear programming. Mathematical Programming 49, 145–162 (1990)
Mika, S., Ratsch, G., Weston, J., Scholkopf, B., Mullers, K.: Fisher discriminant analysis with kernels. In: Neural Networks for Signal Processing IX, 1999. Proceedings of the 1999 IEEE Signal Processing Society Workshop, pp. 41–48 (August 1999)
Miller, J., Goodman, R., Smyth, P.: Objective functions for probability estimation. In: International Joint Conference on Neural Networks, vol. I, pp. 881–886 (1991)
Ng, T., Grag, H.: A maximum a-posteriori identification criterion for wavelet domain watermarking. International Journal of Wireless and Mobile Computing 3(4), 265–270 (2009)
Nopriadi, Yamashita, Y.: Maximum a posteriori based kernel classifier trained by linear programming. In: Hancock, E.R., Wilson, R.C., Windeatt, T., Ulusoy, I., Escolano, F. (eds.) SSPR&SPR 2010. LNCS, vol. 6218, pp. 493–502. Springer, Heidelberg (2010)
Rätsch, G., Onoda, T., Müller, K.: Soft margins for adaboost. Tech. Rep. NC-TR-1998-021, Royal Holloway College, University of London, UK 42(3), 287–320 (1998)
Rennie, J.D.M.: Maximum-margin logistic regression. (February 2005), http://people.csail.mit.edu/jrennie/writing
Sueiro, J., Arribas, J., Munoz, S., Vidal, A.: Cost functions to estimate a posteriori probabilities in multiclass problems. IEEE Trans. Neural Networks 10, 645–656 (1999)
Xu, Z., Huang, K., Zhu, J., King, I., Lyu, M.: A novel kernel-based maximum a posteriori classification method. Science Direct. Neural Networks 22, 977–987 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yokota, T., Yamashita, Y. (2011). Quadratically Constrained Maximum a Posteriori Estimation for Binary Classifier. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2011. Lecture Notes in Computer Science(), vol 6871. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23199-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-23199-5_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23198-8
Online ISBN: 978-3-642-23199-5
eBook Packages: Computer ScienceComputer Science (R0)