Abstract
In this paper, we develop a novel approach to perform kernel parameter selection for Kernel Fisher discriminant analysis (KFDA) based on the viewpoint that optimal kernel parameter is associated with the maximum linear separability of samples in the feature space. This makes our approach for selecting kernel parameter of KFDA completely comply with the essence of KFDA. Indeed, this paper is the first paper to determine the kernel parameter of KFDA using a search algorithm. Our approach proposed in this paper firstly constructs an objective function whose minimum is exactly equivalent to the maximum of linear separability. Then the approach exploits a minimum search algorithm to determine the optimal kernel parameter of KFDA. The convergence properties of the search algorithm allow our approach to work well. The algorithm is also simple and not computationally complex. Experimental results illustrate the effectiveness of our approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Mika, S., Rätsch, G., Weston, J., et al.: Fisher Discriminant Analysis with Kernels. In: Hu, Y H, Larsen, J., Wilson, E., Douglas, S. (eds.) Neural Networks for Signal Processing IX, pp. 41–48. IEEE Computer Society Press, Los Alamitos (1999)
Muller, K.-R., Mika, S., Ratsch, G., Tsuda, K., Scholkopf, B.: An Introduction to Kernel-based Learning Algorithms. IEEE Trans. On Neural Network 12(1), 181–201 (2001)
Billings, S.A., Lee, K.L.: Nonlinear Fisher Discriminant Analysis Using a Minimum Square Error Cost Function and the Orthogonal Least Squares Algorithm. Neural Networks 15(1), 263–270 (2002)
Yang, J., Jin, Z.H., Yang, J.Y., Zhang, D., Frangi, A.F.: Essence of Kernel Fisher Discriminant: KPCA plus LDA. Pattern Recognition 37(10), 2097–2100 (2004)
Xu, Y., Yang, J.-Y., Lu, J., Yu, D.J.: An Efficient Renovation on Kernel Fisher Discriminant Analysis and Face Recognition Experiments. Pattern Recognition 37, 2091–2094 (2004)
Xu, Y., Yang, J.-Y., Yang, J.: A Reformative Kernel Fisher Discriminant Analysis. Pattern Recognition 37, 1299–1302 (2004)
Xu, Y., Zhang, D., Jin, Z., Li, M., Yang, J.-Y.: A Fast Kernel-based Nonlinear Discriminant Analysis for Multi-class Problems. Pattern Recognition 39(6), 1026–1033 (2006)
Duda, R., Hart, P.: Pattern Classification and Scene Analysis. Wiley, New York (1973)
Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenface vs. Fisherface: Recognition Using Class Specific Linear Projection. IEEE Trans. Pattern Anal. And Mach. Intelligence 19(10), 711–720 (1997)
Xu, Y., Yang, J.Y., Jin, Z.: Theory Analysis on FSLDA and ULDA. Pattern Recognition 36(12), 3031–3033 (2003)
Xu, Y., Yang, J.-Y., Jin, Z.: A Novel Method for Fisher Discriminant Analysis. Pattern Recognition 37(2), 381–384 (2004)
Centeno, T.P., Lawrence, N.D.: Optimising Kernel Parameters and Regularisation Coefficients for Non-linear Discriminant Analysis. Journal of Machine Learning Research 7, 455–491 (2006)
Shawkat, A., Kate, A.S.: Automatic Parameter Selection for Polynomial Kernel. In: Proceedings of the IEEE International Conference on Information Reuse and Integration, USA, pp. 243–249 (2003)
Volker, Roth,: Outlier Detection with One-class Kernel Fisher Discriminants. In: Saul, L.K., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems 17, pp. 1169–1176. MIT Press, Cambridge, MA (2005)
Schittkowski, K.: Optimal Parameter Selection in Support Vector Machines. Journal of Industrial and Management Optimization 1(4), 465–476 (2005)
Carl, Staelin.: Parameter Selection for Support Vector Machines, Technical report, HP Laboratories Israel (2003)
McKinnon, K.I.M.: Convergence of the Nelder-Mead to a No Stationary Point[J]. SIAM Journal Optimization 9, 148–158 (1998)
Lagarias, J.G., Reeds, J.A., Wright, M.H., et al.: Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions. SIAM Journal of Optimization 9(1), 112–147 (1998)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Xu, Y., Liu, C., Zhang, C. (2007). Determine the Kernel Parameter of KFDA Using a Minimum Search Algorithm. In: Huang, DS., Heutte, L., Loog, M. (eds) Advanced Intelligent Computing Theories and Applications. With Aspects of Artificial Intelligence. ICIC 2007. Lecture Notes in Computer Science(), vol 4682. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74205-0_46
Download citation
DOI: https://doi.org/10.1007/978-3-540-74205-0_46
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74201-2
Online ISBN: 978-3-540-74205-0
eBook Packages: Computer ScienceComputer Science (R0)