Flexible Hypersurface Fitting with RBF Kernels

Abstract

This paper gives a method of flexible hypersurface fitting with RBF kernel functions. In order to fit a hypersurface to a given set of points in an Euclidean space, we can apply the hyperplane fitting method to the points mapped to a high dimensional feature space. This fitting is equivalent to a one-dimensional reduction of the feature space by eliminating the linear space spanned by an eigenvector corresponding to the smallest eigenvalue of a variance covariance matrix of data points in the feature space. This dimension reduction is called minor component analysis (MCA), which solves the same eigenvalue problem as kernel principal component analysis and extracts the eigenvector corresponding to the least eigenvalue. In general, feature space is set to an Euclidean space, which is a finite Hilbert space. To consider an MCA for an infinite Hilbert space, a kernel MCA (KMCA), which leads to an MCA in reproducing kernel Hilbert space, should be constructed. However, the representer theorem does not hold for a KMCA since there are infinite numbers of zero-eigenvalues would appear in an MCA for the infinite Hilbert space. Then, the fitting solution is not determined uniquely in the infinite Hilbert space, contrary to there being a unique solution in a finite Hilbert space. This ambiguity of fitting seems disadvantageous because it derives instability in fitting, but it can realize flexible fitting. Based on this flexibility, this paper gives a hypersurface fitting method in the infinite Hilbert space with RBF kernel functions to realize flexible hypersurface fitting. Although some eigenvectors of the matrix defined from kernel function at each sample are considered, we have a candidate of a reasonable solution among the simulation result under a specific situation. It is seen that the flexibility of our method is still effective through simulations.