This paper solves the problem of nonrigid deformation based on labeled and unlabeled feature points between two images or surfaces. The principle is to obtain the labeled deformation control point pairs on the correspondence images and the unlabeled feature control points in the source image, and estimate the deformation function between the deformation control point pairs for each pixel or voxel. To achieve a deformation function with more structure information preserved, we consider the nonrigid deformation function estimating as a semi-supervised learning problem, which exploits the manifold regularization to preserve the intrinsic geometry information of the points contained in the object. Moreover, we choose to define the nonrigid deformation function in a reproducing kernel Hilbert space and a closed-form solution can be derived. To reduce the computation complexity, we also adopt a sparse approximation to realize a fast implementation. The method can be applied to 2D image deformation and 3D surface deformation, where extensive experiments on the data show that our methods are superior to state-of-the-art methods.