A main issue in machine learning theoretical research is to analyze the generalization performance of learning algorithms. The previous results describing the generalization performance of learning algorithms are based on either complexity of hypothesis space or stability property of learning algorithms. In this paper we go far beyond these classical frameworks by establishing the first generalization bounds of learning algorithms in terms of uniform stability and the covering number of function space for regularized least squares regression and SVM regression. To have a better understanding the results obtained in this paper, we compare the obtained generalization bounds with previously known results.