In this paper, we present a novel algorithm for incremental principal component analysis. Based on the largest-eigenvalue-theory, i.e. the eigenvector associated with the largest eigenvalue of a symmetry matrix can be iteratively estimated with any initial value, we propose an iterative algorithm, referred as LET-IPCA, to incrementally update the eigenvectors corresponding to the leading eigenvalues. LET-IPCA is covariance matrix free and seamlessly connects the estimations of the leading eigenvectors by cooperatively preserving the most dominating information, as opposed to the state-of-the-art algorithm CCIPCA, in which the estimation of each eigenvector is independent. The experiments on both the MNIST digits database and the CMU PIE face database show that our proposed algorithm is much superior to CCIPCA in both convergency speed and accuracy.