Alzheimer's disease (AD) is a sort of nervous system disease, and it may cause amnesia and executive dysfunction etc. AD seriously reduces the quality of people's life, so it is very important to improve the diagnosis accuracy of AD in its prodromal stage, mild cognitive impairment (MCI). In recent years, multi-modal methods had been proven to be effective in prediction of AD and MCI by utilizing the complementary information across different modalities in AD data. In this paper, we propose self-paced sample weighting based low-rank representation (SPLRR) to explore the latent correlation across different modalities. By imposing rank minimization on different modalities regression coefficients, we can capture the intrinsic structure among modalities. Meanwhile, we introduce self-paced learning to allot the corresponding weight to samples based on the contribution of each sample to the label in the current modality. Experiments on the Alzheimer's disease Neuroimaging Initiative (ADNI) database show that the SPLRR model obtains the better classification performance than the state-of-the-art methods.