The study of healthy brain development helps to better understand both brain transformation and connectivity patterns, which happen during childhood to adulthood. This study presents a sparse machine learning solution across whole-brain functional connectivity measures of three datasets, derived from resting state functional magnetic resonance imaging (rs-fMRI) and two task fMRI data including a working memory n-back task (nb-fMRI) and an emotion identification task (em-fMRI). The fMRI data are collected from the Philadelphia Neurodevelopmental Cohort (PNC) for the prediction of brain age in adolescents. Due to extremely large variable-to-instance ratio of PNC data, a high-dimensional matrix with several irrelevant and highly correlated features is generated, and hence a sparse learning approach is necessary to extract effective features from fMRI data. We propose a sparse learner based on the residual errors along the estimation of an inverse problem for extreme learning machine (ELM). Our proposed method is able to overcome the overlearning problem by pruning several redundant features and their corresponding output weights. The proposed multimodal sparse ELM classifier based on residual errors is highly competitive in terms of classification accuracy compared to its counterparts such as conventional ELM, and sparse Bayesian learning ELM.