In this paper, we introduce a probabilistic outlier model which is seamlessly integrated into machine learning frameworks (e.g., boosting and deep neural network) to accurately identify outliers in training samples. With two Gamma mixtures, the proposed model can estimate the distribution of inlier and outlier samples respectively and generates their posterior probabilities. The iterative removal method gradually reduces the outliers and boosts the performance of classifier by correcting the bias and variance caused by outliers. Experimental results on INRIA pedestrian dataset and MNIST handwritten-digit dataset demonstrate that the proposed outlier removal method achieves superior detection performance.