This paper considers the linear-discriminant analysis (LDA) problem in the undersampled situation, in which the number of features is very large and the number of observations is limited. Sparsity is often incorporated in the solution of LDA to make a well interpretation of the results. However, most of the existing sparse LDA algorithms pursue sparsity by means of the ℓ₁-norm. In this paper, we give elaborate analysis for nonconvex penalties, including the ℓ₀-based and the sorted ℓ₁-based LDA methods. The latter one can be regarded as a bridge between the ℓ₀ and ℓ₁ penalties. These nonconvex penalty-based LDA algorithms are evaluated on the gene expression array and face database, showing high classification accuracy on realworld problems.