This paper presents a basic color image discriminant (CID) model and its general version for color image recognition. The CID models seek to unify the color image representation and recognition tasks into one framework. The proposed models, therefore, involve two sets of variables: a set of color component combination coefficients for color image representation and one or multiple projection basis vectors for color image discrimination. An iterative basic CID algorithm and its general version are designed to find the optimal solution of the proposed models. The general CID (GCID) algorithm is further extended to generate three color components (such as the three color components of the RGB color images) for further improvement of the recognition performance. Experiments using the face recognition grand challenge (FRGC) database and the biometric experimentation environment (BEE) system show the effectiveness of the proposed models and algorithms. In particular, for the most challenging FRGC version 2 Experiment 4, which contains 12 776 training images, 16 028 controlled target images, and 8014 uncontrolled query images, the proposed method achieves the face verification rate (ROC III) of 78.26% at the false accept rate (FAR) of 0.1%.