Facial expressions are the most informative activities on the human face. Facial movements can be analyzed in order to recognize facial expressions. These movements are extracted using the facial geometry or texture. The paper addresses the facial expression recognition based on Local Binary Patterns (LBP) extracted from the texture information. The LBP operator and its extensions are applied to different color models which are gray-scale, RGB, oRGB, YCbCr and HSV. Frontal face images among six basic facial expressions which are anger, disgust, fear, happiness, sadness and surprise are considered. Support Vector Machine (SVM) is employed as the classifier. The simulation results on BU-3DFE database shows that LBP features of different color models affects recognition performance significantly.