A feature extraction method for Facial Expression Recognition Systems is proposed based on Spectral Supervised Canonical Correlation Analysis. For proper classification of expression it has been trained with Rethinking recurrent neural network. The Cohn Kanade Extensive and JAFFE databases are used in this paper. The images have been preprocessed using image normalization and then contrast limited adaptive histogram equalization to remove the illumination variance and noises. After down-sampling, the dimensions with factor data is provided to Spectral Supervised Canonical Correlation Analysis (SSCCA) which constructs affinity matrix that incorporates both the local structure and class information of the data points provided. Spectral feature is used for extracting features with more discriminative details, and revealing the nonlinear manifold structure of the data. SSCCA can effectively utilize the local structural information to discover low frequency coefficients more precisely. The method yields to more accurate and effective extraction compared to other methods. Data is provided to Rethinking recurrent neural network for training purpose. Meanwhile, the proposed method is more robust and effective compared to other methods in this field.