Facial expressions are a spontaneous way of perceiving emotions, which can provide information related to the cognitive state of a person. Facial expression recognition of the elderly is an important aid to better care them, according to their state of mind, although it can be a difficult task because their expressions might not be as easily perceived as those from younger persons. We proposed a model to classify the facial expressions of the elderly, presenting the differences between facial expression recognition in the elder and in other age group, as well as methods to surpass these difficulties. Viola Jones with Haar Features was used to extract the faces and Gabor Filter to extract the facial characteristics. These characteristics are classified using a Multiclass Support Vector Machine. We got an accuracy of 90.32%, 84.61%and 66.6%, when detecting the neutral state, happiness and sadness respectively in the elderly. In the other age group, we got an accuracy of 95.24%, 88.57%, and 80%, while detecting the neutral, happiness, and sadness states and concluded that aging influences negatively the facial expressions recognition tasks.