Daily exercise has played a significant role for people in staying healthy; however, some people cannot do moderate exercise continuously. In this paper, we proposed an exercise recognition system using facial image information to make daily exercise management convenient. The proposed system gets facial image information from a built-in camera on a mobile device and can recognize nine exercises by a support vector machine's classifier. When a user faces the camera during the exercise, the system gets time-series data consisting of 62 features on the face. Via leave-one-subject-out cross-validation, the average classification accuracy reached up to 88.2%.