In this study, we propose a method for estimating a user's fatigue resulting from mental load, such as office work, based on changes of luminance in facial images. Because these changes are influenced by vital signs such as heart rate and blood pressure, we consider that the level of fatigue is predictable with high accuracy by combining the features in the changes of luminance in the facial area. We thus detected 13 facial parts and estimated subjects' fatigue using feature values based on luminance changes in each facial part. The experimental results revealed that fatigue states could be estimated with an accuracy of 92%.