Affective computing strives to recognize a person's affective state (e.g., emotion, mood) based on what can be observed. However, electroencephalogram (EEG) and video technologies have not been widely adopted for daily life affect monitoring due to obtrusiveness and privacy concerns. Although the connection between affective states and biophysical data collected with unobtrusive wrist-worn wearables in lab settings has been established successfully, the number of studies for affect recognition in the wild is still limited, and current methods have not yet provided the accuracy necessary for robust applications. In this study, we propose a smart mood-aware emotion detection method. The proposed emotion recognition method extracts the most distinctive features from the physiological data and adds the output of the automated mood detection system as an input to improve performance. The effect of the division of self-report scales into emotion classes is also investigated. The proposed system obtained higher emotion recognition accuracies than most in-the-wild studies when we tested it with the daily life data collected from 14 participants for one week.