Human emotions and behaviors are reciprocal components that shape each other in everyday life. While the past research on each element has made use of various physiological sensors in many ways, their interactive relationship in the context of daily life has not yet been explored. In this work, we present a wearable affective life-log system (ALIS) that is robust as well as easy to use in daily life to accurately detect emotional changes and determine the cause-and-effect relationship between emotions and emotional situations in users’ lives. The proposed system records how a user feels in certain situations during long-term activities using physiological sensors. Based on the long-term monitoring, the system analyzes how the contexts of the user’s life affect his/her emotional changes and builds causal structures between emotions and observable behaviors in daily situations. Furthermore, we demonstrate that the proposed system enables us to build causal structures to find individual sources of mental relief suited to negative situations in school life.