Emotions are grounded in contextual experience. While natural language processing tools typically look at textual content to find clues pertaining to an author's emotional state, factors occurring throughout the day, such as weather or news exposure, may prime one toward a particular emotional response. In this paper, we explore five types of external factors and through extensive analyses show their impact and correlation with a user's emotional state. Ultimately, we show that when combining all extrinsic features, we are able to predict with an accuracy of 67% the emotional state of a user.