Real-time contextual and metaphorical affect sensing from open-ended multithreaded dialogue is challenging but essential for the building of effective intelligent user interfaces. In this paper, we focus on context-based affect detection using emotion modeling in personal and social communication context. It mainly focuses on emotion evolvement and prediction in such communication contexts. The work also accompanies research on metaphorical affect interpretation. It mainly targets on the interpretation of metaphorical expressions with semantic preference violations. Evaluation results indicate that the new developments on metaphorical and contextual affect sensing enabled an affect inspired AI agent to outperform its previous version in affect sensing tasks.