Personality trait recognition holds significant importance in psychology and affective computing, due to its role in understanding human behavior and emotions. However, existing approaches often treat personality and emotion as separate tasks, overlooking the potential of leveraging emotion-related cues in the data for personality modeling. To address this limitation, we propose a multimodal multitask learning framework that utilizes emotional cues to enhance personality trait recognition. Experimental results demonstrate that our proposed framework outperforms state-of-the-art methods on the First Impression benchmark in text-audio bi-modality, showcasing the effectiveness of harnessing emotional cues for personality modeling.