A fuzzy logic based multi-modal gesture recognition method is proposed, where both camera images and hand motion data (given by a 3D acceleration sensor put on human wrists) are used to notify the emotion of humans to robots in real time. To demonstrate the validity, it is applied to a part of the home party scenario enjoyed by five eye robots and four human participants, where 8 types of human emotional-gestures are successfully recognized by robots. The proposed method aims to realize the casual communication between robots and humans in the mascot robot system that has been developed by authors' group.