The recognition of the surrounding context from video recordings offers interesting possibilities for context awareness of video capable mobile devices. Multimodal analysis provides means for improved recognition accuracy and robustness in different use conditions. We present a mul-timodal video context recognition system fusing audio and video cues with support vector machines (SVM) and simple rules with genetic algorithm (GA) optimized weights. Mul-timodal recognition is shown to outperform the unimodal approaches in recognizing between 21 everyday contexts. The highest correct classification rate of 0.844 is achieved with SVM-based fusion.