Music listening is a very personal and situational behavior for modern people who always carry smartphones in everyday life. Therefore, contextual information, such as user's current activity and mood state could be used to greatly improve music recommendations. In this paper, we develop a smartphone-based mobile system that includes two core modules for recognizing human activities and then accordingly recommending music. In the proposed method, a deep residual bidirectional gated recurrent neural network is applied to obtain high activity recognition accuracy from accelerometer signals on the smartphone. In order to improve the performance of tempo-oriented music classification, an ensemble of dynamic classification using a long-term modulation spectrum and sequence classification using a short-term spectrogram is used. Music recommendation is performed using the relationship between the recognized human activities and the music files indexed by tempo-oriented music classification that reflects user preference models in order to achieve high user satisfaction. The results of comprehensive experiments on real data confirm the accuracy of the proposed activity-aware music recommendation framework.