Hotspot detection and classification for a fleet of vehicles is usually performed based on GPS data sampled from the vehicles. In this paper, we explore how the integration of satellite images can improve GPS-based hotspot classification. We propose a system composed of a deep Convolutional Neural Network (CNN) for image classification and a Random Forest classifier that combines GPS-based features with the CNN output for hotspot classification. We introduce also a novel metric for scoring place detection and classification systems, able to account for both detection and classification errors. The new metric is used to assess experimentally the effectiveness of our system in combining the two sources of information.