Driving risk classification is usually used for evaluating and reducing traffic accidents. It is of great significance to improve urban traffic problems, such as traffic jams and road accidents. Most related works use different representation methods with the data from the advanced driving assistance system and then use statistical approaches or machine learning to analyze the driving risk. However, because of the unbalance of positive and negative samples, the performance is usually not satisfactory. In this article, we propose a driving risk classification method via unbalanced time series samples. It first employs MeanShift with automatic bandwidth to cluster samples and expand its volume according to similarity. Then, it adopts a regional division module to simulate the outside environment and extracts state transition features by the Markov feature module to get more detailed information. By combining the stacking classification module with three convolutional neural networks, the performance is further improved. Finally, a city transfer module is designed with the drivers’ inherent attributes and sample weights to enhance the generalization of the model. The experimental results verify the effectiveness and generality of the classification model on driving risk. It can also be transferred to time series analysis in other fields.