Toll Vehicle Classification is an important task. Indeed, it has many uses in traffic management and toll collection systems. In this paper, Vinci Autoroutes group Networks (the biggest French Highways concession) are considered, where every year, millions of vehicles are classified in realtime. Then, a small decrease in classification performance can have serious economic losses. Therefore, the accuracy and the time complexity become critical for the toll collection system. The current classification algorithm uses the scene features' to detect vehicles classes. However, it requires a large labeled datasets, and has a limitations when multiple vehicles are in the scene. Herein, we propose a novel context-aware vehicle classification method that takes profit from the semantic spatial relationship of the objects. The experiments show that our method is performing as accurately as the existing model with significantly lower labeled datasets (74 times smaller). Moreover, the obtained accuracy of the proposed method is 99.97% compared to 99.79% achieved by the current method when using the same training set.