Most of the traffic accidents are caused by human related factors. One of the most important human related factor in terms of traffic accident risk is aggressive driving behavior. Although driver aggressiveness is related with psychological reasons, it can be detected by observation of driving behavior using different sensing devices. This work presents a driver aggressiveness detection method exploiting the visual data obtained by on vehicle camera. The proposed method uses this visual data in order to extract features such as lane departure rate and possible collision time by detection of road lines and vehicles on the road. These features are used to train a classifier which decides on whether an aggressive behavior is performed in related driving session. According to conducted tests the proposed method is observed to achieve 90% correct detection rate of aggressive driving behavior.