Vehicle surveillance in complex dark traffic scenes has been a key research topic, as the background is dramatically altered due to the reflections from headlights on normal, snowy, and rainy roads. Under dark conditions, a vehicle's headlights and rear lights are used for foreground extraction. The presented algorithm provides several steps, including the detection, pairing, and tracking of headlights and rear lights. First, the headlights are automatically extracted by a novel approach called azimuthally blur, which uses the exponentially attenuating nature of reflected light. This approach is robust on highly reflective scenes because it makes the headlights orthogonal to the reflections. The headlights are then paired by partitioning the image into subgroups such that in each group, the headlights remain equidistant. The optimized tracker based on the maximum a posteriori (MAP) probability estimator is employed for further analysis such as speed estimation. This whole scheme is computationally inexpensive and can be deployed in application-specific integrated circuits. The proposed approach has outperformed state-of-the-art methods in challenging unlit traffic scenes.