Relative localization in GNSS-denied environments is an essential task in advanced driver assistance systems or autonomous driving. In this paper, we present a loosely coupled visual-multi-sensor odometry algorithm for relative localization. Inertial Measurement Unit (IMU), vehicle speed, and steering angle measurements are fused in an Unscented Kalman Filter (UKF) to estimate the vehicle’s pose and velocity as well as the IMU biases. Relative pose estimates between two camera frames are used to update the UKF and further increase precision in the estimated localization. Our system is able to localize a vehicle in real-time from arbitrary states such as an already moving car which is a challenging scenario. We evaluate our visual-multi-sensor algorithm on real-world datasets recorded in inner-city and rural areas and compare it two state-of-the-art Visual-Inertial Odometry (VIO) algorithms. We report a lower relative odometry error in particular at the start of motion estimation with lower computational cost.