In the Multi-Object Tracking (MOT) scenario, on top of existing image-based approaches, 3D information can also be essential to improving tracking performance. This paper introduces a way to leverage 3D information, and establishes a Bayesian model that can easily be integrated with 2D and 3D features. Furthermore, to remedy fragmented trajectories due to detection failures in the tracking-by-detection framework, we propose a novel detection-by-tracking method that prevents trajectory interruption. We evaluate our online tracking approach on the KITTI tracking benchmark, and show that by using our proposed 2D and 3D features along with detection-by-tracking, we are able to achieve state-of-the-art results on both cars and pedestrians with a runtime as fast as 16-50 FPS.