With the development of society, technological progress, and new needs, autonomous driving has become a trendy topic in smart cities. Due to technological limitations, autonomous driving is used mainly in limited and low-speed scenarios such as logistics and distribution, shared transport, unmanned retail, and other systems. On the other hand, the natural driving environment is complicated and unpredictable. As a result, to achieve all-weather and robust autonomous driving, the vehicle must precisely understand its environment. The self-driving cars are outfitted with a plethora of sensors to detect their environment. In order to provide researchers with a better understanding of the technical solutions for multi-sensor fusion, this paper provides a comprehensive review of multi-sensor fusion 3D object detection networks according to the fusion location, focusing on the most popular LiDAR and cameras currently in use. Furthermore, we describe the popular datasets and assessment metrics used for 3D object detection, as well as the problems and future prospects of 3D object detection in autonomous driving.