We develop a machine learning-based pedestrian detection and alert system that can operate both during the day and at night using a visual camera, an infrared camera, and a radar sensor. The visible camera is used to detect pedestrians during the daytime while the infrared camera during the nighttime. Whereas the radar sensor is utilized to detect the presence of pedestrians including their range and directions of motion. We have developed and conducted actual experimentation of the system in a vehicle. We achieved an average accuracy of 98% based on our proposed multi-sensor data analysis using a deep learning algorithm that classifies a pedestrian’s presence during the day and at night and alerts the driver in a real-time monitoring system.