Although modern cars are equipped with advanced technologies to be faster, more comfortable and safer, one essential piece of the driving system, the driver, is missing in the picture. Among the physiological measures used for wellness purposes, heart rate variability has been shown to be directly associated with mental and physical status, and is easy to measure. In this paper, to maintain the driver's comfort and enhance the driving safety, we propose a non-contact, video-based approach to continuously monitor the driver's heart rate variability under real-world driving circumstances. Previously, several methods were proposed for similar goals under laboratory conditions, where simple face detectors and independent component analysis approach were used, and they may fail in both image understanding and signal processing steps under real-world circumstances in driving. Here we propose using advanced facial landmark and pose estimation, and independent vector analysis to extract heart rate variability. Our preliminary experimental results demonstrated that the proposed approach works better than the previous state-of-arts.