Accurate Remote PPG Waveform Recovery from Video Using a Multi-task Learning Temporal Model

Abstract

Remote photoplethysmography (rPPG) offers a convenient, non-contact method for extracting cardiac-related signals from video. Despite its significant potential for comprehensive cardiac health monitoring, existing methods are limited to extracting heart rate because they can only recover heart-rate-correlated periodic patterns rather than the complete and precise PPG waveform needed for thorough biometric analysis. To address this issue, we designed a multi-loss model aimed at accurately restoring rPPG waveforms, focusing on capturing critical fiducial points and pulse contours. Our model employs a multi-task learning architecture that integrates primary rPPG signal reconstruction mean squared error (MSE) loss, peak loss, trough loss, and signal-to-noise ratio (SNR) loss to enhance signal recovery. Additionally, we incorporated Temporal Shift Modules (TSM) and Long Short-Term Memory (LSTM) networks to capture both short-term and long-term temporal dependencies, effectively handling low-quality or cross-dataset training data. The experimental results show that our model significantly improves rPPG signal restoration on the PURE and UBFC-rPPG datasets, outperforming two representative models, DeepPhys and TS-CAN, by reducing systolic peak and foot/onset estimation errors by over 30%, accurately capturing diastolic peaks and dicrotic notches, and achieving a DTW distance of 6.54, indicating enhanced waveform contour recovery.