Abstract
Measuring vital signs is usually done by sensors attached to the human body. In clinical cases, the patients are being monitored by contacted devices that alert the medical staff when the patient situation becomes unstable. However, in non-clinical cases, there are situations when vital signs measurements can be used to prevent dangerous situations, like the driver monitoring task. Monitoring the driver’s vital signs has become popular for the last few years due to its significant role in preventing accidents. However, this task is challenging since contact sensors are inconvenient for the driver and can’t be used in this case. In the paper, we propose a contactless camera-based approach to calculate the respiratory rate of drivers. We suggest using the Openpose human pose estimation model to estimate the position of the chest keypoint, followed by an optical flow-based neural network (SelFlow) to calculate the keypoint displacement. After that, we clean this signal using filtering and detrending as well as count the number of peaks/troughs in a time window of one minute. We evaluated our approach in real driving conditions and it works precisely when the vehicle is stopped or moves with a speed below 3 km/h. When the vehicle moves there are a lot of additional driver motions that significantly reduce the accuracy of the respiratory rate detection. We also compared our results with the ROI approach proposed by researchers from Microsoft and concluded that the proposed approach is more accurate in vehicle cabins.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kashevnik, A., Othman, W., Ryabchikov, I., Shilov, N.: Estimation of motion and respiratory characteristics during the meditation practice based on video analysis. Sensors. 21(11), 3771 (2021). https://www.mdpi.com/1424-8220/21/11/3771
Liu, X., Fromm, J., Patel, S., McDuff, D.: Multi-task temporal shift attention networks for on-device contactless vitals measurement. arXiv preprint arXiv:2006.03790 (2020)
Cao, Z., Hidalgo Martinez, G., Simon, T., Wei, S., Sheikh, Y.A.: Openpose: realtime multi-person 2d pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. (2019)
Liu, P., Lyu, M.R., King, I., Xu, J.: Selflow: self-supervised learning of optical flow. In: CVPR (2019)
Magdalena Nowara, E., Marks, T.K., Mansour, H., Veeraraghavan, A.: SparsePPG: towards driver monitoring using camera-based vital signs estimation in near-infrared. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1272–1281 (2018)
Blöcher, T., Schneider, J., Schinle, M., Stork, W.: An online PPGI approach for camera based heart rate monitoring using beat-to-beat detection. In: 2017 IEEE Sensors Applications Symposium (SAS), pp. 1–6 (2017)
Zhang, Q., Wu, Q., Zhou, Y., Wu, X., Ou, Y., Zhou, H.: Webcam-based, non-contact, real-time measurement for the physiological parameters of drivers. Measurement 100, 01 (2017)
Yang, F., et al.: Non-contact driver respiration rate detection technology based on suppression of multipath interference with directional antenna. Information. 11(4), 192 (2020). https://www.mdpi.com/2078-2489/11/4/192
Fiedler, M.-A., Rapczyński, M., Al-Hamadi, A.: Fusion-based approach for respiratory rate recognition from facial video images. IEEE Access. 8, 130 036–130 047 (2020)
Scebba, G., Da Poian, G., Karlen, W.: Multispectral video fusion for non-contact monitoring of respiratory rate and apnea. IEEE Trans. Biomed. Eng. 68(1), 350–359 (2021)
Sun, X., Xiao, B., Liang, S., Wei, Y.: Integral Human Pose Regression. CoRR 2017, abs/1711.08229. http://xxx.lanl.gov/abs/1711.08229 [1711.08229]
He, K., Girshick, R., Dollár, P.: Rethinking imageNet pre-training. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4918–4927 (2019)
Makowski, D., et al.: Neurokit2: a python toolbox for neurophysiological signal processing. Behav. Res. Methods. 53, 1–8 (2021)
Khodadad, D., et al.: Optimized breath detection algorithm in electrical impedance tomography. Physiol. Measur. 39(9), 094001 (2018)
Kashevnik, A., Lashkov, I., Gurtov, A.: Methodology and mobile application for driver behavior analysis and accident prevention. IEEE Trans. Intell. Transp. Syst. 21, 2427–2436 (2020). https://doi.org/10.1109/TITS.2019.2918328
Kashevnik, A., Lashkov, I., Ponomarev, A., Teslya, N., Gurtov, A.: Cloud-based driver monitoring system using a smartphone. IEEE Sens. J. 20, 6701–6715 (2020). https://doi.org/10.1109/JSEN.2020.2975382
Kashevnik, A., Ali, A., Lashkov, I., Shilov, N.: Seat belt fastness detection based on image analysis from vehicle in-cabin camera. In: 2020 26th Conference of Open Innovations Association (FRUCT), pp. 143–150 (2020)
Kashevnik, A., Ali, A., Lashkov, I., Zubok, D.: Human Head Angle Detection Based on Image Analysis, pp. 233–242 (2020)
Acknowledgment
The research has been supported by the Russian Science Foundation project # 18-71-10065. Experiments and evaluation (Sect. 5) are partially due to Russian State Research # FFZF-2022-0005.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Othman, W., Kashevnik, A., Ryabchikov, I., Shilov, N. (2023). Contactless Camera-Based Approach for Driver Respiratory Rate Estimation in Vehicle Cabin. In: Arai, K. (eds) Intelligent Systems and Applications. IntelliSys 2022. Lecture Notes in Networks and Systems, vol 543. Springer, Cham. https://doi.org/10.1007/978-3-031-16078-3_29
Download citation
DOI: https://doi.org/10.1007/978-3-031-16078-3_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16077-6
Online ISBN: 978-3-031-16078-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)