Abstract
Recent advances in-vehicle monitoring of drivers’ statistics attempt to leverage Machine-Learning (ML)-based techniques for prediction, credited to its high accuracy and less cost for deploying redundant sensors/cameras inside the vehicle. However, existing approaches for in-vehicle ML-based predictors heavily rely on the raw driver data for processing, which mcan push the burdens on multiple components, ranging from on-vehicle devices to Internet-of-Vehicles (e.g., on-vehicle computation and storage units). To obtain a better balance between the performance and cost, we propose FIne-Grained CONstructs (FIGCONs) of collected drivers’ statistics, to further exploit its applicability. We do so by integrating it with state-of-the-art ML-based approaches to predict in-vehicle statistics. The experimental results deliver three key findings, which showcase the benefits of FIGCONs and the potentials for practical deployments of in-vehicle ML-based predictors. We hope our work can stimulate more future works and potential practices in similar manners.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bazarevsky, V., Kartynnik, Y., Vakunov, A., Raveendran, K., Grundmann, M.: BlazeFace: sub-millisecond neural face detection on mobile GPUs. CoRR, abs/1907.05047 (2019)
Blockchain-based security attack resilience schemes for autonomous vehicles in industry 4.0: a systematic review. Comput. Electr. Eng. 86, 106717 (2020)
Bradski, G.R., Kaehler, A.: Learning OpenCV - Computer Vision with the OpenCV Library: Software that Sees. O’Reilly, Sebastopol (2008)
Duan, Y., Liu, J., Ming, X., Jin, W., Song, Z., Peng, X.: Characterizing and optimizing differentially-private techniques for high-utility, privacy-preserving internet-of-vehicles. In: International Conference on Human-Computer Interaction (2023)
Duan, Y., Liu, J., Jin, W., Peng, X.: Characterizing differentially-private techniques in the era of internet-of-vehicles (2022)
Google. Augmented faces developer guide for AR foundation. https://developers.google.com/ar/develop/unity-arf/augmented-faces/developer-guide?hl=en
Haouij, N.E., Poggi, J.-M., Sevestre-Ghalila, S., Ghozi, R., Jaïdane, M.: AffectiveROAD system and database to assess driver’s attention. In: Proceedings of the 33rd Annual ACM Symposium on Applied Computing, pp 800–803 (2018)
Healey, J.A., Picard, R.W.: Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans. Intell. Transp. Syst. 6(2), 156–166 (2005)
Huang, Z., et al.: Face2Multi-modal: in-vehicle multi-modal predictors via facial expressions. In: Adjunct Proceedings of the 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2020, Virtual Event, Washington, DC, USA, 21–22 September 2020, pp. 30–33. ACM (2020)
Primeo Inc., Hardware acceleration: Edgeboost nodes (2021)
Jain, A., Koppula, H.S., Soh, S., Raghavan, B., Singh, A., Saxena, A.: Brain4Cars: car that knows before you do via sensory-fusion deep learning architecture. arXiv preprint arXiv:1601.00740 (2016)
Jin, W., Duan, Y., Liu, J., Huang, S., Xiong, Z., Peng, X.: BROOK dataset: a playground for exploiting data-driven techniques in human-vehicle interactive designs. Technical report-Feb-01 at User-Centric Computing Group, University of Nottingham Ningbo China (2022)
Jin, W., Ming, X., Song, Z., Xiong, Z., Peng, X.: Towards emulating internet-of-vehicles on a single machine. In AutomotiveUI 2021: 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Leeds, United Kingdom, 9–14 September 2021 - Adjunct Proceedings, pp. 112–114. ACM (2021). https://doi.org/10.1145/3473682.3480275
Kartynnik, Y., Ablavatski, A., Grishchenko, I., Grundmann, M.: Real-time facial surface geometry from monocular video on mobile GPUs. CoRR, abs/1907.06724 (2019)
Khodairy, M.A., Abosamra, G.: Driving behavior classification based on oversampled signals of smartphone embedded sensors using an optimized stacked-LSTM neural networks. IEEE Access 9, 4957–4972 (2021)
Liu, J., et al.: BROOK dataset: a playground for exploiting data-driven techniques in human-vehicle interactive designs. In: International Conference on Human-Computer Interaction (2023)
Liu, J., et al.: HUT: enabling high-UTility, batched queries under differential privacy protection for internet-of-vehicles. Technical Report-Feb-02 at User-Centric Computing Group, University of Nottingham Ningbo China (2022)
Mao, H., Tang, J., Zhao, X., Tang, M., Jiang, Z.: A driver drowsiness detection scheme based on 3D convolutional neural networks. Int. J. Pattern Recogn. Artif. Intell., 36(2), 2252007:1–2252007:21 (2022)
MideaPipe (2020). https://google.github.io/mediapipe/
Ming, X., et al.: Enabling efficient emulation of internet-of-vehicles on a single machine: practices and lessons. In: International Conference on Human-Computer Interaction (2023)
Moukafih, Y., Hafidi, H., Ghogho, M.: Aggressive driving detection using deep learning-based time series classification. In: Koprinkova-Hristova, P.D., Yildirim, T., Piuri, V., Iliadis, L.S., Camacho, D. (eds.) IEEE International Symposium on INnovations in Intelligent SysTems and Applications, INISTA 2019, Sofia, Bulgaria, 3–5 July 2019, pp. 1–5. IEEE (2019)
Peng, X., Huang, Z., Sun, X.: Building BROOK: a multi-modal and facial video database for human-vehicle interaction research, pp. 1–9 (2020). https://arxiv.org/abs/2005.08637
Song, Z., Wang, S., Kong, W., Peng, X., Sun, X.: First attempt to build realistic driving scenes using video-to-video synthesis in OpenDS framework. In: Adjunct Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2019, Utrecht, The Netherlands, 21–25 September 2019, pp. 387–391. ACM (2019). https://doi.org/10.1145/3349263.3351497
Song, Z., Duan, Y., Jin, W., Huang, S., Wang, S., Peng, X.: Omniverse-OpenDS: enabling agile developments for complex driving scenarios via reconfigurable abstractions. In: HCI in Mobility, Transport, and Automotive Systems: 4th International Conference, MobiTAS 2022, Held as Part of the 24th HCI International Conference, HCII 2022, Virtual Event, June 26-July 1, 2022, Proceedings, pp. 72–87. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-04987-3_5
Sun, X., et al.: Exploring personalised autonomous vehicles to influence user trust. Cogn. Comput. 12(6), 1170–1186 (2020). https://doi.org/10.1007/s12559-020-09757-x
Wahlstrom, E., Masoud, O., Papanikolopoulos, N.: Vision-based methods for driver monitoring. In: Proceedings of the 2003 IEEE International Conference on Intelligent Transportation Systems, vol. 2, pp. 903–908 (2003)
Wang, J., Xiong, Z., Duan, Y., Liu, J., Song, Z., Peng, X.: The importance distribution of drivers’ facial expressions varies over time!. In: 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 148–151 (2021)
Wang, S., et al.: Oneiros-OpenDS: an interactive and extensible toolkit for agile and automated developments of complicated driving scenes. In: HCI in Mobility, Transport, and Automotive Systems: 4th International Conference, MobiTAS 2022, Held as Part of the 24th HCI International Conference, HCII 2022, Virtual Event, June 26–July 1, 2022, Proceedings, vol. 13335, pp. 88–107. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-04987-3_6
Xiong, J., Bi, R., Zhao, M., Guo, J., Yang, Q.: Edge-assisted privacy-preserving raw data sharing framework for connected autonomous vehicles. IEEE Wirel. Commun. 27(3), 24–30 (2020)
Xiong, Z., et al.: Face2statistics: user-friendly, low-cost and effective alternative to in-vehicle sensors/monitors for drivers. In: Krömker, H. (ed.) HCI in Mobility, Transport, and Automotive Systems – 4th International Conference, MobiTAS 2022, Held as Part of the 24th HCI International Conference, HCII 2022, Virtual Event, June 26–July 1, 2022, Proceedings, Lecture Notes in Computer Science, vol. 13335, pp. 289–308. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-04987-3_20
Zhang, Yu., Jin, W., Xiong, Z., Li, Z., Liu, Y., Peng, X.: demystifying interactions between driving behaviors and styles through self-clustering algorithms. In: Krömker, H. (ed.) HCII 2021. LNCS, vol. 12791, pp. 335–350. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78358-7_23
Acknowledgement
We thank the anonymous reviewers from HCI 2023 and all members from User-Centric Computing Group for their valuable feedback and comments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bi, Z., Ming, X., Liu, J., Peng, X., Jin, W. (2023). FIGCONs: Exploiting FIne-Grained CONstructs of Facial Expressions for Efficient and Accurate Estimation of In-Vehicle Drivers’ Statistics. In: Krömker, H. (eds) HCI in Mobility, Transport, and Automotive Systems. HCII 2023. Lecture Notes in Computer Science, vol 14049. Springer, Cham. https://doi.org/10.1007/978-3-031-35908-8_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-35908-8_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35907-1
Online ISBN: 978-3-031-35908-8
eBook Packages: Computer ScienceComputer Science (R0)