Abstract
In order to reduce the probability of human error when driving and provide support for the design of control method of in-vehicle information system, hand motion analysis is important to evaluate driver performance. This paper introduces a depth-based method using hand motion analysis to evaluate the performance of the secondary task when driving. The secondary task is designed to use the central control panel in the car to adjust the temperature of the air conditioner. 3D spatial hand coordinates are determined by background subtraction on depth images. After obtaining the 3D spatial hand coordinate of each frame, the velocity, the operation time and the hand movement trajectory length of the secondary task can be determined. The angle of velocity between consecutive frames (AOV) is calculated to analyze the consistency of hand motion. Besides, sample entropy (SampEn) is used to measure the regularity of change in speed and direction of hand movement velocity. The experimental data show that the participant completes the secondary task with shorter operation time and trajectory length, fewer times of retrace, higher consistency ratio and smaller SampEn of AOV and speed when the primary task is less difficult. The results indicate that when the primary task is less difficult, the performance of the secondary task is better. Our method proposed in this paper can effectively evaluate the performance of the secondary task when driving.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Yang, L., Ma, R., Zhang, H.M., Guan, W., Jiang, S.: Driving behavior recognition using EEG data from a simulated car-following experiment. Accid. Anal. Prev. 116, 30–40 (2018)
Lee, B.G., Chong, T.W., Lee, B.L., Park, H.J., Kim, Y.N., Kim, B.: Wearable mobile-based emotional response-monitoring system for drivers. IEEE Trans. Hum.-Mach. Syst. 47(5), 636–649 (2017)
Maccora, J., Manousakis, J.E., Anderson, C.: Pupillary instability as an accurate, objective marker of alertness failure and performance impairment. J. Sleep Res. 28(2), e12739 (2019)
Shaik, K.B., Ganesan, P., Kalist, V., Sathish, B.S., Jenitha, J.M.M.: Comparative study of skin color detection and segmentation in HSV and YCbCr color space. Procedia Comput. Sci. 57, 41–48 (2015)
Kolkur, S., Kalbande, D., Shimpi, P., Bapat, C., Jatakia, J.: Human skin detection using RGB, HSV and YCbCr color models. arXiv preprint arXiv:1708.02694 (2017)
Bandini, A., Zariffa, J.: Analysis of the hands in egocentric vision: a survey. IEEE Trans. Pattern Anal. Mach. Intell. (2020)
Li, C., Kitani, K.M.: Pixel-level hand detection in ego-centric videos. In: Conference on Computer Vision and Pattern Recognition, pp. 3570–3577. IEEE (2013)
Urooj, A., Borji, A.: Analysis of hand segmentation in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4710–4719 (2018)
Cai, M., Lu, F., Sato, Y.: Generalizing hand segmentation in egocentric videos with uncertainty-guided model adaptation. In: Conference on Computer Vision and Pattern Recognition, pp. 14392–14401. IEEE/CVF (2020)
Bambach, S., Lee, S., Crandall, D.J., Yu, C.: Lending a hand: detecting hands and recognizing activities in complex egocentric interactions. In: International Conference on Computer Vision, pp. 1949–1957. IEEE (2015)
Ren, Z., Yuan, J., Zhang, Z.: Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera. In: 19th International Conference on Multimedia, pp. 1093–1096. ACM (2011)
Rogez, G., Supancic, J.S., Ramanan, D.: Understanding everyday hands in action from RGB-D images. In: International Conference on Computer Vision, pp. 3889–3897. IEEE (2015)
Sridhar, S., Mueller, F., Zollhöfer, M., Casas, D., Oulasvirta, A., Theobalt, C.: Real-time joint tracking of a hand manipulating an object from RGB-D input. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 294–310. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_19
Kang, B., Tan, K.H., Jiang, N., Tai, H.S., Tretter, D., Nguyen, T.: Hand segmentation for hand-object interaction from depth map. In: Global Conference on Signal and Information Processing (GlobalSIP), pp. 259–263. IEEE (2017)
Dosis, A., et al.: Synchronized video and motion analysis for the assessment of procedures in the operating theater. Arch. Surg. 140(3), 293–299 (2005)
Zago, M., et al.: Educational impact of hand motion analysis in the evaluation of fast examination skills. Eur. J. Trauma Emerg. Surg. 46(6), 1421–1428 (2020)
Zia, A., Sharma, Y., Bettadapura, V., Sarin, E.L., Essa, I.: Video and accelerometer-based motion analysis for automated surgical skills assessment. Int. J. Comput. Assist. Radiol. Surg. 13(3), 443–455 (2018). https://doi.org/10.1007/s11548-018-1704-z
Sharma, Y., et al.: Video based assessment of OSATS using sequential motion textures. Georgia Institute of Technology (2014)
Richman, J.S., Moorman, J.R.: Physiological time-series analysis using approximate entropy and sample entropy. Am. J. Physiol.-Heart Circ. Physiol. 278, H2039 (2000)
Zhao, Y., Wang, Z., Lu, Y., Fu, S.: A visual-based approach for manual operation evaluation. In: Harris, D., Li, W.-C. (eds.) HCII 2020. LNCS (LNAI), vol. 12186, pp. 281–292. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49044-7_23
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wen, H., Wang, Z., Fu, S. (2021). Secondary Task Behavioral Analysis Based on Depth Image During Driving. In: Kurosu, M. (eds) Human-Computer Interaction. Design and User Experience Case Studies. HCII 2021. Lecture Notes in Computer Science(), vol 12764. Springer, Cham. https://doi.org/10.1007/978-3-030-78468-3_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-78468-3_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78467-6
Online ISBN: 978-3-030-78468-3
eBook Packages: Computer ScienceComputer Science (R0)