Skip to main content
SpringerLink
Log in

High-speed gaze detection using a single FPGA for driver assistance systems

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The performance of driver gaze detection by video-based eye-tracking often encounters problems in lowcomputing speed, high-power consumption, and installation space constraints inside the vehicle. In this paper, we present an eye-tracking system that uses a single field-programmable-gate-array chip to overcome the aforementioned problems. In the detection system, the image quality is 640 \(\times\) 480 pixels with an 80 fps frame rate. Eye feature extraction is conducted using the enhanced semantics-based vague image representation approach. A succinct fully-connected neural network is then employed to classify various directions of sightline. Our experimental results exhibited a noticeable recognition speed at 0.52 \(\upmu\)s using a 100 MHz system clock and had an average detection rate of 92%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Mavely, A.G., Judith, J.E., Sahal, P.A., Kuruvilla, S.A.: Eye gaze tracking based driver monitoring system. In: 11th IEEE International Conference on Circuits and Systems (ICCS), vol. 1, no. 3, pp. 364–367 (2017)

  2. Lee, M.L., Howard, M.E., Horrey, W.J., Liang, Y., Anderson, C., Shreeve, M.S., O’Brien, C.S., Czeisler, C.A.: High risk of near-crash driving events following night-shift work. Proc. Natl. Acad. Sci. 113(1), 176–181 (2015)

    Article  Google Scholar 

  3. Sun, Y., Yu, X.: An innovative nonintrusive driver assistance system for vital signal monitoring. IEEE Trans. Biomed. Health Inform. 18(6), 1932–1939 (2014)

    Article  MathSciNet  Google Scholar 

  4. Zhu, X.M., Zheng, W.L., Lu, B.L., Chen, X.P., Chen, S.G., Wang, C.H.: EOG-based drowsiness detection using convolutional neural networks. In: 10th IEEE International Conference on Neural Networks (IJCNN), vol. 6, no. 11, pp. 128–134 (2014)

  5. Al-Rahayfeh, A., Faezipour, M.: Eye tracking and head movement detection: a state-of-art survey. IEEE J. Transl. Eng. Health Med. 1, 12 (2013)

    Article  Google Scholar 

  6. Choi, I.H., Hong, S.H., Kim, Y.G.: Real-time categorization of driver’s gaze zone using the deep learning techniques. In: 2nd IEEE International Conference on Big Data and Smart Computing (BIGCOMP), vol. 18, no. 26, pp. 143–148 (2016)

  7. Fridman, L., Lee, J., Reimer, B., Victor, V.: Owl and Lizard: patterns of head pose and eye pose in driver gaze classification. IET Comput. Vis. 10(4), 308–314 (2016)

    Article  Google Scholar 

  8. Vicente, F., Huang, Z., Xiong, X., Torre, D.L.F., Zhang, W., Levi, D.: Driver gaze tracking and eyes Off the road detection system. IEEE Trans. Intell. Transp. Syst. 16(4), 2014–2027 (2015)

    Article  Google Scholar 

  9. Zhai, F., Yang, Z., Song, Y., Ma, H.: A detection model for driver’s unsafe states based on real-time face-vision. In: 1st International Conference on Image Analysis and Signal Processing (IASP), vol. 4, pp. 12–14 (2010)

  10. Xu, J., Min, J., Hu, J.: Real-time eye tracking for the assessment of driver fatigue. Healthc. Technol. Lett. 5(2), 54–58 (2018)

    Article  Google Scholar 

  11. Hon, g, Qin, H.: Drivers drowsiness detection in embedded system. In: 2nd IEEE International Conference on Vehicular Electronics and Safety, vol. 5, pp. 13–15 (2007)

  12. Hansen, D.W., Ji, Q.: In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 478–500 (2010)

    Article  Google Scholar 

  13. Liu, T., Yang, Y., Huang, G.B., Yeo, Y.K., Lin, Z.: Driver distraction detection using semi-supervised machine learning. IEEE Trans. Intell. Transp. Syst. 17(4), 1108–1120 (2016)

    Article  Google Scholar 

  14. Said, S., AlKork, S., Beyrouthy, T., Hassan, M., Abdellatif, O., Abdraboo, M.F.: Real time eye tracking and detection- a driving assistance system. Adv. Sci. Technol. Eng. Syst. J. 3(6), 446–454 (2018)

    Article  Google Scholar 

  15. Lai, H.C., Savvides, M., Chen, T.: Proposed FPGA hardware architecture for high frame rate (\(\gg\)100 fps) face detection using feature cascade classifiers. In: 1st IEEE International Conference on Biometrics: Theory, Applications, and Systems, vol. 16, pp. 27–29 (2007)

  16. Chun, H., Papakonstantinou, A., Chen, D.: A novel SoC architecture on FPGA for ultra fast face detection. In: 27th IEEE International Conference on Computer Design (ICCD), vol. 4, no. 7, pp. 412–418 (2009)

  17. Ishii, I., Ichida, T., Gu, Q., Takaki, T.: 500-fps face tracking system. J. Real Time Image Proc. 8(4), 379–388 (2013)

    Article  Google Scholar 

  18. Yu, Y.H., L, T.T., Chen, P.Y., Kwok, N.: On-chip real-time feature extraction using semantic annotations for object recognition. J. Real Time Image Proc. 15(2), 249–264 (2018)

    Article  Google Scholar 

  19. Sandnes, F.E., Neyse, L., Huang, Y.P.: Simple and practical skin detection with static RGB-color lookup tables: a visualization-based study. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC), vol. 9, no. 12, pp. 2370–2375 (2016)

  20. Huang, H., Guo, J.: Application of multiple order morphology transformation on edge detection for liver ct image. In: 7th IEEE International Conference on Information Technology and Artificial Intelligence Conference (ITAIC), vol. 20, no. 21, pp. 107–110 (2014)

  21. Devadethan S., Titus G., Purushothaman S.: Face detection and facial feature extraction based on a fusion of knowledge based method and morphological image processing, In: International Conference on Emerging Research Areas: Magnetics, Machines and Drives (AICERA/iCMMD), 24–26 July 2014, pp. 5 (2014)

  22. Guo, K., Sui, L., Qiu, J., Yu, J., Wang, J., Yao, S., Han, S., Wang, Y.: Angel-eye: a complete design flow for mapping CNN onto embedded FPGA. IEEE Trans. Comput. Aided Des. Integr. Circ. Syst. 37(1), 35–47 (2018)

    Article  Google Scholar 

  23. Ding, Z., Zhao, F., Wang, T., Shu, W., Wu, M.: Hecto-scale frame rate face detection system for SVGA source on FPGA board. In: IEEE 19th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), vol. 1, no. 3, pp. 37–40 (2011)

  24. Qin, H., Xu, X., Hu, Z., Zhang, D.: Eye tracking system based on SOPC. In: 21th IEEE International Conference on Electronics, Circuits and Systems (ICECS), vol. 7, no. 10, pp. 171–174 (2014)

  25. Kocić O., Simić A., Bjelica M.Z., Maruna T.:Optimization of driver monitoring ADAS algorithm for heterogeneous platform. In: 24th International Conference on Telecommunications Forum (TELFOR), 22–23 Nov 2016, pp. 4 (2016)

Download references

Acknowledgements

Supports from the Ministry of Science and Technology, under Grant MOST106-2221-E-194-060-MY2, Advanced Institute of Manufacturing with High-Tech Innovation (AIM-HI), and Center for Innovative Research on Aging Society (CIRAS) from The Featured Areas Research Center Program within the framework of Higher Education Sprout Project by the Ministry of Education (MoE) in Taiwan are gratefully acknowledged.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, National Chung Cheng University, No. 168, University Rd., Min-Hsiung Township, Chiayi County, 621, Taiwan

    Ying-Hao Yu, Yi-Siang Ting & Norbert Michael Mayer

  2. School of Mechanical and Manufacturing Engineering, The University of New South Wales, Sydney, NSW, 2052, Australia

    Ngaiming Kwok

Authors
  1. Ying-Hao Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi-Siang Ting
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ngaiming Kwok
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Norbert Michael Mayer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ying-Hao Yu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, YH., Ting, YS., Kwok, N. et al. High-speed gaze detection using a single FPGA for driver assistance systems. J Real-Time Image Proc 18, 681–690 (2021). https://doi.org/10.1007/s11554-020-01004-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-01004-8

Keywords

Search

Navigation