Skip to main content
SpringerLink
Log in

A real-time non-intrusive FPGA-based drowsiness detection system

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Automotive has gained several benefits from the Ambient Intelligent researches involving the deployment of sensors and hardware devices into an intelligent environment surrounding people, meeting users’ requirements and anticipating their needs. One of the main topics in automotive is to anticipate driver needs and safety, in terms of preventing critical and dangerous events. Considering the high number of caused accidents, one of the most relevant dangerous events affecting driver and passengers safety is driver’s drowsiness and hypovigilance. This paper presents a low-intrusive, real-time driver’s drowsiness detection system for common vehicles. The proposed system exploits the “bright pupil” phenomenon generated by a 850 nm IR source light embedded on the car dashboard. This visual effect, due to the retina’s property of reflecting the 90% of the incident light, makes easier the detection of driver’s eyes. At the same time, the “bright pupil” effect is used to quantify the driver’s drowsiness level as the percentage of time in which the driver’s eyes are closed more than 80%. The efficiency of the image processing chain, together with an embedded hardware device exploiting the availability of mature reconfigurable hardware technology, such as Field Programmable Gate Array, allow to implement a real-time detection system able to process an entire 720 × 576 frame in 16.7 ms. The effectiveness of the proposed system has been successfully tested with a human subject operating in real conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Åkerstedt T, Folkard S (1997) The three-process model of alertness and its extension to performance, sleep latency, and sleep length. Chronobiol Int 14(2):115–123

    Article  Google Scholar 

  • Artaud P, Planque S, Lavergne C, Cara H, de Lepine P, Tarriere C, Gueguen B (1994) An on-board system for detecting lapses of alertness in car driving. In: Proceedings of the fourteenth international conference on enhanced safety of vehicles, vol 1, Munich, Germany

  • Boverie S, Giralt A, Le Quellec J, Hirl A (1998) Intelligent system for video monitoring of vehicle cockpit. In: Proceedings of the international Congress expo ITS: advanced controls vehicle navigation systems, SAE International, pp 1–5

  • Celoxica (2005a) RC200/203 Manual, Doc. No. 1

  • Celoxica (2005b) Handel-C Language Reference Manual, Doc. No. RM-1003-4.2

  • Celoxica (2005c) DK Design Suite user guide, Doc. No. UM-2005-4.2

  • Celoxica (2005d) PixelStreams Manual, Doc. No. 1

  • Damousis I, Tzovaras D (2008) Fuzzy fusion of eyelid activity indicators for hypovigilance-related accident prediction. IEEE Trans Intell Transp Syst 9(3):491

    Article  Google Scholar 

  • Dawson D, Lamond N, Donkin K, Reid K (1998) Quantitative similarity between the cognitive psychomotor performance decrement associated with sustained wakefulness and alcohol intoxication. Managing fatigue in transportation: Selected Papers from the 3rd Fatigue in Transportation Conference, pp 231–56

  • Dinges D, Mallis M (1998) Managing fatigue by drowsiness detection: can technological promises be realized? Managing fatigue in transportation. In: Selected Papers from the 3rd fatigue in transportation conference, pp 209–229

  • Dinges D, Mallis M, Maislin G, Powell J (1998) Evaluation of techniques for ocular measurement as an index of fatigue and the basis for alertness management. NHTSA Report No DOT HS 808:762

  • D’Orazio T, Leo M, Spagnolo P, Guaragnella C, ISSIA C, Bari I (2004) A neural system for eye detection in a driver vigilance application. In: Proceedings of the 7th international IEEE conference on intelligent transportation systems, pp 320–325

  • Ducatel K, Bogdanowicz M, Scapolo F, Leijten J, Burgelman JC (2001) Scenarios for ambient intelligence in 2010, ISTAG, pp 1–58, ISBN 92-894-0735-2

  • Grace R (2001) Drowsy driver monitor and warning system. In: International driving symposium on human factors in driver assessment, training and vehicle design, pp 201–208

  • Grace R, Byrne V, Bierman D, Legrand J, Gricourt D, Davis B, Staszewski J, Carnahan B (1998) A drowsy driver detection system for heavy vehicles. In: Proceedings of the 17th DASC. The AIAA/IEEE/SAE digital avionics systems conference, vol 2, pp 1–8

  • Grace R, Byrne V, Bierman D, Legrand J, Gricourt D, Davis B, Staszewski J, Carnahan B (1999) A drowsy driver detection system for heavy vehicles. In: Proceedings of the 17th AIAA/IEEE/SAE digital avionics systems conference (DASC), vol 2

  • Gu H, Ji Q, Zhu Z (2002) Active facial tracking for fatigue detection. In: Proceedings of the sixth IEEE workshop on applications of computer vision

  • Hamada T, Ito T, Adachi K, Nakano T, Yamamoto S (2003) Detecting method for drivers’ drowsiness applicable to individual features. In: Intelligent transportation systems, vol 2, pp 1405–1410

  • Hartley L, Horberry T, Mabbott N, Krueger G (2000) Review of fatigue detection and prediction technologies. National Road Transport Commission report 642(54469)

  • Hjelmås E, Low B (2001) Face detection: a survey. Comput Vis Image Underst 83(3):236–274

    Article  MATH  Google Scholar 

  • IEEE (2009) IEEE 1609—family of standards for wireless access in vehicular environments (WAVE). http://www.standards.its.dot.gov/fact_sheet.asp?f=80

  • Ji Q, Zhu Z, Lan P (2004) Real-time nonintrusive monitoring and prediction of driver fatigue. IEEE Trans Veh Technol 53(4):1052–1068

    Google Scholar 

  • Kleinberger T, Jedlitschka A, Storf H, Steinbach-Nordmann S, Prueckner S (2009) An approach to and evaluations of assisted living systems using ambient intelligence for emergency monitoring and prevention. In: Universal access in human–computer interaction intelligent and ubiquitous interaction environments, pp 199–208

  • Kojima N, Kozuka K, Nakano T, Yamamoto S (2001) Detection of consciousness degradation and concentration of a driver for friendly information service. In: Proceedings of the IEEE international vehicle electronics conference, pp 31–36

  • Lan KC, Chou CM (2008) Realistic mobility models for vehicular ad hoc network (vanet) simulations. In: Proceedings of the 8th International Conference on ITS telecommunications (ITST 2008), pp 362–366

  • Lavergne C, De Lepine P, Artaud P, Planque S, Domont A, Tarriere C, Arsonneau C, Yu X, Nauwink A, Laurgeau C et al (1996) Results of the feasibility study of a system for warning of drowsiness at the steering wheel based on analysis of driver eyelid movements. In: Proceedings of the fifteenth international technical conference on the enhanced safety of vehicles, vol 1, Melbourne, Australia

  • Liang Y, Reyes M, Lee J (2007) Real-time detection of driver cognitive distraction using support vector machines. IEEE Trans Intell Transp Syst 8(2):340–350

    Article  Google Scholar 

  • Mabbott N, Lydon M, Hartley L, Arnold P (1999) Procedures and devices to monitor operator alertness whilst operating machinery in open-cut coal mines. Stage 1: state-of-the-art review. ARRB Transport Res Rep RC 7433

  • Manvi SS, Kakkasageri MS, Pitt J (2009) Multiagent based information dissemination in vehicular ad hoc networks. Mobile Inform Syst 5(4):363–389

    Google Scholar 

  • Morimoto C, Koons D, Amir A, Flickner M (2000) Pupil detection and tracking using multiple light sources. Image Vis Comput 18(4):331–335

    Article  Google Scholar 

  • Moriyama T, Kanade T (2007) Automated individualization of deformable eye region model and its application to eye motion analysis. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR’07), pp 1–6

  • Moriyama T, Kanade T, Xiao J, Cohn J (2006) Meticulously detailed eye region model and its application to analysis of facial images. In: IEEE transactions on pattern analysis and machine intelligence, pp 738–752

  • Nehmer J, Becker M, Karshmer A, Lamm R (2006) Living assistance systems: an ambient intelligence approach. In: Proceedings of the 28th international conference on Software engineering, ACM

  • Ogawa K, Shimotani M (1997) A Drowsiness detection system. Mitsubishi Electric Advance, pp 13–16

  • Onken R (1994) DAISY, an adaptive, knowledge-based driver monitoring and warning system. In: Proceedings of the intelligent vehicles’ 94 symposium, pp 544–549

  • Peters B, Anund A (2003) System for effective assessment of driver vigilance and warning according to traffic risk estimation (awake). Revision II. AWAKE IST-2000-28062, Document ID: Del. 7_1

  • Qadri NN, Altaf M, Fleury M, Ghanbari M (2010) Robust video communication over an urban VANET. Mobile Inform Syst 6(3):259–280

    Google Scholar 

  • Rakotonirainy A, Tay R (2004) In-vehicle ambient intelligent transport systems (i-vaits): Towards an integrated research. In: Proceedings of the 7th international IEEE conference on intelligent transportation systems, pp 648–651

  • Ramos C (2007) Ambient intelligence—a state of the art from artificial intelligence perspective. Progress in Artificial intelligence, pp 285–295

  • Rikert T, Jones M (1998) Gaze Estimation Using Morphable Models. In: Proceedings of the 3rd international conference on face & gesture recognition, IEEE Computer Society

  • Rimini-Doering M, Manstetten D, Altmueller T, Ladstaetter U, Mahler M (2001) Monitoring driver drowsiness and stress in a driving simulator. In: Proceedings of the first international driving symposium on human factors in driver assessment, training and vehicle design, pp 58–63

  • Rongben W, Lie G, Bingliang T, Lisheng J (2004) Monitoring mouth movement for driver fatigue or distraction with one camera. In: Proceedings of the 7th international IEEE conference on intelligent transportation systems, pp 314–319

  • Saito S (1992) Does fatigue exist in a quantitative measurement of eye movements? Ergonomics 35(5):607–615

    Article  Google Scholar 

  • Shin G, Chun J (2007) Vision-based multimodal human computer interface based on parallel tracking of eye and hand motion. In: Proceedings of the international conference on convergence information technology, pp 2443–2448

  • Smith P, Shah M, da Vitoria Lobo N (2000) Monitoring head/eye motion for driver alertness with one camera. In: Proceedings of the 15th international conference on pattern recognition, vol 4

  • Smith P, Shah M, da Vitoria Lobo N (2003) Determining driver visual attention with one camera. IEEE Trans Intell Transp Syst 4(4):205–218

    Article  Google Scholar 

  • Su MS, Chen CY, Cheng KY (2002) An automatic construction of a person’s face model from the person’s two orthogonal views. In: Proceedings of the conference on geometric modeling and processing

  • Tang J, Zhang J (2009) Eye tracking based on grey prediction. In: Proceedings of the first international workshop on education technology and computer science (ETCS’09), vol 2

  • Ueno H, Kaneda M, Tsukino M (1994) Development of drowsiness detection system. In: Proceedings of the vehicle navigation and information systems conference, pp 15–20

  • Varaiya P (1993) Smart cars on smart roads: problems of control. IEEE Trans Autom Control 38(2):195–207

    Article  MathSciNet  Google Scholar 

  • Vitabile S, Bono S, Sorbello F (2007) An embedded real-time automatic lane-keeping system. In: Apolloni B et al (eds) Knowledge-based intelligent information and engineering systems—KES 2007/WIRN 2007, Lecture notes in artificial intelligence 4692(Part I):647–654

  • Vitabile S, Bono S, Sorbello F (2008) An embedded real-time automatic lane-keeper for automatic vehicle driving. In: Proceedings of the second international conference on complex, intelligent and software intensive systems (CISIS 2008). IEEE Press, Barcelona, pp 279–285

  • Wahlstrom E, Masoud O, Papanikolopoulos N (2003) Vision-based methods for driver monitoring. In: Proceedings of the IEEE conference on intelligent transportation systems, pp 903–908

  • Wierwille W (1999) Historical perspective on slow eyelid closure: Whence PERCLOS. In: Technical proceedings of ocular measures of driver alertness conference, Herndon, VA (FHWA Technical Report No. MC-99-136). Federal Highway Administration, Office of Motor Carrier and Highway Safety, Washington, DC, pp 31–53

  • Yammamoto K, Higuchi S (1992) Development of a drowsiness warning system. J Soc Automot Eng Jpn 46(9):127–133

    Google Scholar 

  • Zhang C, Lin X, Lu R, Ho PH, Shen X (2008) An efficient message authentication scheme for vehicular communications. IEEE Trans Veh Technol 57(6):3357–3368

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Antonio Ingrassia, Marco Mancuso, and Angelo Mogavero for their valuable support in the system testing phase.

Author information

Authors and Affiliations

  1. Dipartimento di Biopatologia e Biotecnologie Mediche e Forensi, University of Palermo, Via del Vespro, 129, 90127, Palermo, Italy

    Salvatore Vitabile

  2. Dipartimento di Ingegneria Informatica, University of Palermo, Viale delle Scienze, 90128, Palermo, Italy

    Alessandra De Paola & Filippo Sorbello

Authors
  1. Salvatore Vitabile
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alessandra De Paola
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Filippo Sorbello
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Salvatore Vitabile.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vitabile, S., De Paola, A. & Sorbello, F. A real-time non-intrusive FPGA-based drowsiness detection system. J Ambient Intell Human Comput 2, 251–262 (2011). https://doi.org/10.1007/s12652-011-0049-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-011-0049-z

Keywords

Search

Navigation