Drowsy driving is a major cause of road accidents. This paper analyses the drivers' behavior in the state of fatigue driving and introduces the latest developments of drowsy driving detection technology. In this study we also propose a drowsy driving detection based on the driver's physiological signals such as eye activity measures, the inclination of the driver's head, sagging posture, heart beat rate, skin electric potential, and electroencephalographic (EEG) activities, as well as response characteristics, decline in gripping force on the steering wheel and lane keeping characteristics. By developing a hierarchical model that is able to collect the sensing data, analyze the driving behavior and the reactions to the driver, it can provide a safe and a comfortable driving environment. Combining different indications of drowsiness and processing the contextual information to predict whether a driver is drowsy, the system not only issues a warning for the driver, but also provides the drowsy driving information to transportation control center and other vehicles if necessary.