Detection of Checking Action on Parking Significant for Cognitive Dysfunction Patients

Abstract

Cognitive dysfunction patient could have symptoms such as attention disorder, execute function disorder and so on. These symptoms may cause unsafe driving in daily life. The degree of these symptoms can be evaluated by neuropsychological examination, however, the correspondence relationship between these symptoms and unsafe driving is uncertain. Therefore, it is difficult to judge the patient’s driving aptitude only by neuropsychological examination. To solve this problem, we are developing unsafe driving detection system. It requires some small wireless sensors measuring triaxial angular velocity and acceleration to be attached on user’s head, toes and steering wheel, and GPS sensor on a car. Since many of the cognitive dysfunction patients have symptom of attention disorder, it assumed that safety checking actions tend to be careless. Based on this assumption, we have analyzed the driver’s checking action on intersections and changing lane. Here discusses the result of analysis assumingly complex checking action on parking cars. When parking a car, driver must control the car and pay attention to rear, front and both side simultaneously. From the video analysis of the experiment with small wireless sensors and a real car, cognitive dysfunction patients tend to use rear view mirror to accomplish their safety checking. Especially, the right side mirror checking action was significantly high compare to the one without cognitive dysfunction. Safety checking action, accomplished only by rear view mirror is more dangerous than checking directly by turning around. On investigating of how much the attached sensors can acquire these driver’s checking actions, it was clarified that these sensors can identify only the direction of head, however cannot identify if the drivers were looking at the target object directly or through rear view mirror. It should be studied further that how much can our unsafe driving detection system distinguish direct and indirect one.

1 Introduction

When a part of the brain is affected by apoplexy, brain tumor or injury to the head, cognitive dysfunction symptoms including attention disorder and execute function disorder may appear. Although these symptoms can be improved by medical treatment, it may be dangerous to drive a car in daily life, depending on the degree of the symptom. In Japan, under the road traffic law, driving license can be suspended or canceled in case of problems with recognition, judgment or operation, which are identified through aptitude tests. However, there are no standard guidelines to judge the driving aptitude of the cognitive dysfunction patients. In some hospitals, neuropsychological examination is used to evaluate the degrees of symptoms and driving simulators are used to measure the reaction time to sudden dangers on the road and avoidance operation such as braking and steering. However, the correspondence relationship between these symptoms and unsafe driving is uncertain and such simulators do not give the sense of acceleration and deceleration to the user, and visual resolution or coverage angle of the display is limited, there is a certain gap between real and virtual driving. We have been developing the Driving Skill Evaluation System [1] for cognitive dysfunction drivers, which acquires the driver’s behaviors with wearable and wire-less motion sensors and GPS sensor [2, 3]. In this paper, we focus on the difference of safety checking action between drivers with cognitive dysfunction and drivers without cognitive dysfunction. We report the result of analysis on patients’ driving data acquired from the experiments by using our system on a designed “private course”.

2 Experiment on Designed Parking Area

The experiments are conducted with subjects equipped with wearable wireless motion sensors using real cars on “private course” in Toyama Driving Education Center, Japan. Figure 1 shows the top view of designed “private course” for the experiment. The course includes several kinds of road conditions, such as signalized/nonsignalized, intersections with/without stop sign and parking area, and roads with several kinds of speed limitation that take 10–15 min to drive. Figure 1 also shows enlarged view of the parking area and expected car parking trajectory. On parking the car, drivers must go pass the front of parking space, stop once and move backward to the parking space. Drivers can retry this parking procedure as much as they need.

Fig. 1.

Designed “private course” and enlarged view of parking area

The subjects are 14 cognitive dysfunction patients and 13 adults without cognitive dysfunction. Six video cameras are installed inside and outside of the car in order to record driving behavior in detail. These video cameras record the drivers from forward, side and backward. One video camera is also attached near the driver’s foot in order to record pedal action.

3 Detected Differences from Video Analysis

As shown in Fig. 2, the direction of safety check on parking was classified subjectively to following five groups, front forward, right front, left front, right rear and left rear.

Fig. 2.

Classification of safety checking directions

Figure 3 shows three differences found by the video analysis. Two differences are found by the video analysis. One difference is the direction of face angle on safety checking. The number of safety checking actions for the right rear direction is significantly less than left rear (a). The reason is assumed that right-side steering wheel car was used and most of rear safety checking were accomplished with turning lest-side in this experiment. And most of these checking actions were classified as the safety checking of left rear direction, not a right rear direction. Another difference can be found between the cognitive dysfunction drivers and without cognitive dysfunction. The number of safety checking actions for the left front and left rear by the drivers without cognitive dysfunction tend to be larger than ones with cognitive dysfunction (b). The rate of safety checking action for right front by the drivers with cognitive dysfunction drivers are significantly larger than ones without cognitive dysfunction (c). These result can indicate the difference of means to conduct rear safety check. The rear checking by the drivers with cognitive dysfunction is mainly accomplished through the right-side mirror, on the contrary, drivers without cognitive dysfunction mainly accomplished them directly by turning around. However, in the video analysis, checking rear, through the mirror or direct check to right front could not be separated precisely. With eye direction tracking system, these two actions can be distinguished.

Fig. 3.

Differences found by video analysis

4 Unsafe Driving Detection System

We have been developing the Driving Skill Evaluation System with small wireless sensors and evaluate driving aptitude of patients with cognitive dysfunction [4]. In this experiment, this system was used to capture driver’s action. We put the sensors on the driver’s head and toes, as well as the car’s steering wheel and body, in order to measure their movements. The data from a car, once stopped after passing the front of the parking space until when the car is entirely stopped on the parking space, was captured as the safety checking action data on parking. Face direction angle was calculated and analyzed as safety checking action from these data.

We obtain yaw angle (relative angle around the vertical axis from the ground) of the head from sensor data of acceleration and angular velocity by Kalman filter, where constant offset of angular velocity is removed based on the data while the car stops before starting. In order to obtain relative yaw angle from the car body, we subtract yaw angle of a car body from that of the driver’s head. However, there still remains irregular offset drift of angular velocity, which affects the estimated yaw angle. To remove the irregular offset, we calculate the reference value from the middle point between the minimal and maximal values of the yaw angle obtained above in a certain period. We obtain a corrected yaw angle (face direction) by subtracting the reference value from the yaw angle. Furthermore, we also obtain rotation angle of the steering wheel from sensor data of acceleration and angular velocity by Kalman filter.

5 Deference Detection by the Sensors

Figures 4 and 5 shows the accumulated safety checking time, for each head angle of the drivers with cognitive dysfunction and without cognitive dysfunction on parking respectively calculated from the attached sensor data.

Fig. 4.

Accumulated safety checking time (without cognitive dysfunction driver)

Fig. 5.

Accumulated safety checking time (with cognitive dysfunction driver)

In Figs. 4 and 5, 0-degree angle indicates right in front forward, and negative value indicates left angle of front forward, positive value indicates right angle of front forward. To compare with the result of video analysis shown in chapter three, these safety checking actions must be classified into five groups which are used in video analysis. In Figs. 3 and 4 it is confirmed that the accumulated time of safety checking action around −75° and 75° looks less than other close angles. Then, the border line C and the border line D was decided to ±75° respectively. And border A and B decided to ±10° respectively because the scope of both side check should include the front corner check of the car and measured angle of the right corner is 75°. Using these decided value, captured safety checking actions were classified to five groups. It is confirmed that the result of comparing video analysis, the sensor data also indicates that the number of safety checking action for rear directly is significantly less than those for other directions. However, the rate of are not significant while both has same tendency on the video analysis.

It cannot be detected that the difference of the number of safety checking actions for the left front and left rear between the drivers with cognitive dysfunction and ones without cognitive dysfunction, which was detected on the video analysis. While the rate of safety checking action for right front by the drivers with cognitive dysfunction drivers tends to be larger than ones without cognitive dysfunction.

6 Conclusion

We conducted experiments equipped with wearable wireless motion sensors using real cars. The video analysis of the safety checking action found the typical safety check action to the rear direction by the cognitive dysfunction patients which can be the candidate of their feature. However, it is uncertain why the feature appears, but the following hypothesis are considered. The cognitive dysfunction patients tend to check rear through rear view mirror, while drivers without cognitive dysfunction tend to check directly by turning around. To verify this hypothesis, the experiment with eye tracking system should be conducted. And we investigated adaptability of our Driving Skill Evaluation System [1] to distinguish above typical checking actions of cognitive dysfunction drivers and clarified that our system can detect a part of those typical safety checking action.

It should be studied further that with eye tracking system, in order to analyze the reason why the typical safety checking action occurs. Accordingly, our Driving Skill Evaluation System [1] must be improved to judge more detailed information.