Integrating Video Information over Time. Example: Face Recognition from Video

Abstract

The ability to integrate information over time in order to come to a conclusion is a strength of cognitive systems. It allows the system, e.g., to

• 1 verify insecure observations: This is the case when data is noisy or of low-quality, or if conditions in general are non-optimal.

• 2 exploit general knowledge about spatio-temporal relations: This allows the system to exploit the specific dynamics of an object as an additional feature for, e.g., recognition, interpretation and prediction of actions of other agents.

• 3 In general, using dynamics allows the system to recursively generate and verify hypotheses for object and scene interpretation and to generate warnings when ‘implausible’ hypotheses occur or to circumvent them altogether.

We have studied the effectiveness of temporal integration for recognition purposes by using the face recognition as an example study case. Face recognition is a prominent problem and has been studied more extensively than almost any other recognition problem. An observation is that face recognition works well in ideal conditions. If those conditions, however, are not met, then all present algorithms break down disgracefully. This problem appears to be general to all vision techniques that intend to extract visual information out of low-SNR image data. It is exactly a strength of cognitive systems that they are able to cope with non-ideal situations. In this chapter we will deal with this problem.

This work was partially supported by the DARPA HumanID Grant N00014-00-1-0908.