Elsevier

Pattern Recognition

Volume 46, Issue 8, August 2013, Pages 2220-2227
Pattern Recognition

Novel and efficient pedestrian detection using bidirectional PCA

https://doi.org/10.1016/j.patcog.2013.01.007Get rights and content

Abstract

The detection of pedestrian has attracted much research in the past decade due to the essential role it plays in intelligent video surveillance and vehicle vision systems. However, the existing algorithms do not meet the requirement of real applications in terms of detection performance. This paper proposes a new robust algorithm for pedestrian detection based on image reconstruction using bidirectional PCA (BDPCA). Unlike PCA, since it is a straightforward image projection technique, BDPCA preserves the shape structure of objects and is computationally effective. Due to these advantages, BDPCA is a promising tool for object detection and recognition. The algorithm was tested on two datasets, INRIA and PennFudanPed. Our experiment proved that using BDPCA with vertical edge images was the most suitable for pedestrian detection. The comparison between BDPCA, PCA, and histogram of oriented gradient (HOG) based methods demonstrates superior accuracy and robustness of the proposed algorithm to the others.

Highlights

► This paper proposes a new algorithm for pedestrian detection using Bidirectional PCA (BDPCA). ► A comparison against PCA and HOG-based methods proved the accuracy and robustness of the BDPCA. ► The correct detection rate of BDPCA is higher than those of HOG and PCA with 5% and 52%, respectively.

Introduction

For the past decade, many applications in which detecting people plays an essential role has been developed. Such major applications are video surveillance systems, airport security, driving assistance systems, automatic driving cars, smart home, and robotics. The importance of these applications makes pedestrian detection a topic worthy of studying. Although there has been much effort to outperform pedestrian detection algorithms, the accuracy of the existing algorithms is still far from the requirement of real applications. The reasons why pedestrian detection is difficult can be summarized as follows:

  • 1.

    Diversity in appearance: The appearance of a human can extremely be varied by changing pose, clothes, or the objects being carried, or viewpoints of camera. In addition, people have a large variation in size. Therefore, a pedestrian detection algorithm has to be able to cope with these variations.

  • 2.

    Environment diversity: In this research, the environment includes the background where people are detected, illumination, and weather conditions. Since pedestrian detection is used in a wide range of applications, the background can be as diverse and complex as inside a building, campus, airport, road, or urban. This complexity is one of the biggest challenges to pedestrian detection. Due to this wide range of applications, pedestrian detection also suffers from the problem of illumination or weather changing.

  • 3.

    Partial occlusion: Since people appear in dynamic and uncontrolled backgrounds, partial occlusions surely happen at any time. Therefore, as same as any object detection problem, partial occlusion needs to be considered in developing pedestrian detection algorithms.

  • 4.

    Camera motion: In some applications, such as surveillance systems or airport security, the cameras are fixed, hence the backgrounds are static. In this case, motion can be used as an efficient cue for pedestrian detection. However, in other applications, such as driving assistance systems, automatic driving cars, or robots, both the camera and the objects in a scene are moving, which makes it difficult to extract pedestrian motion in this case.

  • 5.

    Real-time processing: The major applications that require pedestrian detection also demands real-time processing as their vital question. Pedestrian detection is an essential part in these systems; however, it is only a single step in the whole system. Thus, it needs to be done as fast as possible to preserve the real-time processing characteristic of the whole system.

Fig. 1 shows difficult cases of detecting pedestrian due to occlusion, pose variation, camera viewpoint, and illumination change. In literature, numerous methods have been proposed for detecting pedestrians from images. This paper provides a brief summary and analysis of the existing methods in Section 2. In addition, it proposes a new method for automatically detecting pedestrians in still images based on Bidirectional PCA (BDPCA). Bidirectional PCA was originated from [1] for image recognition (mainly focuses on the face recognition problem). Unlike classical PCA, BDPCA is a straightforward image projection technique; hence it does not require converting an image to a high dimensional image vector. BDPCA extracts features from an image by reducing the dimension in both column and row directions, thus it requires less computation than PCA [2].

The proposed method includes two steps: training and classification. In the training step, BDPCA is applied to a set of pedestrian images and another set of non-pedestrian images. For each space (pedestrian or non-pedestrian), this step produces a descriptor consisting of the mean image, row, and column projectors, so called, 2D eigen-descriptor. In the classification step, an input image is scanned by a window of a certain size (in this work, 64×180 pixels) defining ROI (Region of interest). And, each ROI, is reconstructed by the 2D eigen-descriptors of pedestrian and non-pedestrian space, then determined whether it belongs to the pedestrian space or not based on reconstruction errors.

This paper implements BDPCA using different source images such as grayscale, edge, and vertical-edge images of the original images, and a complete performance analysis was carried out. The analysis shows that the 2D eigen-descriptor of the vertical-edge image is the most suitable for pedestrian detection. Furthermore, a comparison between BDPCA using the vertical-edge, PCA, and HOG-based methods demonstrates the superiority of the proposed method to the other methods in both accuracy and robustness for detecting pedestrians in unconstrained environment.

The rest of this paper is organized as follows: Section 2 provides a brief summary and analysis of the existing methods for automatic pedestrian detection. Section 3 describes the BDPCA-based pedestrian detection, and Section 4 compares the performance between the proposed method and the existing methods such as PCA and HOG. Finally, Section 5 states the conclusions and future works.

Section snippets

Literature review

In general, the process of pedestrian detection is divided into two subsequent steps: ROI selection and classification. There are several approaches to generate candidate ROIs for the classification step. The simplest approach is brute-force window sliding, which uses a fixed size detector to scan across the image at multiple scales and locations. This approach usually suffers from high processing time. In the case of static cameras, background subtraction can be performed to extract ROIs,

Bidirectional PCA

Bidirectional PCA (BDPCA) was proposed in [1] as a generalization of Yang's 2DPCA [21]. Unlike classical PCA, BDPCA does not require to map an image to a high dimensional vector. BDPCA adopts the concept of row and column eigenvectors to directly compute the feature matrix from an image.

Let X1,X2,,XN be a training set of N images. The size of Xi(i=1,,N) is m×n. X¯ is the mean of all training images. The row total scatter matrix Strow and column total scatter matrix Stcol are defined as

Database preparation

For a fair evaluation, a training set and two test sets are prepared using the images from INRIA [13] and PennFudanPed databases [22]. The training set consists of 1237 positive samples and 3891 negative samples, which are obtained from the training set of INRIA database. The first test set contains 589 positive samples and 453 negative images. All these samples are from the test set of INRIA database. The second test set has 423 positive images, which are constructed using the images from the

Conclusions and future works

This paper proposes a new efficient algorithm for pedestrian detection based on reconstructing images using BDPCA. A complete performance analysis was carried out and found that BDPCA vertical-edge descriptor was the most suitable feature for pedestrian detection. A comparison between BDPCA, PCA, and HOG-based methods proved the accuracy and robustness of the BDPCA-based method. Using vertical edge with BDPCA improves the performance of pedestrian detection about 5%. Although the proposed

Conflict of interest statement

None declared.

Acknowledgement

This work was supported by the Industrial Strategic Technology Development Program, 10039149, funded by MKE, Republic of Korea.

Thi-Hai-Binh Nguyen received her BS in Applied Mathematics from Hanoi National University, Vietnam, and MS degree in Information Engineering at Inha University, Korea. She is currently pursuing PhD degree in Information Engineering at Inha University, Korea. Her study includes biometrics, pattern recognition and video surveillance system.

References (22)

  • W. Yang et al.

    Laplacian bidirectional PCA for face recognition

    Neurocomputing

    (2010)
  • W. Zuo, K. Wang, D. Zhang, Bi-directional PCA with assembled matrix distance metric, in: IEEE International Conference...
  • Q. Tian, H. Sun, Y. Luo, D. Hu, Nighttime pedestrian detection with a normal camera using svm classifier, in: Advances...
  • D. Gavrila, Pedestrian detection from a moving vehicle, in: Sixth European Conference on Computer Vision-Part II, 2000,...
  • S. Oh, S. Kim, S. Oh, Pedestrian detection with convolutional neural networks, pedestrian collision warning systems...
  • M. Szarvas, A. Yoshizawa, M. Yamamoto, J. Ogata, Pedestrian detection with convolutional neural networks, in: IEEE...
  • L. Malagon-Borja et al.

    Object detection using image reconstruction with PCA

    Image and Vision Computing

    (2007)
  • C. Papageorgiou et al.

    A trainable system for object detection

    International Journal of Computer Vision

    (2000)
  • A. Mohan et al.

    Example-based object detection in images by components

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2001)
  • M. Enzweiler et al.

    Monocular pedestrian detectionsurvey and experiments

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2009)
  • M. Jones, P. Viola, Detecting pedestrians using patterns of motion and appearance, in: IEEE International Conference on...
  • Cited by (0)

    Thi-Hai-Binh Nguyen received her BS in Applied Mathematics from Hanoi National University, Vietnam, and MS degree in Information Engineering at Inha University, Korea. She is currently pursuing PhD degree in Information Engineering at Inha University, Korea. Her study includes biometrics, pattern recognition and video surveillance system.

    Hakil Kim received BS degree in Control & Instrumentation Engineering from Seoul National University, Korea, in 1983, and MS and PhD degrees in Electrical and Computer Engineering from Purdue University in 1985 and 1990, respectively. He is currently a professor of School of Information & Communication Engineering at Inha University, Incheon, Korea, and a member of Biometrics Engineering Research Center (BERC) at Yonsei University, Seoul, Korea. He has been actively participating in the WG5 (Testing and Reporting) of ISO/IEC JTC1-SC37 and ITU-T/SG17 WP2/Q.9 Telebiometrics as a Rapporteur.

    1

    Tel.: +84 973 222528.

    View full text