Elsevier

Neurocomputing

Volume 257, 27 September 2017, Pages 97-103
Neurocomputing

A model for fine-grained vehicle classification based on deep learning

https://doi.org/10.1016/j.neucom.2016.09.116Get rights and content

Abstract

A model for fine-grained vehicle classification based on deep learning is proposed to handle complicated transportation scene. This model comprises of two parts, vehicle detection model and vehicle fine-grained detection and classification model. Faster R-CNN method is adopted in vehicle detection model to extract single vehicle images from an image with clutter background which may contains serval vehicles. This step provides data for the next classification model. In vehicle fine-grained classification model, an image contains only one vehicle is fed into a CNN model to produce a feature, then a joint bayesian network is used to implement the fine-grained classification process. Experiments show that vehicle’s make and model can be recognized from transportation images effectively by using our method. Furthermore,in order to build a large scale database easier, this paper comes up with a novel network collaborative annotation mechanism.

Introduction

With the promotion of Intelligent Transportation System(ITS) in intelligent city, core technologies applied in ITS develop rapidly and have been updated constantly. In 1970s, only magnetic coils were used to detect vehicles, but now, other technologies like radar, ultrasonic, infrared rays and video image are very popular in practice [1]. As more and more digital video surveillances have been equipped to transportation roads, so visual vehicle detection methods have become research issues of computer vision scientists recently [2].

As an application domain of object detection, vehicle detection plays an important role in ITS, unmanned intelligent car and public security [3]. After detection, we can further classify them in more detail, if applied in public security, it can helps to arrest criminals quickly. Even if the criminals drive away, according to car make, model, color and plate number, we can start all the cameras in the city, which can automatically detect, recognize and locate the car. In this scene, fine-grained classification of vehicle is indispensable.

But in fact, object’s intra-class difference is subtle, even sometimes intra-class difference is bigger than inter-class [4] [5], so the research subject of fine-grained classification is very challenging, which can advance the development of face detection [6], action recognition [7] and automatic scene description [8] and so on.

If fine-grained classification of vehicle is applied in transportation and public security, we can acquire more meta information like vehicle make, model, logo, production year, max speed and acceleration and so on [9]. By acquiring these information dynamically, we can build a large intelligent transportation system that can monitor the whole city’s road. Further, we can analyse the vehicles on the road at different time to find the discipline of people’s going out, then we can schedule transportation rules accordingly, these will make cities more smart and intelligent.

Section snippets

Related work

In this paper we focus on three issues, they are how to build a large scale vehicle dataset; how to detect vehicles in natural images; how to fine-grained classify vehicles.

Overall architecture

Natural transportation images usually contain uncertain number of vehicles. If we want to do fine-grained classification on these images, we must first extract all the vehicles. And then, for these extracted images, we use a CNN model to compute features, with which we can classify vehicles easily. The overall architecture is as Fig. 1. This fine-grained vehicle classification model takes an original image with complicated background as input, which is first fed into the vehicle detection

Implementation detail

To implement effective fine-grained vehicle classification by using deep learning method, three following question should be solved. (1) How to build a large scale dataset suitable for fine-grained vehicle classification. (2) In what way can we get detect vehicles in images with cluttered background, and then extract them to provide clean input to the subsequent classification model. (3)How to recognize different vehicle key parts first, and then joint all parts together to make a fine-grained

Vehicle fine-grained classification model

Vehicle fine-grained classification model use an image which contains only a single vehicle as an input to another classification CNN model to generate features, then with a joint bayesian network, the contained vehicle will be correctly classified [54]. As shown in Fig. 9. CNN feature and joint bayesian network have been proved to be effective in face recognition as described in [55]. In this paper, we treat a vehicle as two parts, one part is the inter-difference of different vehicle models,

Experimental result

Dataset used in this paper consists of three parts, whole vehicle dataset, vehicle parts dataset and non-vehicle dataset. The whole dataset is used to train faster R-CNN to build a vehicle detection model, including 73 different vehicle makes and 208 different vehicle models, 174,008 images in all; vehicle parts dataset is employed to train fine-grained classification network which can classify different models of vehicles, this dataset totally contains 33,023 images of headlamp, tail light,

Conclusion

In this paper, a faster R-CNN based vehicle detection model is first used to detect vehicles in images with complicated background, and then the detection result is fed into a fine-grained vehicle classification model which can classify vehicles in more detail. While the vehicle detection model cannot detect all the vehicles in images at 100 percentage, and sometimes it will even deem non-vehicle regions as vehicles, so that has influence on the accuracy of fine-grained classification. At the

Acknowledgment

This work is supported by Natural Science Foundation of Fujian Province of China(Grant No.2013J05103 and No. 2016J01325 and No. 2015J05015) and High-level Personnel of Support Program of Xiamen University of Technology (Grant NO. YKJ14014R) and B Program of Education Department of Fujian Province(Grant No.JB13152).

Shaoyong Yu, Male, a PhD student in Xiamen University. His research interests lie in the areas of computer vision and deep learning. Contact him at [email protected].

References (55)

  • S.A.A. Shah et al.

    Iterative deep learning for image set based face and object recognition

    Neurocomputing

    (2016)
  • GeQ. et al.

    Fire localization based on range-range-range model for limited interior space

    IEEE Trans. Instrument. Meas.

    (2014)
  • K. Alex et al.

    Imagenet classification with deep convolutional neural networks

    Adv. Neural Inf. Process. Syst.

    (2012)
  • M.D. Zeiler et al.

    Visualizing and understanding convolutional networks

    ECCV

    (2014)
  • P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, Overfeat: integrated recognition, localization and detection...
  • A. Chatfield, K. Simonyan, A.Zisserman, Return of the devil in the details: delving deep into convolutional nets,...
  • HongC. et al.

    Multimodal deep autoencoder forhuman pose recovery

    IEEE Trans. Image Process.

    (2015)
  • LinD. et al.

    Deep lac:deep localization ,alignment and classification for fine-grained recognition

    CVPR

    (2015)
  • G. Ross et al.

    Rich feature hierarchies for accurate object detection and semantic segmentation

    CVPR

    (2014)
  • Z.W. Y, W. X, S. M, Generic object detection with dense neural patterns and regionlets, Eprint Arxiv (2014)...
  • Razavian et al.

    Cnn features off-the-shelf: an astounding baseline for recognition

    Computer Vision and Pattern Recognition

    (2014)
  • R.M. Taigman Y

    Deepface: closing the gap to human-level performance in face verification

    Computer Vision and Pattern Recognition

    (2014)
  • R.M. Zhang N

    Deepface: closing the gap to human-level performance in face verification

    Computer Vision and Pattern Recognition

    (2014)
  • G. Y et al.

    Multi-scale orderless pooling of deep convolutional activation features

    Lecture Notes in Computer Science

    (2014)
  • L. Fei-Fei et al.

    One-shot learning of object categories

    PAMI

    (2006)
  • G. Grifn, A. Holub, P. Perona, Caltech-256 object category dataset, Techical Report 7694,Caltech(2007)...
  • J. Shotton et al.

    Textonboost: joint appearance, shape and context modeling for multi-class object recognition and segmentation

    ECCV

    (2006)
  • Cited by (95)

    • Category-consistent deep network learning for accurate vehicle logo recognition

      2021, Neurocomputing
      Citation Excerpt :

      Owing to the increasing demand for automatic identification of vehicles, computer-aided intelligent technologies play an important role in improving the performance of various transportation systems [1,2]. The detection of vehicle license plates and the recognition of vehicle manufacturers are crucial components in these systems [3,4]. In the recognition process, the vehicle logo is certainly the clearest indicator of a vehicle’s manufacturer [5], as vehicle logos with the same maker usually have a unique and standard visual design.

    • Bangladeshi Native Vehicle Classification Employing YOLOv8

      2024, Communications in Computer and Information Science
    View all citing articles on Scopus

    Shaoyong Yu, Male, a PhD student in Xiamen University. His research interests lie in the areas of computer vision and deep learning. Contact him at [email protected].

    Yun Wu, female, PhD. She received her PhD from Xiamen University in 2007. Her research interests lie in the areas of artificial intelligence and big data. His scientific contribution to the AI has more to do with soft computing and the clustering algorithms.

    Wei Li is an associate professor in the School of Computer and Information Engineering at Xiamen University of Technology. His research interests include artificial intelligence, computer graphics. He has a Ph.D. in Basic Theory of Artificial Intelligence from Xiamen University. Contact him at [email protected].

    Song Zhijun, Male, PhD. He received his PhD from Xiamen University in 2013. His research interests lie in the areas of artificial intelligence and big data. His scientific contribution to the AI has more to do with machine consciousness and the logic of mental self-reflection.Contact him at [email protected].

    Wenhua Zeng is a professor in the Department of Cognitive Science at Xiamen University. His research interests include neural network, grid computing and embedded system. He has a Ph.D. in Industry Automation from Zhejiang University in 1986. Contact him at [email protected].

    View full text