A model for fine-grained vehicle classification based on deep learning
Introduction
With the promotion of Intelligent Transportation System(ITS) in intelligent city, core technologies applied in ITS develop rapidly and have been updated constantly. In 1970s, only magnetic coils were used to detect vehicles, but now, other technologies like radar, ultrasonic, infrared rays and video image are very popular in practice [1]. As more and more digital video surveillances have been equipped to transportation roads, so visual vehicle detection methods have become research issues of computer vision scientists recently [2].
As an application domain of object detection, vehicle detection plays an important role in ITS, unmanned intelligent car and public security [3]. After detection, we can further classify them in more detail, if applied in public security, it can helps to arrest criminals quickly. Even if the criminals drive away, according to car make, model, color and plate number, we can start all the cameras in the city, which can automatically detect, recognize and locate the car. In this scene, fine-grained classification of vehicle is indispensable.
But in fact, object’s intra-class difference is subtle, even sometimes intra-class difference is bigger than inter-class [4] [5], so the research subject of fine-grained classification is very challenging, which can advance the development of face detection [6], action recognition [7] and automatic scene description [8] and so on.
If fine-grained classification of vehicle is applied in transportation and public security, we can acquire more meta information like vehicle make, model, logo, production year, max speed and acceleration and so on [9]. By acquiring these information dynamically, we can build a large intelligent transportation system that can monitor the whole city’s road. Further, we can analyse the vehicles on the road at different time to find the discipline of people’s going out, then we can schedule transportation rules accordingly, these will make cities more smart and intelligent.
Section snippets
Related work
In this paper we focus on three issues, they are how to build a large scale vehicle dataset; how to detect vehicles in natural images; how to fine-grained classify vehicles.
Overall architecture
Natural transportation images usually contain uncertain number of vehicles. If we want to do fine-grained classification on these images, we must first extract all the vehicles. And then, for these extracted images, we use a CNN model to compute features, with which we can classify vehicles easily. The overall architecture is as Fig. 1. This fine-grained vehicle classification model takes an original image with complicated background as input, which is first fed into the vehicle detection
Implementation detail
To implement effective fine-grained vehicle classification by using deep learning method, three following question should be solved. (1) How to build a large scale dataset suitable for fine-grained vehicle classification. (2) In what way can we get detect vehicles in images with cluttered background, and then extract them to provide clean input to the subsequent classification model. (3)How to recognize different vehicle key parts first, and then joint all parts together to make a fine-grained
Vehicle fine-grained classification model
Vehicle fine-grained classification model use an image which contains only a single vehicle as an input to another classification CNN model to generate features, then with a joint bayesian network, the contained vehicle will be correctly classified [54]. As shown in Fig. 9. CNN feature and joint bayesian network have been proved to be effective in face recognition as described in [55]. In this paper, we treat a vehicle as two parts, one part is the inter-difference of different vehicle models,
Experimental result
Dataset used in this paper consists of three parts, whole vehicle dataset, vehicle parts dataset and non-vehicle dataset. The whole dataset is used to train faster R-CNN to build a vehicle detection model, including 73 different vehicle makes and 208 different vehicle models, 174,008 images in all; vehicle parts dataset is employed to train fine-grained classification network which can classify different models of vehicles, this dataset totally contains 33,023 images of headlamp, tail light,
Conclusion
In this paper, a faster R-CNN based vehicle detection model is first used to detect vehicles in images with complicated background, and then the detection result is fed into a fine-grained vehicle classification model which can classify vehicles in more detail. While the vehicle detection model cannot detect all the vehicles in images at 100 percentage, and sometimes it will even deem non-vehicle regions as vehicles, so that has influence on the accuracy of fine-grained classification. At the
Acknowledgment
This work is supported by Natural Science Foundation of Fujian Province of China(Grant No.2013J05103 and No. 2016J01325 and No. 2015J05015) and High-level Personnel of Support Program of Xiamen University of Technology (Grant NO. YKJ14014R) and B Program of Education Department of Fujian Province(Grant No.JB13152).
Shaoyong Yu, Male, a PhD student in Xiamen University. His research interests lie in the areas of computer vision and deep learning. Contact him at [email protected].
References (55)
- et al.
Image classification by visual bag-of-words refinement and reduction
Neurocomputing
(2016) - et al.
Incorporating visual adjectives for image classification
Neurocomputing
(2016) - et al.
A massively parallel pipelined reconfigurable design for m-pln based neural networks for efficient image classification
Neurocomputing
(2016) - et al.
Cubature information filters with correlated noises and their applications in decentralized fusion
Signal Process.
(2014) - et al.
Semantic preserving distance metric learning and applications
Inf. Sci.
(2014) - et al.
A novel bilogically inspired elm-based network for image recognition
Neurocomputing
(2016) - et al.
Discriminant structure embedding for image recognition
Neurocomputing
(2016) Special issue on big data driven intelligent transportation systems
Neurocomputing
(2016)- et al.
Learning torank using user clicks and visual features for image retrieval
IEEE Trans. Cybern.
(2015) - et al.
Analysis on strong tracking filtering for linear dynamic systems
Math. Probl. Eng.
(2015)
Iterative deep learning for image set based face and object recognition
Neurocomputing
Fire localization based on range-range-range model for limited interior space
IEEE Trans. Instrument. Meas.
Imagenet classification with deep convolutional neural networks
Adv. Neural Inf. Process. Syst.
Visualizing and understanding convolutional networks
ECCV
Multimodal deep autoencoder forhuman pose recovery
IEEE Trans. Image Process.
Deep lac:deep localization ,alignment and classification for fine-grained recognition
CVPR
Rich feature hierarchies for accurate object detection and semantic segmentation
CVPR
Cnn features off-the-shelf: an astounding baseline for recognition
Computer Vision and Pattern Recognition
Deepface: closing the gap to human-level performance in face verification
Computer Vision and Pattern Recognition
Deepface: closing the gap to human-level performance in face verification
Computer Vision and Pattern Recognition
Multi-scale orderless pooling of deep convolutional activation features
Lecture Notes in Computer Science
One-shot learning of object categories
PAMI
Textonboost: joint appearance, shape and context modeling for multi-class object recognition and segmentation
ECCV
Cited by (95)
Category-consistent deep network learning for accurate vehicle logo recognition
2021, NeurocomputingCitation Excerpt :Owing to the increasing demand for automatic identification of vehicles, computer-aided intelligent technologies play an important role in improving the performance of various transportation systems [1,2]. The detection of vehicle license plates and the recognition of vehicle manufacturers are crucial components in these systems [3,4]. In the recognition process, the vehicle logo is certainly the clearest indicator of a vehicle’s manufacturer [5], as vehicle logos with the same maker usually have a unique and standard visual design.
Bangladeshi Native Vehicle Classification Employing YOLOv8
2024, Communications in Computer and Information ScienceVehicle Type Detection and Classification Using Enhanced ReliefF Algorithm and Long Short-Term Memory Network
2023, Journal of The Institution of Engineers (India): Series BVehicle Detection in Indian Traffic Using an Anchor-Free Object Detector
2023, Lecture Notes in Networks and SystemsA Vehicle Classification System for Intelligent Transport System using Machine Learning in Constrained Environment
2023, International Journal of Advanced Computer Science and Applications
Shaoyong Yu, Male, a PhD student in Xiamen University. His research interests lie in the areas of computer vision and deep learning. Contact him at [email protected].
Yun Wu, female, PhD. She received her PhD from Xiamen University in 2007. Her research interests lie in the areas of artificial intelligence and big data. His scientific contribution to the AI has more to do with soft computing and the clustering algorithms.
Wei Li is an associate professor in the School of Computer and Information Engineering at Xiamen University of Technology. His research interests include artificial intelligence, computer graphics. He has a Ph.D. in Basic Theory of Artificial Intelligence from Xiamen University. Contact him at [email protected].
Song Zhijun, Male, PhD. He received his PhD from Xiamen University in 2013. His research interests lie in the areas of artificial intelligence and big data. His scientific contribution to the AI has more to do with machine consciousness and the logic of mental self-reflection.Contact him at [email protected].
Wenhua Zeng is a professor in the Department of Cognitive Science at Xiamen University. His research interests include neural network, grid computing and embedded system. He has a Ph.D. in Industry Automation from Zhejiang University in 1986. Contact him at [email protected].