Skip to main content

Advertisement

SpringerLink
Log in

Exploring Multi-Loss Learning for Multi-View Fine-Grained Vehicle Classification

  • Regular paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Electronic Toll Collection is the combination of multiple components, either technical or operational, organized to optimize system efficiency for specific requirements. Four main components constitute an ETC system: Automated Vehicle Identification (AVI), Automated Vehicle Classification (AVC), Customer Service and Violation Enforcement. The AVI involves the identification of vehicles through the transmission of a unique identifier between an in-vehicle device and a tollbooth or roadside reader. To strengthen the reliability of this process, we propose a computer vision solution applied to AVI with subscription/membership. A camera system is set up to perform vehicle verification by extracting attributes of the vehicle to compare with those found in the membership. We focus on solving vehicle make and model classification by developing a fine-grained vehicle classification system that exploits the multi-camera composition of the system by powering a convolutional neural network with multiple views of the vehicle. We propose a multi-view network that extracts features from multiple views and combines them with late fusion to classify the make and model of the vehicle. We also propose a strategy to give each independent view a contribution to network learning. The presented evaluations show that using information from different views of a vehicle improves the classification performance of the make and model, especially in challenging tolling scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Sánchez, H.C., Parra, N.H., Alonso, I.P., Nebot, E., Fernández-Llorca, D.: Are we ready for accurate and unbiased fine-grained vehicle classification in realistic environments? IEEE Access 9, 116338 (2021). https://doi.org/10.1109/ACCESS.2021.3104340

    Article  Google Scholar 

  2. De Oliveira, I.O., Laroca, R., Menotti, D., Fonseca, K.V.O., Minetto, R.: Vehicle-Rear: A new dataset to explore feature fusion for vehicle identification using convolutional neural networks. IEEE Access 9, 101065 (2021). https://doi.org/10.1109/access.2021.3097964

    Article  Google Scholar 

  3. Wang, C., Cheng, J., Wang, Y., Qian, Y.: Hierarchical scheme for vehicle make and model recognition. Transportation Research Record: Journal of the Transportation Research Board p 036119812110197. https://doi.org/10.1177/03611981211019743 (2021)

  4. Berg, T., Liu, J., Lee, S.W., Alexander, M.L., Jacobs, D.W., Belhumeur, P.N.: Birdsnap: Large-scale fine-grained visual categorization of birds. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2019–2026 (2014)

  5. Van Horn, G., Branson, S., Farrell, R., Haber, S., Barry, J., Ipeirotis, P., Perona, P., Belongie, S.: Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 595–604 (2015)

  6. Khosla, A., Jayadevaprakash, N., Yao, B., Fei-Fei, L.: Novel Dataset for Fine-Grained Image Categorization : Stanford Dogs. In: First Workshop on Fine-Grained Visual Categorization, CVPR 2011 (2011)

  7. Nilsback, M., Zisserman, A.: Automated flower classification over a large number of classes. In: 2008 Sixth Indian Conference on Computer Vision, Graphics Image Processing, pp. 722–729 (2008)

  8. Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3d object representations for fine-grained categorization. In: 2013 IEEE International Conference on Computer Vision Workshops, pp. 554–561 (2013)

  9. Yang, L., Luo, P., Loy, C.C., Tang, X.: A large-scale car dataset for fine-grained categorization and verification. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3973–3981 (2015)

  10. Yan, K., Tian, Y., Wang, Y., Zeng, W., Huang, T.: Exploiting multi-grain ranking constraints for precisely searching visually-similar vehicles. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 562–570 (2017)

  11. Seeland, M., Mäder, P.: Multi-view classification with convolutional neural networks. PLOS ONE 16(1), 1 (2021). https://doi.org/10.1371/journal.pone.0245230

    Article  Google Scholar 

  12. Touvron, H., Vedaldi, A., Douze, M., Jegou, H.: Fixing the train-test resolution discrepancy. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp 8252–8262. Curran Associates Inc (2019)

  13. Cubuk, E.D., Zoph, B., Mané, D., Vasudevan, V., Le, Q.V.: Autoaugment: Learning augmentation strategies from data. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 113–123 (2019)

  14. Han, K., Guo, J., Zhang, C., Zhu, M.: Attribute-aware attention model for fine-grained representation learning. In: Proceedings of the 26th ACM International Conference on Multimedia (Association for Computing Machinery, New York, NY, USA), MM ’18, p. 2040–2048. https://doi.org/10.1145/3240508.3240550 (2018)

  15. Ge, W., Lin, X., Yu, Y.: Weakly supervised complementary parts models for fine-grained image classification from the bottom up. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3029–3038 (2019)

  16. Hu, T., Qi, H., Huang, Q., Lu, Y.: See better before looking closer: Weakly supervised data augmentation network for fine-grained visual classification. arXiv:1901.09891 (2019)

  17. Imran, A., Athitsos, V.: Domain adaptive transfer learning on visual attention aware data augmentation for fine-grained visual categorization. In: Bebis, G., Yin, Z., Kim, E., Bender, J., Subr, K., Kwon, B.C., Zhao, J., Kalkofen, D., Baciu, G. (eds.) Advances in Visual Computing, pp 53–65. Springer International Publishing, Cham (2020)

  18. Du, R., Chang, D., Bhunia, A.K., Xie, J., Ma, Z., Song, Y.Z., Guo, J. Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M. (eds.): Fine-grained visual classification via progressive multi-granularity training of jigsaw patches, vol. 2020. Springer International Publishing, Cham (2020)

  19. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. https://doi.org/10.1109/CVPR.2009.5206848 (2009)

  20. Zhuang, P., Wang, Y., Qiao, Y.: Learning attentive pairwise interaction for fine-grained classification. In: AAAI, pp. 13,130–13,137 (2020)

  21. Foret, P., Kleiner, A., Mobahi, H., Neyshabur, B.: Sharpness-aware minimization for efficiently improving generalization. In: International Conference on Learning Representations (2021)

  22. Hanselmann, H., Ney, H.: Elope: Fine-grained visual classification with efficient localization, pooling and embedding. In: The IEEE Winter Conference on Applications of Computer Vision (WACV) (2020)

  23. Zhang, F., Li, M., Zhai, G., Liu, Y.: Multi-branch and multi-scale attention learning for fine-grained visual categorization. In: Okoč, J., Skopal, T., Schoeffmann, K., Mezaris, V., Li, X., Vrochidis, S., Patras, I. (eds.) Modeling, MultiMedia, pp 136–147. Springer International Publishing, Cham (2021)

  24. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is Worth 16X16 words: Transformers for image recognition at scale. In: International Conference on Learning Representations (2021)

  25. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)

  26. Ni, X., Huttunen, H.: Vehicle attribute recognition by appearance: Computer vision methods for vehicle type, make and model classification. J Signal Process Syst 93, 357 (2021)

    Article  Google Scholar 

  27. Hu, Q., Wang, H., Li, T., Shen, C.: Deep cnns with spatially weighted pooling for fine-grained car recognition. IEEE Trans. Intell. Transp. Syst. 18(11), 3147 (2017). https://doi.org/10.1109/TITS.2017.2679114

    Article  Google Scholar 

  28. Elkerdawy, S., Ray, N., Zhang, H.: Fine-grained vehicle classification with unsupervised parts co-occurrence learning. In: Leal-Taixé, L., Roth, S. (eds.) Computer Vision – ECCV 2018 Workshops, pp 664–670. Springer International Publishing, Cham (2019)

  29. Xiang, Y., Fu, Y., Huang, H.: Global topology constraint network for fine-grained vehicle recognition. IEEE Trans. Intell. Transp. Syst. 21(7), 2918 (2020). https://doi.org/10.1109/TITS.2019.2921732

    Article  Google Scholar 

  30. Chen, Z., Ying, C., Lin, C., Liu, S., Li, W.: Multi-view vehicle type recognition with feedback-enhancement multi-branch cnns. IEEE Trans. Circuits Syst. Video Technol. 29(9), 2590 (2019)

    Article  Google Scholar 

  31. Ridnik, T., Lawen, H., Noy, A., Ben Baruch, E., Sharir, G., Friedman, I.: Tresnet: High performance gpu-dedicated architecture. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 1400–1409 (2021)

  32. Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E.: Multi-view convolutional neural networks for 3d shape recognition. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 945–953 (2015)

  33. Li, Y., Yang, M., Zhang, Z.: A survey of multi-view representation learning. IEEE Trans Knowl Data Eng 31(10), 1863 (2019). https://doi.org/10.1109/TKDE.2018.2872063

    Article  Google Scholar 

  34. Jia, K., Lin, J., Tan, M., Tao, D.: Deep multi-view learning using neuron-wise correlation-maximizing regularizers. IEEE Trans. Image Process. 28(10), 5121 (2019). https://doi.org/10.1109/TIP.2019.2912356

    Article  MathSciNet  Google Scholar 

  35. Yan, X., Hu, S., Mao, Y., Ye, Y., Yu, H.: Deep multi-view learning methods: A review. Neurocomputing 448, 106 (2021). https://doi.org/10.1016/j.neucom.2021.03.090. https://www.sciencedirect.com/science/article/pii/S0925231221004768https://www.sciencedirect.com/science/article/pii/S0925231221004768

    Article  Google Scholar 

  36. Shvai, N., Hasnat, A., Meicler, A., Nakib, A.: Accurate classification for automatic vehicle-type recognition based on ensemble classifiers. IEEE Trans. Intell. Transp. Syst. 21(3), 1288 (2020)

    Article  Google Scholar 

  37. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2818–2826 (2016)

Download references

Funding

This work was funded by AtoBe – Mobility Technology S.A. Bruno Silva and Fransisco Rodolfo Barbosa-Anda were supported by ISR Research Grants under R&D project Sistemas de Visão Computacional. This work was also partially supported by Fundação para a Ciência e a Tecnologia (FCT) under the project UIDB/00048/2020

Author information

Author notes

  1. Francisco Rodolfo Barbosa-Anda

    Present address: , Salamanca, Guanajuato, Mexico

Authors and Affiliations

  1. Institute of Systems and Robotics, University of Coimbra, Coimbra, Portugal

    Bruno Silva & Francisco Rodolfo Barbosa-Anda

  2. Institute of Systems and Robotics, Department of Electrical and Computers Engineering, Faculty of Science and Technology, University of Coimbra, Coimbra, Portugal

    Jorge Batista

Authors
  1. Bruno Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francisco Rodolfo Barbosa-Anda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jorge Batista
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Bruno Silva: Conceptualization, Methodology, Data collection, Software, Validation, Investigation, Writing-Original Draft, Writing-Review and Editing, Visualization. Francisco Rodolfo Barbosa-Anda: Methodology, Data collection, Validation, Writing-Original Draft, Writing-Review and Editing. Jorge Batista: Funding, Conceptualization, Validation, Investigation, Supervision, Writing-Review and Editing.

Corresponding author

Correspondence to Bruno Silva.

Ethics declarations

Ethics approval

Not applicable (this article does not contain any studies with human participants or animals performed by any of the authors).

Consent for Publication

All authors have approved the manuscript and agree with its publication on Journal of Intelligent & Robotic Systems.

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Availability of Data and Materials

CompCarsMV sub-dataset is available from the corresponding author on reasonable request.

Consent to participate

Not applicable (this article does not contain any studies with human participants or animals performed by any of the authors).

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by A-to-Be - Mobility Technology, S.A.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Silva, B., Barbosa-Anda, F.R. & Batista, J. Exploring Multi-Loss Learning for Multi-View Fine-Grained Vehicle Classification. J Intell Robot Syst 105, 43 (2022). https://doi.org/10.1007/s10846-022-01626-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-022-01626-z

Keywords

Search

Navigation