Skip to main content
Springer Nature Link
Log in

Faster R-CNN based on frame difference and spatiotemporal context for vehicle detection

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Vehicle detection is a very important part in intelligent transportation system. In order to improve the detection speed without sacrificing the accuracy, this paper propose an improved Faster R-CNN algorithm based on frame difference and spatiotemporal context to realize real-time detection of vehicles. We improve the training and testing speed of Faster R-CNN by improving the RPN module. Different from the original Faster R-CNN’s anchor based strategy, the inter frame difference in this paper is mainly used to extract the region of interest of the target. And we introduce spatiotemporal context information to assist our detection. Among them, the spatial context is formed by adding the association information outside the target area to enhance the expression of target information and improve the accuracy of target detection, while the anchor filtering of the original Faster R-CNN can be carried out by integrating the temporal context information, so as to improve the detection efficiency. This improved RPN has strong pertinence to the detection of moving vehicles. This strategy not only makes our branch network parallelly process with the original Faster R-CNN, but also avoids the extra time consumption caused by the addition of algorithm. More importantly, it can be simply added to the existing Faster R-CNN based application system without algorithm adjustment or network retraining. Experimental results show that the proposed method has high detection efficiency and low sensitivity to background changes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data availability

The data that support the findings in this study are available from the corresponding author upon reasonable request.

References

  1. Wang, Y., Jiang, Z., Li, Y., Hwang, J.N., Liu, H.: Rodnet: A real-time radar object detection network cross-supervised by camera-radar fused object 3d localization. IEEE J. Selec. Topics Signal Process. PP99, 1–1 (2021)

    Google Scholar 

  2. Yang, T., Li, J.: Remote sensing image object detection based on improved yolov3 in deep learning environment. J. Circuits Syst. Computers, 32(15). (2023)

  3. Ma, Y., Chai, L., Jin, L., Yu, Y., Yan, J.: Avs-yolo: Object detection in aerial visual scene. Int. J. Pattern Recognit. Artif. Intell., 36(01) (2022)

  4. Yamamoto, M., Sultana, R., Ohashi, G.: Nighttime traffic sign and pedestrian detection using refinedet with time-series information. IEEJ Trans. Electr. Electron. Eng. 18(3), 408–417 (2023)

    Article  Google Scholar 

  5. Upadhye, S., Neelakandan, S., Thangaraj, K., Babu, D.V., Arulkumar, N., Qureshi, K.: Modeling of real time Traffic flow Monitoring System Using deep Learning and Unmanned Aerial Vehicles. Journal of mobile multimedia (2023)

  6. Liu, T., Du, S., Liang, C., Zhang, B., Feng, R.: A novel multi-sensor fusion based object detection and recognition algorithm for intelligent assisted driving. IEEE Access., PP(99), 1–1. (2021)

  7. Kim, S.W., Ko, K., Ko, H., Leung, V.C.M.: Edge network-assisted real-time object detection framework for autonomous driving. (2020)

  8. Hao, J.G.G.: A multi-target corner pooling-based neural network for vehicle detection. Neural computing & applications, 32(18). (2020)

  9. Guo, Y., Liang, R.L., Cui, Y.K., Zhao, X.M., Meng, Q.: A domain-adaptive method with cycle perceptual consistency adversarial networks for vehicle target detection in foggy weather. IET Intel. Transport Syst.(7), 16. (2022)

  10. Hao, L.Y., Li, J., Guo, G.: A multi-target Corner pooling-based Neural Network for Vehicle Detection, vol. 18. Springer Science and Business Media LLC( (2020)

  11. Gebreel, A.Y.: An Overview of Machine Learning, deep Learning, and Artificial Intelligence. OSF Preprints (2023)

  12. Cances, J.P.: An overview on deep learning techniques for video compressive sensing. Appl. Sci., 12. (2022)

  13. Edara, D.C., Sistla, V., Kolli, V.K.K.: Deep finesse network model with multichannel syntactic and contextual features for target-specific sentiment classification. Appl. Intelligence: Int. J. Artif. Intell. Neural Networks Complex. Problem-Solving Technol. 8, 52 (2022)

    Google Scholar 

  14. Compton, S., Riley, L.K.: Overview detection of infectious agents in laboratory rodents: traditional and molecular techniques. (2022)

  15. Majumder, U.K., Blasch, E.P., Garren, D.A.: Deep learning for radar and communications automatic target recognition. Microw. J. 1, 65 (2022)

    Google Scholar 

  16. Ryzhkov, L., Sushchenko, O.: Approach to Positioning of Target using UAV Equipment. 2020 IEEE 6th International Conference on Methods and Systems of Navigation and Motion Control (MSNMC). IEEE. (2020)

  17. Frantisek Jabloncík, Harga, L., Koniar, D., Bulava, J.: Svm texture classification and r-cnn approach on medical image. 2022 ELEKTRO (ELEKTRO), 1–4. (2022)

  18. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Regionbased convolutional networks for accurate object detection and segmentation. TPAMI. 7(5), 8 (2015)

    Google Scholar 

  19. Nazir, A., Wani, M.: You only look once - object detection models: a review. 2023 10th International Conference on Computing for Sustainable Global Development (INDIACom), 1088–1095. (2023)

  20. Beideman, C., Chandrasekaran, K., Xu, C.: Multicriteria cuts and size-constrained k-cuts in hypergraphs. Mathematical Programming. (2023)

  21. Yin, S., Li, H.: Hot region selection based on selective search and modified fuzzy c-means in remote sensing images. IEEE J. Sel. Top. Appl. Earth Observations Remote Sens. 13, 5862–5871 (2020)

    Article  Google Scholar 

  22. He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in Deep Convolutional Networks for Visual Recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37, 1904–1916 (2015)

    Article  Google Scholar 

  23. Girshick, R.: Fast r-cnn. Computer Science (2015)

  24. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)

    Article  Google Scholar 

  25. Tobita, K., Mima, K.: Azimuth angle detection method combining akaze features and optical flow for measuring movement accuracy. J. Robot. Mechatron. (2023)

  26. Sobral, A.: A Vacavant.A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Computer Vision and Image Understanding,pp.4–21,2014

  27. Xu, F., Li, G.: Feature extraction algorithm of basketball trajectory based on the background difference method. Mathematical Problems in Engineering, 2022. (2022)

  28. Yuan Guowu, C., Zhiqiang, G.J.: A moving object detection Algorithm based on a combination of Optical Flow and three-frame difference. J. Chin. Comput. Syst.,3(3):668–6712013

  29. Li, C., Zhang, Y., Gao, G., Liu, Z., Liao, L.: Context-aware cross-level Attention Fusion Network for Infrared Small Target Detection. Journal of Applied Remote Sensing (2022)

  30. Spedalieri, G., Pirandola, S.: Performance of coherent-state Quantum Target Detection in the Context of Asymmetric Hypothesis Testing. IET Quantum Communication (2021)

  31. Alom, M.Z., Taha, T.M., Yakopcic, C., Westberg, S., Asari, V.K.: The History Began from Alexnet. a comprehensive survey on deep learning approaches (2018)

  32. Everingham, M., Winn, J.: The PASCAL Visual object classes challenge 2007 (VOC2007) development Kit[J]. Int. J. Comput. Vision. 111(1), 98–136 (2006)

    Article  Google Scholar 

  33. Shao, F., Wang, X., Meng, F., Zhu, J., Wang, D., Dai, J., Improved Faster, R.-C.N.N.: Traffic sign detection based on a second region of interest and highly possible regions proposal network. Sensors (Basel). May 1719(10):2288. (2019)

  34. Mittal, U., Chawl, P., Tiwari, R.: EnsembleNet: A hybrid approach for vehicle detection and estimation of traffic density based on faster R-CNN and YOLO models. Neural Comput. Appl. 35(6), 4755–4774 (2023)

    Article  Google Scholar 

  35. Zhang, F., Li, C., Yang, F.: Vehicle detection in urban traffic surveillance images based on convolutional neural networks with. Feature Concatenation[J] Sens. 19(3) (2019). https://doi.org/10.3390/s19030594

  36. Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv arXiv, (2022)

  37. Tabassum, N., El-Sharkawy, M.: Vehicle detection in adverse Weather: A multi-head attention Approach with Multimodal Fusion. J. Low Power Electron. Appl. 14(2), 23 (2024). https://doi.org/10.3390/jlpea14020023

    Article  Google Scholar 

Download references

Funding

This research was funded by the National Natural Science Foundation of China (grant number: 61671470).

Author information

Authors and Affiliations

  1. College of Field Engineering, Army Engineering University of PLA, Nanjing, 210007, China

    Heng Zhang, Faming Shao, Weijun Chu, Juying Dai, Xiangpo Zhang & Congcong Gong

  2. College of National Defense Engineering, Army Engineering University of PLA, Nanjing, 210007, China

    Xingde Li

Authors
  1. Heng Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Faming Shao
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Weijun Chu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Juying Dai
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Xingde Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Xiangpo Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Congcong Gong
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

All authors have contributed equally.

Corresponding author

Correspondence to Faming Shao.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, H., Shao, F., Chu, W. et al. Faster R-CNN based on frame difference and spatiotemporal context for vehicle detection. SIViP 18, 7013–7027 (2024). https://doi.org/10.1007/s11760-024-03370-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-024-03370-3

Keywords