Skip to main content
SpringerLink
Log in

Faster-FIIS-GMS: a novel object detection framework for instance search

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Visual sensors are widely deployed in many promising detection fields, such as precision measurement, visual object detection and search. Instance search is a special object search method based on visual sensor, which is of fundamental importance to many applications, including the area of civil and military. Although a quantity of instance search solutions has been proposed, it remains a critical challenge to provide a real-time, lightweight labeling, easy to deploy and able to simultaneous-object-positioning-and-classification method. To address the challenge, a novel instance search method named Faster-FIIS-GMS is described in this paper. The improved Faster R-CNN with the MRFR (Multi-Receptive Field Residual module) and the APM (Adaptive Proposals Mechanism) is proposed to achieve more accurate and time-saving object detection and background information elimination simultaneously. Next, a novel image matching method named Fast Intensity Initial Screening Algorithm (FIIS) is introduced to filter foreground information in an image preliminarily and then combined with Grid-based Motion Statistics (GMS) to achieve the final searching. The performance of Faster-FIIS-GMS is analyzed and validated by experimental tests in a dataset annotated by manual, and the experimental results demonstrate that the proposed method has great potential in instance search for real scene environment, as it achieves high accuracy and satisfactory real time performance using the general computer. We also provide the comparison to state-of-the-arts to investigate the superiority of the proposed scheme at the end of the work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data availability

The INSTRE datasets analyzed during the current study are available at https://pan.baidu.com/s/1gdEfJ9L

References

  1. Bay H, Ess A, Tuytelaars T et al (2007) Speeded-Up Robust Features (SURF). Comput Vis Image Underst 110(3):346–359

    Article  Google Scholar 

  2. Bian JW, Lin WY, Liu Y et al (2020) GMS: Grid-Based Motion Statistics for fast, ultra-robust feature correspondence. Int J Comput Vis 128(6):1580–1593. https://doi.org/10.1007/s11263-019-01280-3

  3. Christian S, Liu W, Jia YQ et al (2015) Going deeper with convolutions. IEEE Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/CVPR.2015.7298594

  4. Lowe DG (2004) Distinctive Image Features from Scale-Invariant Key-points. Int J Comput Vision 60:91–110

    Article  Google Scholar 

  5. Dou JF, Qin Q, Tu ZM (2018) Robust image matching based on the information of SIFT. Optik 171:850–861

    Article  Google Scholar 

  6. Eva M, Kevin M and Xavier G.i.N, et al, (2018) Saliency weighted convolutional features for instance search. Proceedings of the International Conference on Content-Based Multimedia Indexing 1–6

  7. Huang X, Wan X and Peng DF, (2020) “Robust Feature Matching with Spatial Smoothness Constraints,” Remote Sensing 12 https://doi.org/10.3390/rs12193158

  8. Huang YF and Hsieh YS, “Image retrieval based on AND/OR-construction models,” Multimedia Tools and Applications 79 37–38 27293–27320

  9. Jimenez A, Jose MA, Xavier G.i.N (2017) Class-weighted convolutional features for visual instance search. arXiv: 1707 02581

  10. Leng JX, Liu Y (2019) An enhanced SSD with feature fusion and visual reasoning for object detection. Neural Comput Appl 31(10):6459–6558

    Article  Google Scholar 

  11. Li ZL, Xu K, Xie JF et al (2020) Deep Multiple Instance Convolutional Neural Networks for Learning Robust Scene Representations. IEEE Trans Geosci Remote Sens 58(5):3685–3702

    Article  Google Scholar 

  12. Lin J, Zhu Y, Zhao WL (2021) Instance search based on weakly supervised feature learning Neurocomputing 424 117 124

  13. Mei SH, Min WQ, Duan H et al (2019) Instance-level object retrieval via deep region CNN. Multimedia tools and applications 78(10):132471–213261

    Article  Google Scholar 

  14. Nair LR, Subramaniam K and Venkatesan GKDP, et al., “An effective image retrieval system using machine learning and fuzzy c- means clustering approach,” Multimedia Tools and Applications 79 15–16 10123–10140

  15. Redmon J and Farhadi A, (2017) “YOLO9000: Better, Faster, Stronger,” 30th IEEE Conference on Computer Vision and Pattern Recognition 6517–6525 https://doi.org/10.1109/CVPR.2017.690

  16. Ren SQ, He KM, Girshick R et al (2017) Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 39(6):1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031

    Article  Google Scholar 

  17. Rublee E, Rabaud V, Konolige K et al (2011) ORB: An efficient alternative to SIFR or SURF. Procceedings 58(11):2564–2571

    Google Scholar 

  18. Saleem S, Bais A, Sablatnig R (2016) Towards feature points-based image matching between satellite imagery and aerial photographs of agriculture land. Computers and electronics agriculture 126:12–20. https://doi.org/10.1016/j.compag.2016.05.005

    Article  Google Scholar 

  19. Salvador A, Giro-I-Nieto X, Marques F, et al. (2016) Faster R-CNN features for instance search. IEEE Conference on Computer Vision and Pattern Recognition Workshops 394–401

  20. Uzyildrim FE, Ozuysal M (2016) Instance detection by keypoint matching beyond the nearest neighbor. SIViP 10(8):1527–1534

    Article  Google Scholar 

  21. Wan JW, Niu L, Bai B et al (2020) Graph Regularized Deep Discrete Hashing for Multi-Label Image Retrieval. IEEE Signal Process Lett 27:1994–1998

    Article  Google Scholar 

  22. Yannis K, Clayton M and Simon O, (2016) “Cross-Dimensional Weighting for Aggregated Deep Convolutional Features,” Proceedings of the European Conference on Computer Vision 9–16

  23. Yang Y, Chen Z, Li XL et al (2020) Robust template matching with large angle localization. Neurocomputing 395:495–504

    Article  Google Scholar 

  24. Zhang LG (2020) Shen Zhou and Li YB et al, “Image object detection and semantic segmentation based on convolutional neural network.” Neural Comput Appl 32(7):1949–1958

    Article  Google Scholar 

  25. Zhang SZ, Cheng D, Gong YH et al (2018) Pedestrian search in surveillance videos by learning discriminative deep features. Neurocomputing 283:120–128

    Article  Google Scholar 

  26. Zhang Y, Feng Y and Liu D et al, (2020) “FRWCAE: joint Faster R-CNN and Wasserstein convolutional auto-encoder for instance retrieval,” Applied Intelligence 50 7 1 14

  27. Zhou JM, Cheng X and Han RZ, et al, (2020) “Image Precise Matching With Illumination Robust in Vehicle Visual Navigation,” IEEE Access 8 0.1109/ACCESS.2020.2994542

Download references

Funding

This work was supported in part by Key R&D program of Jiangsu Province under Grant BE2021679, Key R & D and Transformation program of Qinghai Province under Grant 2022-QY-208, National Disabled Persons' Federation project under Grant 2021CDPFAT-26.

Author information

Authors and Affiliations

  1. School of Instrument Science & Engineering, Southeast University, Nanjing, 210096, China

    Xiaochen Liu & Tao Zhang

  2. Key Laboratory of Micro-Inertial Instrument and Advanced Navigation Technology, Ministry of Education, Southeast University, Nanjing, 210096, China

    Xiaochen Liu & Tao Zhang

  3. Key Laboratory of Instrumentation Science & Dynamic Measurement, Ministry of Education, North University of China, Taiyuan, 030051, China

    Chong Shen

Authors
  1. Xiaochen Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chong Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tao Zhang.

Ethics declarations

Confict of interest

The authors declare that they have no conficts of interest.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Zhang, T. & Shen, C. Faster-FIIS-GMS: a novel object detection framework for instance search. Multimed Tools Appl 82, 46939–46960 (2023). https://doi.org/10.1007/s11042-023-15616-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15616-2

Keywords

Search

Navigation