Skip to main content
SpringerLink
Log in

Image Retrieval Using Object Semantic Aggregation Histogram

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Simulating primates’ ability to make fine visual discriminations for extracting visual features remains a challenge. To address this issue, a novel method was proposed, which was named the object semantic aggregation histogram. By using the developed method, the mid-level object features and high-level semantic features can be aggregated into a discriminative compact representation for image retrieval. The proposed method includes three main highlights: (1) two adaptive semantic kernels were proposed to bridge the object features and semantic features. It can serve as a connecting link for detailed object representation. (2) Both object and semantic features were aggregated to provide a discriminative representation by depicting the target objects, semantic cognition, and spatial layouts. This approach is consistent with the coarse-to-fine nature of perception in the visual hierarchy. (3) A simple yet generic method was introduced to implement dimensionality reduction and whitening. It can provide a good choice of using various regularizations to decide optimal compact representation. Experiments on benchmark datasets confirmed that the proposed method can effectively improve the retrieval performance in terms of mAP. The mAPs of the proposed method using 128-dimensionality representation were significantly greater than that of the CroW, SBA, DSFH, and DTFH methods by 0.042, 0.035, 0.061, and 0.029 on the Oxford5k dataset and by 0.019, 0.017, 0.083, and 0.034 on the Holidays dataset. The proposed method is regarded as an effective and competitive method of aggregating multiple visual hierarchy features, while no complex handcrafting or training is required.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The author has used third-party data and therefore does not own the data, and those benchmark datasets are public. The code of the proposed method is available on request.

References

  1. Li X, Yang J, Ma J. Recent developments of content-based image retrieval (CBIR). Neurocomputing. 2021;452:675–89.

    Article  Google Scholar 

  2. Kalantidis Y, Mellina C, Osindero S. Cross-dimensional weighting for aggregated deep convolutional features. Proc Eur Conf Comput Vis. 2016;685–701.

  3. Zhu J, Wang J, Pang S, Guan W, Li Z, Li Y, et al. Co-weighting semantic convolutional features for object retrieval. J Vis Commun Image Represent. 2019;62:368–80.

    Article  Google Scholar 

  4. Xu J, Wang C, Qi C, Shi C, Xiao B. Unsupervised semantic-based aggregation of deep convolutional features. IEEE Trans Image Process. 2019;28(2):601–11.

    Article  MathSciNet  MATH  Google Scholar 

  5. Lu F, Liu G-H. Image retrieval using contrastive weight aggregation histograms. Digit Signal Process. 2022;123:103457.

    Article  Google Scholar 

  6. Lu Y, Yin J, Chen Z, Gong H, Liu Y, Wang W, et al. Revealing detail along the visual hierarchy: neural clustering preserves acuity from V1 to V4. Neuron. 2018;98(2):417–28.

    Article  Google Scholar 

  7. Zhou Y, Zeng F-Z, Zhao H-M, et al. Hierarchical visual perception and two-dimensional compressive sensing for effective content-based color image retrieval. Cogn Comput. 2016;8:877–89.

    Article  Google Scholar 

  8. Krizhevsky A, Sutskever I, Hinton G.E. ImageNet classification with deep convolutional neural networks. Proc Int Conf Neural Inf Process Syst. 2012;1097–105.

  9. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. Proc Int Conf Learn Represent. San Diego, CA. 2015.

  10. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Proc IEEE Conf Comput Vis Pattern Recognit. 2016;770–78.

  11. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C. MobileNetV2: inverted residuals and linear bottlenecks. Proc IEEE/CVF Conf Comput Vis Pattern Recognit. 2018;4510–20.

  12. Forcén JI, Pagola M, Barrenechea E, Bustince H. Co-occurrence of deep convolutional features for image search. Image Vis Comput. 2020;97(5):103909.

    Article  Google Scholar 

  13. Zhang B, Wang Q, Lu X, Wang F, Li P. Locality-constrained affine subspace coding for image classification and retrieval. Pattern Recognit. 2020;100:107167.

    Article  Google Scholar 

  14. Bai C, Li H, Zhang J, Huang L, Zhang L. Unsupervised adversarial instance-level image retrieval. IEEE Trans Multimedia. 2021;23:2199–207.

    Article  Google Scholar 

  15. Chen B, Wu Z, Davis L, Lim S. Efficient object embedding for spliced image retrieval. Proc IEEE/CVF Conf Comput Vis Pattern Recognit. 2021;14960–70.

  16. Zhang B-J, Liu G-H, Hu J-K. Filtering deep convolutional features for image retrieval. Int J Pattern Recognit Artif Intell. 2022;36(01):2252003.

    Article  Google Scholar 

  17. Li Y, Kong X, Zheng L, Tian Q. Exploiting hierarchical activations of neural network for image retrieval. Proc ACM Int Conf Multimedia. 2016;132–36.

  18. Liu G-H, Yang J-Y. Deep-seated features histogram: a novel image retrieval method. Pattern Recognit. 2021;116:107926.

    Article  Google Scholar 

  19. Liu G-H, Yang J-Y. Exploiting deep textures for image retrieval. Int J Mach Learn Cybern. 2023;14(2):483–94.

    Article  Google Scholar 

  20. Jia X, Siegle JH, Durand S, Heller G, Ramirez T, Koch C, et al. Multi-regional module-based signal transmission in mouse visual cortex. Neuron. 2022;110(9):1585–98.E9.

    Article  Google Scholar 

  21. UFLDL Tutorial. PCA Whitening. http://ufldl.stanford.edu/tutorial/unsupervised/PCAWhitening/. Accessed 25 Sep 2022.

  22. Philbin J, Chum O, Isard M, Sivic J, Zisserman A. Object retrieval with large vocabularies and fast spatial matching. Proc IEEE Conf Comput Vis Pattern Recognit. 2007;1–8.

  23. Philbin J, Chum O, Isard M, Sivic J, Zisserman A. Lost in quantization: improving particular object retrieval in large scale image databases. Proc IEEE Conf Comput Vis Pattern Recognit. 2008;1–8.

  24. Radenovic F, Iscen A, Tolias G, Avrithis Y, Chum O. Revisiting Oxford and Paris: large-scale image retrieval benchmarking. Proc IEEE/CVF Conf Comput Vis Pattern Recognit. 2018;5706–15.

  25. Jégou H, Douze M, Schmid C. Hamming embedding and weak geometric consistency for large scale image search. Proc Eur Conf Comput Vis. 2008;304–17.

Download references

Funding

This work was supported in part by the National Natural Science Foundation of China (grant no. 62266008), the Foundation of Guangxi Normal University (grant no. 2021JC007), and the Foundation of Development Research Centre of Guangxi in Humanities and Social Sciences (grant no. ZXZJ202201).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Guangxi Normal University, Guilin, 541004, China

    Fen Lu & Guang-Hai Liu

  2. School of Computer Engineering (Beihai Campus), Guilin University of Electronic Technology, Beihai, 536000, China

    Fen Lu

Authors
  1. Fen Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guang-Hai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Fen Lu: conceptualization; software; validation; writing, original draft; resources; and data curation. Guang-Hai Liu: methodology, review and editing, supervision, revision, funding acquisition, and formal analysis.

Corresponding author

Correspondence to Guang-Hai Liu.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, F., Liu, GH. Image Retrieval Using Object Semantic Aggregation Histogram. Cogn Comput 15, 1736–1747 (2023). https://doi.org/10.1007/s12559-023-10143-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-023-10143-6

Keywords

Search

Navigation