Skip to main content
SpringerLink
Log in

Scene analysis and search using local features and support vector machine for effective content-based image retrieval

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Despite broad investigation in content-based image retrieval (CBIR), issue to lessen the semantic gap between high-level semantics and local attributes of the image is still an important issue. The local attributes of an image such as shape, color, and texture are not sufficient for effective CBIR. Visual similarity is a principal step in CBIR and in the baseline approach. In this article, we introduce a novel approach, which relies on the fusion of visual words of scale-invariant feature transform (SIFT) and binary robust invariant scalable keypoints (BRISK) descriptors based on the visual-bag-of-words approach. The two local feature descriptors are chosen as their fusion adds complementary improvement to CBIR. The SIFT descriptor is capable of detecting objects robustly under cluttering due to its invariance to scale, rotation, noise, and illumination variance. However, SIFT descriptor does not perform well at low illumination or poorly localized keypoints within an image. Due to this reason, the discriminative power of the SIFT descriptor is lost during the quantization process, which also reduces the performance of CBIR. However, the BRISK descriptor provides scale and rotation-invariant scale-space, high quality and adaptive performance in classification based applications. It also performs better at poorly localized keypoints along the edges of an object within an image as compared to the SIFT descriptor. The suggested approach based on the fusion of visual words achieves effective results on the Corel-1K, Corel-1.5K, Corel-5K, and Caltech-256 image repositories as equated to the feature fusion of both descriptors and latest CBIR approaches with the surplus assistances of scalability and fast indexing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

(Reproduced with permission from Li and Wang 2003)

Fig. 2

(Reproduced with permission from Li and Wang 2003)

Fig. 3
Fig. 4

(Reproduced with permission from Li and Wang 2003)

Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

(Reproduced with permission from Li and Wang 2003)

Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  • Ali N, Bajwa KB, Sablatnig R, Mehmood Z (2016) Image retrieval by addition of spatial information based on histograms of triangular regions. Comput Electr Eng 54:539–550

    Article  Google Scholar 

  • Arampatzis A, Zagoris K, Chatzichristofis SA (2013) Dynamic two-stage image retrieval from large multimedia databases. Inf Process Manag 49(1):274–285

    Article  Google Scholar 

  • Arthur D, Vassilvitskii S (2007) k-means++: the advantages of careful seeding. In: Proceedings of the eighteenth annual ACM-SIAM symposium on discrete algorithms. Society for Industrial and Applied Mathematics

  • Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110(3):346–359

    Article  Google Scholar 

  • Chen Y, Wang JZ, Krovetz R (2005) CLUE: cluster-based retrieval of images by unsupervised learning. IEEE Trans Image Process 14(8):1187–1201

    Article  Google Scholar 

  • Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40(2):5

    Article  Google Scholar 

  • Deb K, Padhye N (2014) Enhancing performance of particle swarm optimization through an algorithmic link with genetic algorithms. Comput Optim Appl 57(3):761–794

    Article  MathSciNet  MATH  Google Scholar 

  • Dinesh KR, Hariharan S (2014) Approaches and trends in content based image retrieval. In: Proceedings of 24th international conference in communication, network and computing, CNC. Elsevier, pp 473–477

  • Dubey SR, Singh SK, Singh RK (2015) Rotation and scale invariant hybrid image descriptor and retrieval. Comput Electric Eng 46:288–302

    Article  Google Scholar 

  • ElAlami ME (2014) A new matching strategy for content based image retrieval system. Appl Soft Comput 14:407–418

    Article  Google Scholar 

  • Iakovidou C, Anagnostopoulos N, Kapoutsis AC, Boutalis Y, Chatzichristofis SA (2014) Searching images with MPEG-7 (& mpeg-7-like) powered localized descriptors: the SIMPLE answer to effective content based image retrieval. In: 12th International workshop on content-based multimedia indexing (CBMI). IEEE

  • Krajník T, Cristóforis P, Nitsche M, Kusumam K, Duckett T (2015) Image features and seasons revisited. In: European conference on mobile robots (ECMR). IEEE

  • Leutenegger S, Chli M, Siegwart RY (2011) BRISK: binary robust invariant scalable keypoints. In: IEEE international conference on computer vision (ICCV). IEEE

  • Li J, Wang JZ (2003) Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans Pattern Anal Mach Intell 25(9):1075–1088

    Article  Google Scholar 

  • Liu G-H, Yang J-Y (2013) Content-based image retrieval using color difference histogram. Pattern Recogn 46(1):188–198

    Article  Google Scholar 

  • Liu Y, Zhang D, Lu G, Ma W-Y (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282

    Article  MATH  Google Scholar 

  • Liu G-H, Zhang L, Hou Y-K, Li Z-Y, Yang J-Y (2010) Image retrieval based on multi-texton histogram. Pattern Recogn 43(7):2380–2389

    Article  MATH  Google Scholar 

  • Lowe DG (1999) Object recognition from local scale-invariant features. In: Computer vision, 1999. The proceedings of the seventh IEEE international conference. IEEE

  • Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Google Scholar 

  • Mehmood Z, Anwar SM, Ali N, Habib HA, Rashid M (2016) A novel image retrieval based on a combination of local and global histograms of visual words. Math Probl Eng 2016(2016):1–12

    Article  Google Scholar 

  • Mehmood Z, Anwar SM, Altaf M (2018) A novel image retrieval based on rectangular spatial histograms of visual words. Kuwait J Sci 45(1):54–69

    MathSciNet  Google Scholar 

  • Mehmood Z, Mahmood T, Javid MA (2018) Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine. Appl Intell 48(1):166–181

    Article  Google Scholar 

  • Mukherjee D, Wu QJ, Wang G (2015) A comparative experimental study of image feature detectors and descriptors. Mach Vis Appl 26(4):443–466

    Article  Google Scholar 

  • Padhye N, Bhardawaj P, Deb K (2013) Improving differential evolution through a unified approach. J Glob Optim 55(4):771–799

    Article  MathSciNet  MATH  Google Scholar 

  • Peng H, Long F, Ding C (2005) Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27(8):1226–1238

    Article  Google Scholar 

  • Raja NMK, Bhanu KS (2013) Content bases image search and retrieval using indexing by k means clustering technique. Int J Adv Res Comput Commun Eng 2(5):2181–2189

    Google Scholar 

  • Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: an efficient alternative to SIFT or SURF. In: IEEE international conference on computer vision (ICCV). IEEE

  • Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  • Tian X, Jiao L, Liu X, Zhang X (2014) Feature integration of EODH and Color-SIFT: application to image retrieval based on codebook. Sig Process Image Commun 29(4):530–545

    Article  Google Scholar 

  • Tousch A-M, Herbin S, Audibert J-Y (2012) Semantic hierarchies for image annotation: a survey. Pattern Recogn 45(1):333–345

    Article  Google Scholar 

  • Tzelepi M, Tefas A (2018) Deep convolutional learning for content based image retrieval. Neurocomputing 275:2467–2478

    Article  Google Scholar 

  • Uricchio T, Ballan L, Seidenari L, Del Bimbo A (2017) Automatic image annotation via label transfer in the semantic space. Pattern Recogn 71:144–157

    Article  Google Scholar 

  • Yang X, Cai L (2013) Adaptive region matching for region-based image retrieval by constructing region importance index. IET Comput Vis 8(2):141–151

    Article  Google Scholar 

  • Yu J, Qin Z, Wan T, Zhang X (2013) Feature integration analysis of bag-of-features model for image retrieval. Neurocomputing 120:355–364

    Article  Google Scholar 

  • Yuan X, Yu J, Qin Z, Wan T (2011) A SIFT-LBP image retrieval model based on bag of features. In: IEEE international conference on image processing

  • Zeng S, Huang R, Wang H, Kang Z (2016) Image retrieval using spatiograms of colors quantized by Gaussian mixture models. Neurocomputing 171:673–684

    Article  Google Scholar 

  • Zhang D, Islam MM, Lu G (2012) A review on automatic image annotation techniques. Pattern Recogn 45(1):346–362

    Article  Google Scholar 

  • Zhong D, Defée I (2005) DCT histogram optimization for image database retrieval. Pattern Recogn Lett 26(14):2272–2281

    Article  Google Scholar 

Download references

Acknowledgements

All the sources are acknowledged properly, where applicable.

Funding

This work was partially supported by the Machine Learning Research Group; Prince Sultan University Riyadh; Saudi Arabia [RG-CCIS-2017-06-02]; The authors are grateful for this financial support.

Author information

Authors and Affiliations

  1. Department of Software Engineering, University of Engineering and Technology, Taxila, 47050, Pakistan

    Uzma Sharif & Zahid Mehmood

  2. Department of Computer Science, University of Engineering and Technology, Taxila, 47050, Pakistan

    Toqeer Mahmood

  3. Department of Basic Sciences, University of Engineering and Technology, Taxila, 47050, Pakistan

    Muhammad Arshad Javid

  4. College of Computer and Information Systems, Al-Yamamah University, Riyadh, 11512, Saudi Arabia

    Amjad Rehman

  5. College of Computer and Information Sciences, Prince Sultan University, Riyadh, 11586, Saudi Arabia

    Tanzila Saba

Authors
  1. Uzma Sharif
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zahid Mehmood
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Toqeer Mahmood
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Muhammad Arshad Javid
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Amjad Rehman
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tanzila Saba
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All the authors contributed equally.

Corresponding author

Correspondence to Zahid Mehmood.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sharif, U., Mehmood, Z., Mahmood, T. et al. Scene analysis and search using local features and support vector machine for effective content-based image retrieval. Artif Intell Rev 52, 901–925 (2019). https://doi.org/10.1007/s10462-018-9636-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-018-9636-0

Keywords

Search

Navigation