Skip to main content
SpringerLink
Log in

Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In recent years, the rapid growth of multimedia content makes content-based image retrieval (CBIR) a challenging research problem. The content-based attributes of the image are associated with the position of objects and regions within the image. The addition of image content-based attributes to image retrieval enhances its performance. In the last few years, the bag-of-visual-words (BoVW) based image representation model gained attention and significantly improved the efficiency and effectiveness of CBIR. In BoVW-based image representation model, an image is represented as an order-less histogram of visual words by ignoring the spatial attributes. In this paper, we present a novel image representation based on the weighted average of triangular histograms (WATH) of visual words. The proposed approach adds the image spatial contents to the inverted index of the BoVW model, reduces overfitting problem on larger sizes of the dictionary and semantic gap issues between high-level image semantic and low-level image features. The qualitative and quantitative analysis conducted on three image benchmarks demonstrates the effectiveness of the proposed approach based on WATH.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Alzu’bi A, Amira A, Ramzan N (2015) Semantic content-based image retrieval: A comprehensive study. J Vis Commun Image Represent 32:20–54

    Article  Google Scholar 

  2. Castellano G, Fanelli AM, Sforza G, Torsello AM (2016) Shape annotation for intelligent image retrieval. Appl Intell 44(1):179–195

    Article  Google Scholar 

  3. Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: Ideas, influences, and trends of the new age. ACM Comput Surv 40(2):5

    Article  Google Scholar 

  4. Chua T-S, Tang J, Hong R, Li H, Luo Z, Zheng Y (2009) Nus-wide: a real-world web image database from national university of singapore. In: Proceedings of the ACM international conference on image and video retrieval. ACM, p 48

  5. Yousaf RM, Rehman S, Dawood H, Ping G, Mehmood Z, Azam S, Khan AA (2017) Saliency based object detection and enhancements in static images. In: International Conference on Information Science and Applications. Springer, pp 114–123

  6. Sivic J, Zisserman A (2003) Video google: A text retrieval approach to object matching in videos. In: Proceedings of the 9th IEEE International Conference on Computer Vision, 2003. IEEE, pp. 1470–1477

  7. Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2007) Object retrieval with large vocabularies and fast spatial matching. In: IEEE conference on computer vision and pattern recognition, 2007. CVPR’07. IEEE, pp 1–8

  8. Ali N, Bajwa KB, Sablatnig R, Mehmood Zahid (2016) Image retrieval by addition of spatial information based on histograms of triangular regions. Computers & Electrical Engineering

  9. Li J, Wang JZ (2008) Real-time computerized annotation of pictures. IEEE Trans Pattern Anal Mach Intell 30(6):985–1002

    Article  MathSciNet  Google Scholar 

  10. Zhou W, Li H, Yijuan L, Tian Q (2013) Sift match verification by geometric coding for large-scale partial-duplicate web image search. ACM Trans Multimed Comput Commun Appl 9(1):4

    Article  Google Scholar 

  11. Khan R, Barat C, Muselet D, Ducottet C (2012) Spatial orientations of visual word pairs to improve bag-of-visual-words model. In: Proceedings of the British Machine Vision Conference. BMVA Press, pp 89–1

  12. Anwar H, Zambanini S, Kampel M, Vondrovec K (2015) Ancient coin classification using reverse motif recognition: Image-based classification of roman republican coins. IEEE Signal Process Mag 32(4):64–74

    Article  Google Scholar 

  13. Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE computer society conference on Computer vision and pattern recognition, vol 2. IEEE, pp 2169-2178

  14. Mehmood Z, Anwar SM, Altaf M, Ali N (2017) A novel image retrieval based on rectangular spatial histograms of visual words. Kuwait Journal of Science, Kuwait

    Google Scholar 

  15. Ashraf R, Bashir K, Mahmood T (2016) Content-based image retrieval by exploring bandletized regions through support vector machines. J Inf Sci Eng 32(2):245–269

    MathSciNet  Google Scholar 

  16. Zhang D, Md MI, Guojun L (2012) A review on automatic image annotation techniques. Pattern Recogn 45(1):346–362

    Article  Google Scholar 

  17. Liu Y, Zhang D, Guojun L, Ma W-Y (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282

    Article  MATH  Google Scholar 

  18. Das R, Thepade S, Ghosh S (2015) Multi technique amalgamation for enhanced information identification with content based image data. SpringerPlus 4(1):1–26

    Article  Google Scholar 

  19. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  20. Bay H, Tuytelaars T, Van Gool L (2006) Surf: Speeded up robust features. In: Computer visionECCV 2006. Springer, pp 404-417

  21. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR, vol 1. IEEE, pp 886–893

  22. Matas J, Chum O, Urban M, Pajdla T (2004) Robust wide-baseline stereo from maximally stable extremal regions. Image Vis Comput 22(10):761–767

    Article  Google Scholar 

  23. Leutenegger S, Chli M, Siegwart RY (2011) Brisk: Binary robust invariant scalable keypoints. In: 2011 IEEE international conference on computer vision (ICCV). IEEE, pp 2548–2555

  24. Mukherjee D, Wu QMJ, Wang G (2015) A comparative experimental study of image feature detectors and descriptors. Mach Vis Appl 26(4):443–466

    Article  Google Scholar 

  25. Krajnik T, Cristóforis P, Nitsche M, Kusumam K, Duckett T (2015) Image features and seasons revisited. In: 2015 European conference on mobile robots (ECMR). IEEE, pp 1–7

  26. Mahmood T, Nawaz T, Ashraf R, Shah M, Khan Z, Irtaza A, Mehmood Z A survey on block based copy move image forgery detection techniques. In: 2015 international conference on emerging technologies (ICET). IEEE, pp 1–6

  27. Wang C, Zhang B, Qin Z, Xiong J (2013) Spatial weighting for bag-of-features based image retrieval. In: Integrated uncertainty in knowledge modelling and decision making. Springer pp 91–100

  28. Tian X, Jiao L, Liu X, Zhang X (2014) Feature integration of eodh and color-sift: Application to image retrieval based on codebook. Signal Process Image Commun 29(4):530–545

    Article  Google Scholar 

  29. Jing Y, Qin Z, Wan T, Xi Z (2013) Feature integration analysis of bag-of-features model for image retrieval. Neurocomputing 120:355–364

    Article  Google Scholar 

  30. Zeng S, Huang R, Wang H, Kang Z (2016) Image retrieval using spatiograms of colors quantized by gaussian mixture models. Neurocomputing 171:673–684

    Article  Google Scholar 

  31. Walia E, Pal A (2014) Fusion framework for effective color image retrieval. J Vis Commun Image Represent 25(6):1335–1348

    Article  Google Scholar 

  32. Yuan X, Yu J, Qin Z, Wan T (2011) A sift-lbp image retrieval model based on bag of features IEEE International Conference on Image Processing

    Google Scholar 

  33. Dubey SR, Singh SK, Singh RK (2015) Rotation and scale invariant hybrid image descriptor and retrieval. Comput Electr Eng 46:288–302

    Article  Google Scholar 

  34. Wan J, Wang D, Hong Hoi SC, Wu P, Zhu J, Zhang Y, Li J (2014) Deep learning for content-based image retrieval: a comprehensive study. In: Proceedings of the ACM international conference on multimedia. ACM, 157–166

  35. Mehmood Z, Anwar SM, Ali N, Habib HA, Rashid M (2016) A novel image retrieval based on a combination of local and global histograms of visual words. Math Probl Eng 2016

  36. Gupta R, Patil H, Mittal A (2010) Robust order-based methods for feature description. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 334–341

  37. Liu L, Fieguth WP (2012) Texture classification from random features. IEEE Trans Pattern Anal Mach Intell 34(3):574– 586

    Article  Google Scholar 

  38. Mahmood T, Nawaz T, Mehmood Z, Khan Z, Shah M, Ashraf R (2016) Forensic analysis of copy-move forgery in digital images using the stationary wavelets. In: 2016 6th international conference on innovative computing technology (INTECH). IEEE, pp 578–583

  39. Csurka G, Dance C, Fan L, Willamowski J, Bray C (2004) Visual categorization with bags of keypoints. In: Workshop on statistical learning in computer vision, ECCV. vol 1. Prague, pp 1–2

  40. Arthur D, Vassilvitskii S (2007) k-means++: The advantages of careful seeding. In: Proceedings of the 18th annual ACM-SIAM symposium on discrete algorithms. Society for Industrial and Applied Mathematics, pp 1027–1035

  41. Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge university press, Cambridge

    Book  MATH  Google Scholar 

  42. Vedaldi A, Zisserman A Sparse kernel approximations for eff ient classification and detection. In: 2012 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2320–2327

  43. Nowak E, Jurie F, Triggs B (2006) Sampling strategies for bag-of-features image classification. In: Computer Vision–ECCV 2006. Springer, pp 490–503

  44. Wang JZ, Li J, Wiederhold G (2001) Simplicity: Semantics-sensitive integrated matching for picture libraries. IEEE Trans Pattern Anal Mach Intell 23(9):947–963

    Article  Google Scholar 

  45. Li R, Bhanu B, Krawiec K (2007) Hybrid coevolutionary algorithms vs. svm algorithms. In: Proceedings of the 9th annual conference on genetic and evolutionary computation. ACM, pp 456–463

  46. Xiao J, Hays J, Ehinger KA, Oliva A, Torralba A (2010) Sun database: Large-scale scene recognition from abbey to zoo. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 3485–3492

Download references

Author information

Authors and Affiliations

  1. Department of Software Engineering, University of Engineering and Technology, Taxila, 47050, Pakistan

    Zahid Mehmood

  2. Department of Computer Engineering, University of Engineering and Technology, Taxila, 47050, Pakistan

    Toqeer Mahmood

  3. Department of Basic Sciences, University of Engineering and Technology, Taxila, 47050, Pakistan

    Muhammad Arshad Javid

Authors
  1. Zahid Mehmood
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Toqeer Mahmood
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Arshad Javid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zahid Mehmood.

Ethics declarations

Competing Interest

All the authors declare no competing interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mehmood, Z., Mahmood, T. & Javid, M.A. Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine. Appl Intell 48, 166–181 (2018). https://doi.org/10.1007/s10489-017-0957-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-017-0957-5

Keywords

Search

Navigation