Skip to main content
SpringerLink
Log in

Multi-scale local structure patterns histogram for describing visual contents in social image retrieval systems

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Content based image retrieval systems rely heavily on the set of features extracted from images. Effective image representation emerges as a crucial step in such systems. A key challenge in visual content representation is to reduce the so called ‘semantic gap’. It is the inability of existing methods to describe contents in a human-oriented way. Content representation methods inspired by the human vision system have shown promising results in image retrieval. Considerable work has been carried out during the past two decades for developing methods to extract descriptors inspired by the human vision system and attempt to retrieve visual contents efficiently according to the user needs, thereby reducing the semantic gap. Despite the extensive research being conducted in this area, limitations in current image retrieval systems still exist. This paper presents a descriptor for personalized social image collections which utilizes the local structure patterns in salient edge maps of images at multiple scales. The human visual system at the basic level is sensitive to edges, corners, intersections, and other such intensity variations in images generating local structure patterns. Analyzing these patterns at multiple scales allow the most salient fine-grained and coarse-grained features to be captured. The features are accumulated in a local structure patterns histogram to index images allowing flexible querying of visual contents. The retrieval results show that the proposed descriptor ranks well among similar state-of-the-art methods for large social image collections.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Ahmad J, Jan Z, Khan SM (2014) A Fusion of Labeled-Grid Shape Descriptors with Weighted Ranking Algorithm for Shapes Recognition. World Applied Sciences Journal 31(6):1207–1213

    Google Scholar 

  2. Ahmad J, Sajjad M, Mehmood I, Baik SW (2015) SSH : Salient structures histogram for content based image retrieval. In: 18th IEEE International Conference on Network-Based Information Systems (NBiS), Taipei, Taiwan, 2015

  3. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(12):2037–2041

    Article  MATH  Google Scholar 

  4. Canny J (1986) A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 6:679–698

    Article  Google Scholar 

  5. Corel Dataset. http://wang.ist.psu.edu/docs/related.shtml. Accessed 10 July 2015

  6. Emerson CW, Siu-Ngan Lam N, Ouattrochi D (1999) Multi-scale fractal analysis of image texture and patterns. Photogrammetric Engineering and Remote Sensing 65:51–62

    Google Scholar 

  7. Facebook Has a Quarter of a Trillion User Photos (2015) http://mashable.com/2013/09/16/facebook-photo-uploads/

  8. Facebook Photo Facts (2015) http://www.adweek.com/socialtimes/wp-content/uploads/sites/2/2013/11/Superfish-VENN-infographcis_final.jpg

  9. Felzenszwalb P, Oberlin JG (2014) Multiscale fields of patterns. In: Advances in neural information processing systems, pp 82–90

  10. Han J, Ma K-K (2002) Fuzzy color histogram and its use in color image retrieval. IEEE Transactions on Image Processing 11(8):944–952

    Article  Google Scholar 

  11. Hansen T (2003) A neural model of early vision: contrast, contours, corners and surfaces. University of Ulm Faculty of Computer Science: 35–42

  12. Hansen T, Neumann H (2004) Neural mechanisms for the robust representation of junctions. Neural Computation 16(5):1013–1037

    Article  MATH  Google Scholar 

  13. Heikkilä M, Pietikäinen M, Schmid C (2009) Description of interest regions with local binary patterns. Pattern recognition 42(3):425–436

    Article  MATH  Google Scholar 

  14. Huang J, Kumar SR, Mitra M, Zhu W-J, Zabih R (1997) Image indexing using color correlograms. In: Proceedings of the IEEE Computer Society Conference on Computer vision and pattern recognition, 1997, pp 762–768

  15. Huang S, Wang W, Zhang H (2014) Retrieving images using saliency detection and graph matching. In: IEEE International Conference on Image Processing (ICIP) 2014, pp 3087–3091

  16. INRIA Holidays dataset (2015) https://lear.inrialpes.fr/~jegou/data.php. Accessed 10 July 2015

  17. Jégou H, Douze M, Schmid C, Pérez P (2010) Aggregating local descriptors into a compact image representation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp 3304–3311

  18. Jégou H, Douze M, Schmid C (2008) Hamming embedding and weak geometry consistency for large scale image search-extended version

  19. Kasutani E, Yamada A (2001) The MPEG-7 color layout descriptor: a compact image feature description for high-speed image/video segment retrieval. In: Proceedings of IEEE International Conference on Image Processing, 2001. pp 674–677

  20. Li J, Wang JZ (2008) Real-time computerized annotation of pictures. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(6):985–1002

    Article  Google Scholar 

  21. Lin R-J, Lin W-S (2014) A computational visual saliency model based on statistics and machine learning. Journal of vision 14(9):1

    Article  Google Scholar 

  22. Liu G-H, Li Z-Y, Zhang L, Xu Y (2011) Image retrieval based on micro-structure descriptor. Pattern Recognition 44(9):2123–2133

    Article  Google Scholar 

  23. Liu G-H, Yang J-Y (2013) Content-based image retrieval using color difference histogram. Pattern Recognition 46(1):188–198

    Article  Google Scholar 

  24. Liu G-H, Zhang L, Hou Y-K, Li Z-Y, Yang J-Y (2010) Image retrieval based on multi-texton histogram. Pattern Recognition 43(7):2380–2389

    Article  MATH  Google Scholar 

  25. Lu G, Teng S (1999) A novel image retrieval technique based on vector quantization. In: Proceedings of International Conference on Computational Intelligence for Modeling, Control and Automation, 1999. pp 36–41

  26. Manjunath BS, Ma W-Y (1996) Texture features for browsing and retrieval of image data. IEEE Transactions on Pattern Analysis and Machine Intelligence 18(8):837–842

    Article  Google Scholar 

  27. Manjunath BS, Ohm J-R, Vasudevan VV, Yamada A (2001) Color and texture descriptors. IEEE Transactions on Circuits and Systems for Video Technology, 11(6):703–715

    Article  Google Scholar 

  28. Messing DS, Van Beek P, Errico JH (2001) The mpeg-7 colour structure descriptor: image description using colour and local spatial information. In: Proceedings of the International Conference on Image Processing, 2001, pp 670–673

  29. Mu Y, Yan S, Liu Y, Huang T, Zhou B (2008) Discriminative local binary patterns for human detection in personal album. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, 2008, pp 1–8

  30. Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7):971–987

    Article  MATH  Google Scholar 

  31. Olshausen BA, Field DJ (2004) Sparse coding of sensory inputs. Current opinion in neurobiology 14(4):481–487

    Article  Google Scholar 

  32. Ortega M, Rui Y, Chakrabarti K, Porkaew K, Mehrotra S, Huang TS (1998) Supporting ranked boolean similarity queries in MARS. IEEE Transactions on Knowledge and Data Engineering 10(6):905–925

    Article  Google Scholar 

  33. Pass G, Zabih R, Miller J (1997) Comparing images using color coherence vectors. In: Proceedings of the fourth ACM international conference on Multimedia, 1997. ACM, pp 65–73

  34. Portilla J, Simoncelli EP (2000) A parametric texture model based on joint statistics of complex wavelet coefficients. International Journal of Computer Vision 40(1):49–70

    Article  MATH  Google Scholar 

  35. Rahimi M, Moghaddam ME (2013) A content-based image retrieval system based on Color Ton Distribution descriptors. SIViP 1–14. doi:10.1007/s11760-013-0506-6

  36. Smeulders AW, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12):1349–1380

    Article  Google Scholar 

  37. Subrahmanyam M, Maheshwari R, Balasubramanian R (2012) Expert system design using wavelet and color vocabulary trees for image retrieval. Expert Systems with Applications 39(5):5104–5114

    Article  Google Scholar 

  38. Subrahmanyam M, Wu QJ, Maheshwari R, Balasubramanian R (2013) Modified color motif co-occurrence matrix for image indexing and retrieval. Computers & Electrical Engineering 39(3):762–774

    Article  Google Scholar 

  39. Takala V, Ahonen T, Pietikäinen M (2005) Block-based methods for image retrieval using local binary patterns. In: Image analysis. Springer, pp 882–891

  40. Terzić K, Rodrigues JM, du Buf JH (2015) BIMP: A real-time biological model of multi-scale keypoint detection in V1. Neurocomputing 150:227–237

    Article  Google Scholar 

  41. van Ginneken B, ter Haar Romeny BM (2003) Multi-scale texture classification from generalized locally orderless images. Pattern Recognition 36(4):899–911

    Article  Google Scholar 

  42. Vinje WE, Gallant JL (2000) Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287(5456):1273–1276

    Article  Google Scholar 

  43. Vipparthi SK, Murala S, Nagar SK (2015) Dual directional multi-motif XOR patterns: A new feature descriptor for image indexing and retrieval. Optik-International Journal for Light and Electron Optics 126(15):1467–1473

    Article  Google Scholar 

  44. Vipparthi SK, Nagar S (2014) Expert image retrieval system using directional local motif XoR patterns. Expert Systems with Applications 41(17):8016–8026

    Article  Google Scholar 

  45. Wang X, Wang Z (2013) A novel method for image retrieval based on structure elements’ descriptor. Journal of Visual Communication and Image Representation 24(1):63–74

    Article  Google Scholar 

  46. Yang M, Kpalma K, Ronsin J (2008) A survey of shape feature extraction techniques. Pattern recognition:43–90

  47. Yao C-H, Chen S-Y (2003) Retrieval of translated, rotated and scaled color textures. Pattern Recognition 36(4):913–929

    Article  MathSciNet  Google Scholar 

  48. Zhang D, Lu G (2004) Review of shape representation and description techniques. Pattern recognition 37(1):1–19

    Article  Google Scholar 

  49. Zhang J, Tan T (2002) Brief review of invariant texture analysis methods. Pattern recognition 35(3):735–747

    Article  MATH  Google Scholar 

Download references

Acknowledgments

This research is supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2013R1A1A2012904).

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Sejong University, Seoul, Republic of Korea

    Jamil Ahmad & Sung Wook Baik

  2. Department of Computer Science, Islamia College, Peshawar, Pakistan

    Muhammad Sajjad

  3. Department of Multimedia, Sungkyul University, Anyang, Republic of Korea

    Seungmin Rho

Authors
  1. Jamil Ahmad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammad Sajjad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seungmin Rho
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sung Wook Baik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sung Wook Baik.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmad, J., Sajjad, M., Rho, S. et al. Multi-scale local structure patterns histogram for describing visual contents in social image retrieval systems. Multimed Tools Appl 75, 12669–12692 (2016). https://doi.org/10.1007/s11042-016-3436-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3436-9

Keywords

Search

Navigation