Skip to main content
SpringerLink
Log in

Dynamic Mode Decomposition based salient edge/region features for content based image retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Considering the gap between low-level image features and high-level retrieval concept, this paper investigates the effect of incorporating visual saliency based features for content-based image retrieval(CBIR).Visual saliency plays an important role in human perception due to its capability to focus the attention on the point of interest, i.e. an intended target. This selection based processing can be well explored in localized CBIR systems, since in context of CBIR the users will be interested only in certain parts of the image. The proposed methodology uses Dynamic Mode Decomposition framework to extract the saliency map which highlights the part of the image that grabs human attention. Then, based on the saliency map, an efficient salient edge detection model is introduced. Visual saliency based features (salient region, edge) are then combined with texture and color features to formulate the high dimensional feature vector for image retrieval. State-of-the-art learning based CBIR models demands for user feed back to model the retrieval concept. In contrast with these models, proposed CBIR system does not require any user interaction, since it uses perceptual level features for the retrieval task. Performance of the proposed CBIR system is evaluated and confirmed on images from Wang’s dataset using benchmark evaluation metrics like precision and recall. Experimental results reveals that incorporation of saliency features can represent human perception well and produces good retrieval performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp 1597–1604

  2. Bi C, Yuan Y, Zhang R, Xiang Y, Wang Y, Zhang J (2017) A dynamic mode decomposition based edge detection method for art images. IEEE Photonics Journal 9(6):1–13

    Article  Google Scholar 

  3. Borji A, Sihite DN, Itti L (2012) Probabilistic learning of task-specific visual attention. In: 2012 IEEE Conference on computer vision and pattern recognition, vol 470–477. IEEE

  4. Brunton BW, Johnson LA, Ojemann JG, Kutz JN (2016) Extracting spatial–temporal coherent patterns in large-scale neural recordings using dynamic mode decomposition. J Neurosci Methods 258:1–15

    Article  Google Scholar 

  5. Cheng M-M, Mitra NJ, Huang X, Torr PH, Hu S-M (2014) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37 (3):569–582

    Article  Google Scholar 

  6. Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys (Csur) 40(2):5

    Article  Google Scholar 

  7. de Carvalho Soares R, da Silva IR, Guliato D (2012) Spatial locality weighting of features using saliency map with a bag-of-visual-words approach. In: 2012 IEEE 24th International conference on tools with artificial intelligence, vol 1. IEEE, pp 1070–1075

  8. Duygulu P, Barnard K, de Freitas JF, Forsyth DA (2002) Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In: European conference on computer vision. Springer, pp 97–112

  9. Feng S, Xu D, Yang X (2010) Attention-driven salient edge (s) and region (s) extraction with application to cbir. Signal Process 90(1):1–15

    Article  Google Scholar 

  10. Fu H, Chi Z, Feng D (2006) Attention-driven image interpretation with application to image retrieval. Pattern Recogn 39(9):1604–1621

    Article  Google Scholar 

  11. Giouvanakis E, Kotropoulos C (2014) Saliency map driven image retrieval combining the bag-of-words model and plsa. In: 2014 19th International conference on digital signal processing. IEEE, pp 280–285

  12. Grosek J, Kutz JN Dynamic mode decomposition for real-time background/foreground separation in video, arXiv:1404.7592

  13. Guangnan H, Yubin Y, Jiabin R, Jinjie L (2011) Image retrieval based on visual consistency. Journal of Image and Graphics 16(4):503–509

    Google Scholar 

  14. Huang X, Sun L, Guo H, Liu S (2016) Discriminative extreme learning machine to content-based image retrieval with relevance feedback. In: 2016 12th World Congress on Intelligent Control and Automation WCICA. IEEE, pp 3056–3060

  15. Irtaza A, Jaffar MA, Aleisa E, Choi T-S (2014) Embedding neural networks for semantic association in content based image retrieval. Multimedia Tools Appl 72(2):1911–1931

    Article  Google Scholar 

  16. Jing F, Li M, Zhang H. -J., Zhang B (2004) An efficient and effective region-based image retrieval framework. IEEE Trans Image Process 13 (5):699–709

    Article  Google Scholar 

  17. Kanan C, Cottrell G (2010) Robust classification of objects, faces, and flowers using natural image statistics. In: 2010 IEEE Computer society conference on computer vision and pattern recognition. IEEE, pp 2472–2479

  18. Li B, Xiong W, Wu O, Hu W, Maybank S, Yan S (2015) Horror image recognition based on context-aware multi-instance learning. IEEE Trans Image Process 24(12):5193–5205

    Article  MathSciNet  Google Scholar 

  19. Lin C-H, Chen R-T, Chan Y-K (2009) A smart content-based image retrieval system based on color and texture feature. Image Vis Comput 27 (6):658–665

    Article  Google Scholar 

  20. Liu J, Meng F, Mu F, Zhang Y (2014) An improved image retrieval method based on sift algorithm and saliency map, IEEE, FSKD

  21. Liu Z, Shi R, Shen L, Xue Y, Ngan KN, Zhang Z (2012) Unsupervised salient object segmentation based on kernel density estimation and two-phase graph cut. IEEE Transactions on Multimedia 14(4):1275–1289

    Article  Google Scholar 

  22. Liu Y, Zhang D, Lu G, Ma W. -Y. (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282

    Article  Google Scholar 

  23. Mitro J Content-based image retrieval tutorial, arXiv:1608.03811

  24. Müller H, Michoux N, Bandon D, Geissbuhler A (2004) A review of content-based image retrieval systems in medical applications—clinical benefits and future directions. Int J Med Inform 73(1):1–23

    Article  Google Scholar 

  25. Navalpakkam V, Itti L (2006) An integrated model of top-down and bottom-up attention for optimizing detection speed. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol 2. IEEE, pp 2049–2056

  26. Ninassi A, Le Meur O, Le Callet P, Barba D (2007) Does where you gaze on an image affect your perception of quality? applying visual attention to image quality metric. In: 2007 IEEE International Conference on Image Processing, vol 2. IEEE, pp II–169

  27. Ojala T, Pietikäinen M, Mäenpää T (2000) Gray scale and rotation invariant texture classification with local binary patterns. In: European Conference on Computer Vision. Springer, pp 404–420

  28. Papushoy A, Bors AG (2015) Image retrieval based on query by saliency content. Digital Signal Processing 36:156–173

    Article  MathSciNet  Google Scholar 

  29. Pardede J, Sitohang B, Akbar S, Khodra ML (2018) Svm relevance feedback in hsv quantization for cbir. JCP 13(12):1366–1384

    Article  Google Scholar 

  30. Park J, Lee J-Y, Tai Y-W, Kweon IS (2012) Modeling photo composition and its application to photo re-arrangement. In: 2012 19th IEEE International conference on image processing. IEEE, pp 2741–2744

  31. Persoon E, Fu K-S (1977) Shape discrimination using fourier descriptors. IEEE Trans Sys Man Cybern 7(3):170–179

    Article  MathSciNet  Google Scholar 

  32. Rui Y, Huang TS, Ortega M, Mehrotra S (1998) Relevance feedback: A power tool for interactive content-based image retrieval. IEEE Trans Circuits Sys Vid Technol 8(5):644–655

    Article  Google Scholar 

  33. Schmid PJ (2010) Dynamic mode decomposition of numerical and experimental data. J Fluid Mech 656:5–28

    Article  MathSciNet  Google Scholar 

  34. Sikha O, Kumar SS, Soman K (2018) Salient region detection and object segmentation in color images using dynamic mode decomposition. J Comput Sci 25:351–366

    Article  Google Scholar 

  35. Singh A, Singh KK (2017) Satellite image classification using genetic algorithm trained radial basis function neural network, application to the detection of flooded areas. J Vis Commun Image Represent 42:173–182

    Article  Google Scholar 

  36. Walia E, Pal A (2014) Fusion framework for effective color image retrieval. J Vis Commun Image Represent 25(6):1335–1348

    Article  Google Scholar 

  37. Walia E, Vesal S, Pal A (2014) An effective and fast hybrid framework for color image retrieval. Sensing and Imaging 15(1):93

    Article  Google Scholar 

  38. Wan S, Jin P, Yue L (2009) An approach for image retrieval based on visual saliency. In: 2009 International conference on image analysis and signal processing. IEEE, pp 172–175

  39. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Proce 13(4):600–612

    Article  Google Scholar 

  40. Wang JZ, Li J, Wiederhold G (2001) Simplicity: Semantics-sensitive integrated matching for picture libraries. IEEE Trans Pattern Anal Mach Intell (9):947–963

  41. Wang X-Y, Yu Y-J, Yang HY (2011) An effective image retrieval scheme using color, texture and shape features. Computer Standards & Interfaces 33(1):59–68

    Article  Google Scholar 

  42. Wu Y, Liu H, Yuan J, Zhang Q (2018) Is visual saliency useful for content-based image retrieval? Multimedia Tools and Applications 77 (11):13983–14006

    Article  Google Scholar 

  43. Yang J, Yang M-H (2016) Top-down visual saliency via joint crf and dictionary learning. IEEE Tran Pattern Anal Mach Intell 39(3):576–588

    Article  Google Scholar 

  44. Zhang J, Shen L. -s. (2008) A survey of image retrieval based on visual perception. Acta Electronica Sinica 36(3):494

    MathSciNet  Google Scholar 

  45. Zhang H-J, Su Z, Zhu X (2006) Relevance maximizing, iteration minimizing, relevance-feedback, content-based image retrieval (cbir)., US Patent 7,113,944

  46. Zheng L, Wang S, Liu Z, Tian Q (2015) Fast image retrieval: Query pruning and early termination. IEEE Transactions on Multimedia 17(5):648–659

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Computational Engineering & Networking (CEN), Department of Computer Science and Engineering, Amrita School of Engineering, Coimbatore, Amrita Vishwa Vidyapeetham, India

    Sikha O. K. & Soman K. P.

Authors
  1. Sikha O. K.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Soman K. P.
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sikha O. K..

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

K., S.O., P., S.K. Dynamic Mode Decomposition based salient edge/region features for content based image retrieval. Multimed Tools Appl 80, 15937–15958 (2021). https://doi.org/10.1007/s11042-020-10315-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10315-8

Keywords

Search

Navigation