Skip to main content
SpringerLink
Log in

Attention-guided salient object detection using autoencoder regularization

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

A saliency detection task simulates the attention mechanism of the human visual system, which focuses on what draws the most attention in a picture, and performs accurate localization and pixel-level segmentation of the object. Existing detection methods based on neural networks usually perform calculation of the object position information and edge information separately in each layer, resulting in calculation redundancy and insufficient utilization of information. To address this issue, this paper proposes an attention-guided detection network using an autoencoder (AGA-Net). First, using a proposed attention-guided multi-scale (AM) module, results from deep layers can be used to highlight features of the foreground and suppress features of the background and extract different scale features that are more relevant to the detection task. Second, a bi-refinement (BR) module composed of two sub-networks is proposed. One sub-network extracts information of the foreground to find redundant areas in the prediction results, and the other uses background information to supplement missing boundary information. Finally, the new model uses a variational autoencoder (VAE) branch to realize the image restoration task. It shares the encoder module with the object detection task and helps it escape from a local minimum in the converging process. Extensive experiments on six benchmark datasets were conducted and the proposed method is compared with 19 state-of-the-art methods, demonstrating that the new method has the best results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

The datasets and materials used or analysed during the current study are available from the corresponding author on reasonable request.

Code Availability

The code used or analysed during the current study is available from the corresponding author on reasonable request.

References

  1. Borji A, Cheng M-M, Hou Q, Jiang H, Li J (2019) Salient object detection: a survey. Comput Vis Media 5(2):117–150

    Article  Google Scholar 

  2. Wang L, Lu H, Ruan X, Yang M-H (2015) Deep networks for saliency detection via local estimation and global search. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3183–3192

  3. Liu Y, Zhang X-Y, Bian J-W, Zhang L, Cheng M-M (2021) Samnet: stereoscopically attentive multi-scale network for lightweight salient object detection. IEEE Trans Image Process 30:3804–3814

    Article  Google Scholar 

  4. Ji Y, Zhang H, Zhang Z, Liu M (2021) Cnn-based encoder-decoder networks for salient object detection: a comprehensive review and recent advances. Inf Sci 546:835–857

    Article  MathSciNet  Google Scholar 

  5. Qin X, Zhang Z, Huang C, Gao C, Dehghan M, Jagersand M (2019) Basnet: boundary-aware salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7479–7489

  6. Pang Y, Zhao X, Zhang L, Lu H (2020) Multi-scale interactive network for salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9413–9422

  7. Wang S, Liu X, Zhao J, Liu Y, Liu S, Liu Y, Zhao J (2021) Computer auxiliary diagnosis technique of detecting cholangiocarcinoma based on medical imaging: a review. Comput Methods Programs Biomed 208:106265

    Article  Google Scholar 

  8. Shi L, Wang Z, Pan B, Shi Z (2020) An end-to-end network for remote sensing imagery semantic segmentation via joint pixel-and representation-level domain adaptation. IEEE Geosci Remote Sens Lett 18(11):1896–1900

    Article  Google Scholar 

  9. Dogan Y, Keles HY (2020) Semi-supervised image attribute editing using generative adversarial networks. Neurocomputing 401:338–352

    Article  Google Scholar 

  10. Chen S, Tan X, Wang B, Lu H, Hu X, Fu Y (2020) Reverse attention-based residual network for salient object detection. IEEE Trans Image Process 29:3763–3776

    Article  MATH  Google Scholar 

  11. Fan D-P, Zhai Y, Borji A, Yang J, Shao L (2020) Bbs-net: Rgb-d salient object detection with a bifurcated backbone strategy network. In: Proceedings of the european conference on computer vision (ECCV), pp 275–292

  12. Qin X, Zhang Z, Huang C, Dehghan M, Zaiane OR, Jagersand M (2020) U2-net: going deeper with nested u-structure for salient object detection. Pattern Recogn 106:107404

    Article  Google Scholar 

  13. Myronenko A (2019) 3d Mri brain tumor segmentation using autoencoder regularization. Brainlesion: glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries:311–320

  14. He S, Lau RW, Liu W, Huang Z, Yang Q (2015) Supercnn: a superpixelwise convolutional neural network for salient object detection. Int J Comput Vis 115(3):330–344

    Article  MathSciNet  Google Scholar 

  15. Hu T, Yang M, Yang W, Li A (2019) An end-to-end differential network learning method for semantic segmentation. Int J Mach Learn Cybern 10(7):1909–1924

    Article  Google Scholar 

  16. Zhang J, Zhang T, Dai Y, Harandi M, Hartley R (2018) Deep unsupervised saliency detection: a multiple noisy labeling perspective. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9029–9038

  17. Hou Q, Cheng M-M, Hu X, Borji A, Tu Z, Torr PH (2017) Deeply supervised salient object detection with short connections. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3203–3212

  18. Liu N, Han J, Yang M-H (2018) Picanet: learning pixel-wise contextual attention for saliency detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3089–3098

  19. Liu J-J, Hou Q, Cheng M-M, Feng J, Jiang J (2019) A simple pooling-based design for real-time salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3917–3926

  20. Gehring J, Auli M, Grangier D, Yarats D, Dauphin YN (2017) Convolutional sequence to sequence learning. In: Proceedings of the 34th international conference on machine learning, pp 1243–1252

  21. Zhang X, Wang T, Qi J, Lu H, Wang G (2018) Progressive attention guided recurrent network for salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 714–722

  22. Zhang L, Dai J, Lu H, He Y, Wang G (2018) A bi-directional message passing model for salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1741–1750

  23. Sun F, Li W, Guan Y (2019) Self-attention recurrent network for saliency detection. Multimed Tools Appl 78(21):30793–30807

    Article  Google Scholar 

  24. Zhang X, Wang T, Qi J, Lu H, Wang G (2018) Progressive attention guided recurrent network for salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 714–722

  25. Wang H, Dai L, Cai Y, Sun X, Chen L (2018) Salient object detection based on multi-scale contrast. Neural Netw 101:47–56

    Article  Google Scholar 

  26. Wang H, Fan R, Cai P, Liu M (2021) Sne-roadseg+: rethinking depth-normal translation and deep supervision for freespace detection. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems, pp 1140–1145

  27. Su J, Li J, Zhang Y, Xia C, Tian Y (2019) Selectivity or invariance: Boundary-aware salient object detection. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3799–3808

  28. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 248–255

  29. Zhang C, Li G, Du S (2019) Multi-scale dense networks for hyperspectral remote sensing image classification. IEEE Trans Geosci Remote Sens 57(11):9201–9222

    Article  Google Scholar 

  30. Gao S, Cheng M-M, Zhao K, Zhang X-Y, Yang M-H, Torr PH (2021) Res2net: a new multi-scale backbone architecture. IEEE Trans Pattern Anal Mach Intell 43(2):652–662

    Article  Google Scholar 

  31. Wang L, Lu H, Wang Y, Feng M, Wang D, Yin B, Ruan X (2017) Learning to detect salient objects with image-level supervision. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 136–145

  32. Yang C, Zhang L, Lu H, Ruan X, Yang M. -H. (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3166–3173

  33. Li G, Yu Y (2015) Visual saliency based on multiscale deep features. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5455–5463

  34. Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1155–1162

  35. Li Y, Hou X, Koch C, Rehg JM, Yuille AL (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 280–287

  36. Movahedi V, Elder JH (2010) Design and perceptual validation of performance measures for salient object segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 49–56

  37. Zhou H, Xie X, Lai J-H, Chen Z, Yang L (2020) Interactive two-stream decoder for accurate and fast saliency detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9141–9150

  38. Zhang J, Xie J, Barnes N (2020) Learning noise-aware encoder-decoder from noisy labels by alternating back-propagation for saliency detection. In: Proceedings of the european conference on computer vision (ECCV), pp 349–366

  39. Liu Y, Gu Y-C, Zhang X-Y, Wang W, Cheng M-M (2020) Lightweight salient object detection via hierarchical visual perception learning. IEEE Trans Cybern 51(9):4439–4449

    Article  Google Scholar 

  40. Liu J-J, Hou Q, Cheng M-M (2020) Dynamic feature integration for simultaneous detection of salient object, edge, and skeleton. IEEE Trans Image Process 29:8652–8667

    Article  MATH  Google Scholar 

  41. Zeng Y, Zhuge Y, Lu H, Zhang L, Qian M, Yu Y (2019) Multi-source weak supervision for saliency detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6074–6083

  42. Feng M, Lu H, Ding E (2019) Attentive feedback network for boundary-aware salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1623–1632

  43. Wu R, Feng M, Guan W, Wang D, Lu H, Ding E (2019) A mutual learning method for salient object detection with intertwined multi-supervision. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8150– 8159

  44. Wang T, Zhang L, Wang S, Lu H, Yang G, Ruan X, Borji A (2018) Detect globally, refine locally: a novel approach to saliency detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3127–3135

  45. Chen S, Tan X, Wang B, Hu X (2018) Reverse attention for salient object detection. In: Proceedings of the European conference on computer vision (ECCV), pp 234–250

  46. Wang T, Borji A, Zhang L, Zhang P, Lu H (2017) A stagewise refinement model for detecting salient objects in images. In: Proceedings of the IEEE international conference on computer vision, pp 4019–4028

  47. Luo Z, Mishra A, Achkar A, Eichel J, Li S, Jodoin P-M (2017) Non-local deep features for salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6609–6617

  48. Zhang P, Wang D, Lu H, Wang H, Yin B (2017) Learning uncertain convolutional features for accurate saliency detection. In: Proceedings of the IEEE international conference on computer vision, pp 212–221

Download references

Funding

National Key Research and Development Program of China. No. 2018YFB1404402.

Author information

Authors and Affiliations

  1. School of Electronics and Information Engineering, Tongji University, Shanghai, China

    Cheng Xu, Xianhui Liu & Weidong Zhao

Authors
  1. Cheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xianhui Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weidong Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xianhui Liu.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, C., Liu, X. & Zhao, W. Attention-guided salient object detection using autoencoder regularization. Appl Intell 53, 6481–6495 (2023). https://doi.org/10.1007/s10489-022-03917-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03917-2

Keywords

Search

Navigation