skip to main content
research-article

Co-saliency Detection with Graph Matching

Published: 12 April 2019 Publication History

Abstract

Recently, co-saliency detection, which aims to automatically discover common and salient objects appeared in several relevant images, has attracted increased interest in the computer vision community. In this article, we present a novel graph-matching based model for co-saliency detection in image pairs. A solution of graph matching is proposed to integrate the visual appearance, saliency coherence, and spatial structural continuity for detecting co-saliency collaboratively. Since the saliency and the visual similarity have been seamlessly integrated, such a joint inference schema is able to produce more accurate and reliable results. More concretely, the proposed model first computes the intra-saliency for each image by aggregating multiple saliency cues. The common and salient regions across multiple images are thus discovered via a graph matching procedure. Then, a graph reconstruction scheme is proposed to refine the intra-saliency iteratively. Compared to existing co-saliency detection methods that only utilize visual appearance cues, our proposed model can effectively exploit both visual appearance and structure information to better guide co-saliency detection. Extensive experiments on several challenging image pair databases demonstrate that our model outperforms state-of-the-art baselines significantly.

References

[1]
Xiaochun Cao, Zhiqiang Tao, Bao Zhang, Huazhu Fu, and Wei Feng. 2014. Self-adaptively weighted co-saliency detection via rank constraint. IEEE Trans. Image Proc. 23, 9 (2014), 4175--4186.
[2]
Hwann Tzong Chen. 2010. Preattentive co-saliency detection. In Proceedings of the IEEE International Conference on Image Processing. 1117--1120.
[3]
Yi Lei Chen and Chiou Ting Hsu. 2014. Implicit rank-sparsity decomposition: Applications to saliency/co-saliency detection. In Proceedings of the IEEE International Conference on Computer Vision. 2305--2310.
[4]
Ming Ming Cheng, Jonathan Warrell, Wen Yan Lin, Shuai Zheng, Vibhav Vineet, and Nigel Crook. 2013. Efficient salient region detection with soft image abstraction. In Proceedings of the IEEE International Conference on Computer Vision. 1529--1536.
[5]
Minsu Cho, Jungmin Lee, and Kyoung Mu Lee. 2010. Reweighted random walks for graph matching. In Proceedings of the European Conference on Computer Vision. 492--505.
[6]
Zhang Dingwen, Fu Huazhu, Han Junwei, Borji Ali, and Li Xuelong. 2018. A review of co-saliency detection algorithms: Fundamentals, applications, and challenges. ACM Trans. Intell. Syst. Technol. 9, 38 (2018), 1--31.
[7]
Pedro F. Felzenszwalb and Daniel P. Huttenlocher. 2004. Efficient graph-based image segmentation. Int. J. Comput. Vis. 59, 2 (2004), 167--181.
[8]
Huazhu Fu, Xiaochun Cao, and Zhuowen Tu. 2013. Cluster-based co-saliency detection. IEEE Trans. Image Proc. 22, 10 (2013), 3766--3778.
[9]
J. Han, G. Cheng, Z. Li, and D. Zhang. 2018. A unified metric learning-based framework for co-saliency detection. IEEE Trans. Circ. Syst. Vid. Technol. 28, 10 (2018), 2473--2483.
[10]
Jonathan Harel, Christof Koch, and Pietro Perona. 2006. Graph-based visual saliency. In Proceedings of the International Conference on Neural Information Processing Systems. 545--552.
[11]
Kuang-Jui Hsu, Chung-Chi Tsai, Yen-Yu Lin, Xiaoning Qian, and Yung-Yu Chuang. 2018. Unsupervised CNN-based co-saliency detection with graphical optimization. In Proceedings of the European Conference on Computer Vision. 485--501.
[12]
Rui Huang, Wei Feng, and Jizhou Sun. 2015. Saliency and co-saliency detection by low-rank multiscale fusion. In Proceedings of the IEEE International Conference on Multimedia and Expo. 1--6.
[13]
Laurent Itti and Ali Borji. 2012. Exploiting local and global patch rarities for saliency detection. In Proceedings of the IEEE International Conference on Computer Vision. 478--485.
[14]
David E. Jacobs, B. Goldman Dan, and Eli Shechtman. 2010. Cosaliency: Where people look when comparing images. In Proceedings of the ACM Symposium on User Interface Software and Technology. 219--228.
[15]
Koteswar Rao Jerripothula, Jianfei Cai, and Junsong Yuan. 2016. Image co-segmentation via saliency co-fusion. IEEE Trans. Multi. 18, 9 (2016), 1896--1909.
[16]
Huaizu Jiang, Jingdong Wang, Zejian Yuan, Yang Wu, Nanning Zheng, and Shipeng Li. 2013. Salient object detection: A discriminative regional feature integration approach. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2083--2090.
[17]
Marius Leordeanu, Martial Hebert, and Rahul Sukthankar. 2009. An integer projected fixed point method for graph matching and MAP inference. In Proceedings of the International Conference on Neural Information Processing Systems. 1114--1122.
[18]
Hongliang Li, Fanman Meng, and King Ngi Ngan. 2013. Co-salient object detection from multiple images. IEEE Trans. Multi. 15, 8 (2013), 1896--1909.
[19]
Hongliang Li and King Ngi Ngan. 2011. A co-saliency model of image pairs. IEEE Trans. Image Proc. 20, 12 (2011), 3365--3375.
[20]
Teng Li, Bin Cheng, Bingbing Ni, Guangchan Liu, and Shuicheng Yan. 2016. Multitask low-rank affinity graph for image segmentation and image annotation. ACM Trans. Intell. Syst. Technol. 7, 4 (2016), 1--18.
[21]
Xi Li, Weiming Hu, Chunhua Shen, Zhongfei Zhang, Anthony Dick, and Anton Van Den Hengel. 2013. A survey of appearance models in visual object tracking. ACM Trans. Intell. Syst. Technol. 4, 4 (2013), 58.
[22]
Xi Li, Yao Li, Chunhua Shen, Anthony Dick, and Anton Van Den Hengel. 2013. Contextual hypergraph modeling for salient object detection. In Proceedings of the IEEE International Conference on Computer Vision. 3328--3335.
[23]
Xiaohui Li, Huchuan Lu, Lihe Zhang, Xiang Ruan, and Ming-Hsuan Yang. 2013. Saliency detection via dense and sparse reconstruction. In Proceedings of the IEEE International Conference on Computer Vision. 2976--2983.
[24]
Yijun Li, Keren Fu, Zhi Liu, and Jie Yang. 2015. Efficient saliency-model-guided visual co-saliency detection. IEEE Sig. Proc. Lett. 22, 5 (2015), 588--592.
[25]
Zhi Liu, Wenbin Zou, Lina Li, Liquan Shen, and Olivier Le Meur. 2013. Co-saliency detection based on hierarchical segmentation. IEEE Sig. Proc. Lett. 21, 1 (2013), 88--92.
[26]
Zhi Liu, Wenbin Zou, and Olivier Le Meur. 2014. Saliency tree: A novel saliency detection framework. IEEE Trans. Image Proc. 23, 5 (2014), 1937--1952.
[27]
Fanman Meng, Hongliang Li, and Guanghui Liu. 2013. A new co-saliency model via pairwise constraint graph matching. In Proceedings of the International Symposium on Intelligent Signal Processing and Communications Systems. 781--786.
[28]
Cheng Mingming, Zhang Guoxin, N. J. Mitra, Huang Xiaolei, and Hu ShiMin. 2015. Global contrast based salient region detection. IEEE Trans. Patt. Anal. Mach. Intell. 37, 3 (2015), 569--582.
[29]
James Munkres. 1957. Algorithms for the assignment and transportation problems. J. Soc. Indust. Appl. Math. 5, 1 (1957), 32--38.
[30]
Devi Parikh, Jiebo Luo, Dhruv Batra, Adarsh Kowdle, and Tsuhan Chen. 2010. iCoseg: Interactive co-segmentation with intelligent scribble guidance. In Proceedings of the IEEE International Conference on Computer Vision (2010), 3169--3176.
[31]
H. Peng, B. Li, H. Ling, W. Hu, W. Xiong, and S. J. Maybank. 2016. Salient object detection via structured matrix decomposition. IEEE Trans. Patt. Anal. Mach. Intell. 39, 4 (2016), 818--832.
[32]
Federico Perazzi, Philipp Krähenbühl, Yael Pritch, and Alexander Hornung. 2012. Saliency filters: Contrast based filtering for salient region detection. In Proceedings of the Conference on Computer Vision and Pattern Recognition. 733--740.
[33]
Francesc Serratosa. 2014. Fast computation of bipartite graph matching. Patt. Recog. Lett. 45, 8 (2014), 244--250.
[34]
Yicheng Song, Yongdong Zhang, Juan Cao, Jinhui Tang, Xingyu Gao, and Jintao Li. 2014. A unified geolocation framework for web videos. ACM Trans. Intell. Syst. Technol. 5, 3 (2014), 49.
[35]
Zhiyu Tan, Liang Wan, Wei Feng, and Chi Man Pun. 2013. Image co-saliency detection by propagating superpixel affinities. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. 2114--2118.
[36]
Wang Tao and Haibin Ling. 2017. Gracker: A graph-based planar object tracker. IEEE Trans. Patt. Anal. Mach. Intell. 40, 6 (2017), 1494--1501.
[37]
Yu Tian, Junchi Yan, Hequan Zhang, Ya Zhang, Xiaokang Yang, and Hongyuan Zha. 2012. On the convergence of graph matching: Graduated assignment revisited. In Proceedings of the European Conference on Computer Vision. 821--835.
[38]
Alexander Toshev, Jianbo Shi, and Kostas Daniilidis. 2007. Image matching via saliency region correspondences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--8.
[39]
C. Tsai, W. Li, K. Hsu, X. Qian, and Y. Lin. 2019. Image co-saliency detection and co-segmentation via progressive joint optimization. IEEE Trans. Image Proc. 28, 1 (2019), 56--71.
[40]
Chung Chi Tsai, Xiaoning Qian, and Yen Yu Lin. 2017. Image co-saliency detection via locally adaptive saliency map fusion. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 1897--1901.
[41]
Tao Wang, Haibin Ling, and Congyan Lang. 2016. Branching path following for graph matching. In Proceedings of the IEEE Conference European Conference on Computer Vision. 508--523.
[42]
Tao Wang, Haibin Ling, Congyan Lang, and Songhe Feng. 2018. Graph matching with adaptive and branching path following. IEEE Trans. Patt. Anal. Mach. Intell. 40 (2018), 2853--2867.
[43]
Lina Wei, Shanshan Zhao, Omar El Farouk Bourahla, Xi Li, and Fei Wu. 2017. Group-wise deep co-saliency detection. In Proceedings of the International Joint Conferences on Artificial Intelligence Organization. 3041--3047.
[44]
Yunchao Wei, Xiaodan Liang, Yunpeng Chen, Xiaohui Shen, Ming Ming Cheng, Jiashi Feng, Yao Zhao, and Shuicheng Yan. 2016. STC: A simple to complex framework for weakly supervised semantic segmentation. IEEE Trans. Patt. Anal. Mach. Intell. 39, 11 (2016), 2314--2320.
[45]
Yichen Wei, Fang Wen, Wangjiang Zhu, and Jian Sun. 2017. Geodesic saliency using background priors. In Proceedings of the IEEE European Conference on Computer Vision. 29--42.
[46]
Qiong Yan, Li Xu, Jianping Shi, and Jiaya Jia. 2013. Hierarchical saliency detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1155--1162.
[47]
Chuan Yang, Lihe Zhang, Huchuan Lu, Ruan Xiang, and Ming Hsuan Yang. 2013. Saliency detection via graph-based manifold ranking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3166--3173.
[48]
Linwei Ye, Zhi Liu, Junhao Li, WanLei Zhao, and Liquan Shen. 2015. Co-saliency detection via co-salient object discovery and recovery. IEEE Sig. Proc. Lett. 22, 11 (2015), 2073--2077.
[49]
Dingwen Zhang, Junwei Han, Jungong Han, and Ling Shao. 2016. Co-saliency detection based on intrasaliency prior transfer and deep intersaliency mining. IEEE Trans. Neural Netw. Learn. Syst. 27, 6 (2016), 1163--1176.
[50]
Dingwen Zhang, Junwei Han, Chao Li, and Jingdong Wang. 2015. Co-saliency detection via looking deep and wide. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2994--3002.
[51]
D. Zhang, J. Han, and Y. Zhang. 2017. Supervision by fusion: Towards unsupervised learning of deep salient object detector. In Proceedings of the IEEE International Conference on Computer Vision. 4068--4076.
[52]
Dingwen Zhang, Deyu Meng, Chao Li, Lu Jiang, Qian Zhao, and Junwei Han. 2016. A self-paced multiple-instance learning framework for co-saliency detection. In Proceedings of the IEEE International Conference on Computer Vision. 594--602.
[53]
Lihe Zhang, Chuan Yang, Huchuan Lu, Ruan Xiang, and Ming Hsuan Yang. 2016. Ranking saliency. IEEE Trans. Patt. Anal. Mach. Intell. 39, 99 (2016), 1892--1904.
[54]
Yuting Zhang, Xueming Qian, Xianglong Tan, Junwei Han, and Yuanyan Tang. 2016. Sketch-based image retrieval by salient contour reinforcement. IEEE Trans. Multi. 18, 8 (2016), 1604--1615.
[55]
Xiaoju Zheng, Zheng-Jun Zha, and Liansheng Zhuang. 2018. A feature-adaptive semi-supervised framework for co-saliency detection. In ACM Multimedia. 959--966.
[56]
Wangjiang Zhu, Shuang Liang, Yichen Wei, and Jian Sun. 2014. Saliency optimization from robust background detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2814--2821.

Cited By

View all
  • (2024)Mutual Information Regularization for Weakly-Supervised RGB-D Salient Object DetectionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.328524934:1(397-410)Online publication date: 1-Jan-2024
  • (2023)Bidirectional mutual guidance transformer for salient object detection in optical remote sensing imagesInternational Journal of Remote Sensing10.1080/01431161.2023.222949444:13(4016-4033)Online publication date: 14-Jul-2023
  • (2022)Re-Thinking the Relations in Co-Saliency DetectionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2022.315092332:8(5453-5466)Online publication date: Aug-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology
ACM Transactions on Intelligent Systems and Technology  Volume 10, Issue 3
Survey Paper, Research Commentary and Regular Papers
May 2019
302 pages
ISSN:2157-6904
EISSN:2157-6912
DOI:10.1145/3325195
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 April 2019
Accepted: 01 February 2019
Revised: 01 February 2019
Received: 01 May 2018
Published in TIST Volume 10, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Co-saliency detection
  2. Computer vision
  3. graph matching model
  4. graph reconstruction
  5. image understanding

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

  • Fundamental Research Funds for the Central Universities

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)11
  • Downloads (Last 6 weeks)1
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Mutual Information Regularization for Weakly-Supervised RGB-D Salient Object DetectionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.328524934:1(397-410)Online publication date: 1-Jan-2024
  • (2023)Bidirectional mutual guidance transformer for salient object detection in optical remote sensing imagesInternational Journal of Remote Sensing10.1080/01431161.2023.222949444:13(4016-4033)Online publication date: 14-Jul-2023
  • (2022)Re-Thinking the Relations in Co-Saliency DetectionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2022.315092332:8(5453-5466)Online publication date: Aug-2022
  • (2022)Co-saliency-regularized correlation filter for object trackingSignal Processing: Image Communication10.1016/j.image.2022.116655(116655)Online publication date: Feb-2022
  • (2021)GTAE: Graph Transformer–Based Auto-Encoders for Linguistic-Constrained Text Style TransferACM Transactions on Intelligent Systems and Technology10.1145/344873312:3(1-16)Online publication date: 15-Jun-2021
  • (2021)Consistent image processing based on co‐saliencyCAAI Transactions on Intelligence Technology10.1049/cit2.120206:3(324-337)Online publication date: 5-Mar-2021
  • (2021)Multi-frame co-saliency spatio-temporal regularization correlation filters for object trackingJournal of Visual Communication and Image Representation10.1016/j.jvcir.2021.103329(103329)Online publication date: Oct-2021
  • (2020)End-to-End Text-to-Image Synthesis with Spatial ConstrainsACM Transactions on Intelligent Systems and Technology10.1145/339170911:4(1-19)Online publication date: 25-May-2020
  • (2020)Superpixel Region Merging Based on Deep Network for Medical Image SegmentationACM Transactions on Intelligent Systems and Technology10.1145/338609011:4(1-22)Online publication date: 31-May-2020
  • (2020)Adaptive Graph Convolutional Network With Attention Graph Clustering for Co-Saliency Detection2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR42600.2020.00907(9047-9056)Online publication date: Jun-2020

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media