research-article

FSNet: Frequency Domain Guided Superpixel Segmentation Network for Complex Scenes

Authors:

Wenhui WuAuthors Info & Claims

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Pages 4129 - 4137

https://doi.org/10.1145/3581783.3613826

Published: 27 October 2023 Publication History

Abstract

Existing superpixel segmentation algorithms mainly focus on natural image with high-quality, while neglecting the inevitable environment constraint in complex scenes. In this paper, we propose an end-to-end frequency domain guided superpixel segmentation network (FSNet) to generate superpixels with sharp boundary adherence for complex scenes by fusing the deep features in spatial and frequency domains. To utilize the frequency domain information of the image, an improved frequency information extractor (IFIE) is proposed to extract the frequency domain information with sharp boundary features. Moreover, considering the over-sharp feature may damage the semantic information of superpixel, we further design a dense hybrid atrous convolution (DHAC) block to preserve semantic information via capturing wider and deeper semantic information in spatial domain. Finally, the extracted deep features in spatial and frequency domains will be fused to generate semantic perceptual superpixels with sharp boundary adherence. Extensive experiments on multiple challenging datasets with complex boundaries demonstrate that our method achieves the state-of-the-art performance both quantitatively and qualitatively, and we further verify the superiority of the proposed method when applied in salient object detection.

Supplemental Material

MP4 File

Presentation video

Download
26.12 MB

References

[1]

Huazhu Fu, Xiaochun Cao, Dai Tang, Yahong Han, and Dong Xu. Regularity preserved superpixels and supervoxels. IEEE Trans. Multimedia., 16(4):1165--1175, 2014.

Digital Library

[2]

V. Jampani, D. Sun, M. Liu, M. Yang, and J. Kautz. Superpixel sampling networks. In Proc. Eur. Conf. Comput. Vis., pages 363--380, 2018.

Digital Library

[3]

Chong-Yi Li, Ji-Chang Guo, Run-Min Cong, Yan-Wei Pang, and Bo Wang. Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Trans. Image Process., 25(12):5664--5677, 2016.

Digital Library

[4]

Chunle Guo, Ruiqi Wu, Xin Jin, Linghao Han, Zhi Chai, Weidong Zhang, and Chongyi Li. Underwater ranker: Learn which is better and how to be better. In Assoc. Adv. Artif. Intell, 2023.

[5]

Sree Ramya S. P. Malladi, Sundaresh Ram, and Jeffrey J. Rodríguez. Image denoising using superpixel-based pca. IEEE Trans. Multimedia., 23:2297--2309, 2021.

Digital Library

[6]

Raj Kumar Gupta, Alex Yong-Sang Chia, Deepu Rajan, Ee Sin Ng, and Huang Zhiyong. Image colorization using similar images. In Proc. ACM Int. Conf. Multimedia, page 369--378. Association for Computing Machinery, 2012.

Digital Library

[7]

Zhixiang Ren, Yiqun Hu, Liang-Tien Chia, and Deepu Rajan. Improved saliency detection based on superpixel clustering and saliency propagation. In Proc. ACM Int. Conf. Multimedia, page 1099--1102. Association for Computing Machinery, 2010.

Digital Library

[8]

Runmin Cong, Jianjun Lei, Huazhu Fu, Fatih Porikli, Qingming Huang, and Chunping Hou. Video saliency detection via sparsity-based reconstruction and propagation. IEEE Trans. Image Process., 28(10):4819--4831, 2019.

[9]

Runmin Cong, Jianjun Lei, Huazhu Fu, Weisi Lin, Qingming Huang, Xiaochun Cao, and Chunping Hou. An iterative co-saliency framework for rgbd images. IEEE Trans. Cybern., 49(1):233--246, 2019.

[10]

Runmin Cong, Jianjun Lei, Huazhu Fu, Qingming Huang, Xiaochun Cao, and Nam Ling. Hscs: Hierarchical sparsity based co-saliency detection for rgbd images. IEEE Trans. Multimedia., 21(7):1660--1671, 2019.

[11]

Chongyi Li, Runmin Cong, Sam Kwong, Junhui Hou, Huazhu Fu, Guopu Zhu, Dingwen Zhang, and Qingming Huang. Asif-net: Attention steered interweave fusion network for rgb-d salient object detection. IEEE Trans. Cybern., 51(1):88--100, 2020.

[12]

Shuhui Bu, Zhenbao Liu, Junwei Han, and Jun Wu. Superpixel segmentation based structural scene recognition. In Proc. ACM Int. Conf. Multimedia, page 681--684. Association for Computing Machinery, 2013.

Digital Library

[13]

Peng Ying, Jin Liu, Hanqing Lu, and Songde Ma. Exclusive constrained discrimi-native learning for weakly-supervised semantic segmentation. In Proc. ACM Int. Conf. Multimedia, page 1251--1254. Association for Computing Machinery, 2015.

[14]

Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Süsstrunk. Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell., 34(11):2274--2282, 2012.

Digital Library

[15]

S. Lloyd. Least squares quantization in pcm. IEEE Trans. Inf. Theory, 28(2):129--137, 1982.

Digital Library

[16]

Hua Li, Yuheng Jia, Runmin Cong, Wenhui Wu, Sam Tak Wu Kwong, and Chuanbo Chen. Superpixel segmentation based on spatially constrained subspace clustering. IEEE Trans. Ind. Inform., 17(11):7501--7512, 2020.

[17]

Fengting Yang, Qian Sun, Hailin Jin, and Zihan Zhou. Superpixel segmentation with fully convolutional networks. In Proc. Conf. Comput. Vision Pattern Recognit., pages 13961--13970, 2020.

[18]

Yaxiong Wang, Yunchao Wei, Xueming Qian, Li Zhu, and Yi Yang. Ainet: Association implantation for superpixel segmentation. In Proc. Int. Conf. Comput. Vis., pages 7078--7087, 2021.

[19]

Tang, Dai and Fu, Huazhu and Cao, Xiaochun. Topology Preserved Regular Superpixel. In Proc. Int. Conf. Multimedia Expo., pages 765--768, 2012.

[20]

Jie Hu, Li Shen, Samuel Albanie, Gang Sun, and Enhua Wu. Squeeze-and-excitation networks. IEEE Trans. Pattern Anal. Mach. Intell., 42(8):2011--2023, 2020.

Digital Library

[21]

Yiwen Wei, Zhuoran Zheng, and Xiuyi Jia. Uhd underwater image enhancement via frequency-spatial domain aware network. In Proc. Asian Conf. Comput. Vis., pages 299--314, December 2022.

[22]

Michaël Gharbi, Jiawen Chen, Jonathan T. Barron, Samuel W. Hasinoff, and Frédo Durand. Deep bilateral learning for real-time image enhancement. ACM Trans. Graph., 36(4), jul 2017.

[23]

Cankun Zhong and Wing W. Y. Ng. A robust frequency-domain-based graph adaptive network for parkinson's disease detection from gait data. IEEE Trans. Multimedia, pages 1--14, 2022.

Digital Library

[24]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proc. Conf. Comput. Vision Pattern Recognit., pages 770--778, 2016.

[25]

Zaiwang Gu, Jun Cheng, Huazhu Fu, Kang Zhou, Huaying Hao, Yitian Zhao, Tianyang Zhang, Shenghua Gao, and Jiang Liu. Ce-net: Context encoder network for 2d medical image segmentation. IEEE Trans. Med. Imaging., 38(10):2281--2292, 2019.

[26]

Panqu Wang, Pengfei Chen, Ye Yuan, Ding Liu, Zehua Huang, Xiaodi Hou, and Garrison Cottrell. Understanding convolution for semantic segmentation. In Proc. Winter Conf. Appl. Comput. Vis., pages 1451--1460, 2018.

[27]

Pablo Arbeláez, Michael Maire, Charless Fowlkes, and Jitendra Malik. Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 33(5):898--916, 2011.

Digital Library

[28]

Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Semantic foggy scene understanding with synthetic data. Int. J. Comput. Vis, 126(9):973--992, Sep 2018.

Digital Library

[29]

Xiaowei Hu, Chi-Wing Fu, Lei Zhu, and Pheng-Ann Heng. Depth-attentional features for single-image rain removal. In Proc. Conf. Comput. Vision Pattern Recognit., pages 8014--8023, 2019.

[30]

Md Jahidul Islam, Chelsey Edge, Yuyang Xiao, Peigen Luo, Muntaqim Mehtaz, Christopher Morse, Sadman Sakib Enan, and Junaed Sattar. Semantic segmentation of underwater imagery: Dataset and benchmark. In Int. Conf. Intell. Robot. Syst., pages 1769--1776, 2020.

Digital Library

[31]

Ning Zhang, FC Nex, N Kerle, and G Vosselman. Lisu: Low-light indoor scene understanding with joint learning of reflectance restoration. Int. Soc. Photogramm. Remote Sens., 183:470--481, 2022.

[32]

M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The cityscapes dataset for semantic urban scene understanding. In Proc. Conf. Comput. Vision Pattern Recognit., pages 3213--3223, 2016.

[33]

Radhakrishna Achanta and Sabine Süsstrunk. Superpixels and polygons using simple non-iterative clustering. In Proc. Conf. Comput. Vision Pattern Recognit., pages 4895--4904, 2017.

[34]

Ming-Yu Liu, Oncel Tuzel, Srikumar Ramalingam, and Rama Chellappa. Entropy rate superpixel segmentation. In Proc. Conf. Comput. Vision Pattern Recognit., pages 2097--2104, 2011.

Digital Library

[35]

Jiansheng Chen, Zhengqin Li, and Bo Huang. Linear spectral clustering superpixel. IEEE Trans. Image Process., 26(7):3317--3330, 2017.

Digital Library

[36]

Shen, Jianbing and Du, Yunfan and Wang, Wenguan and Li, Xuelong. Lazy Random Walks for Superpixel Segmentation. IEEE Trans. Image Process., 23(4):1451--1462, 2014.

Digital Library

[37]

Yunchao Wei, Xiaodan Liang, Yunpeng Chen, Xiaohui Shen, Ming-Ming Cheng, Jiashi Feng, Yao Zhao, and Shuicheng Yan. Stc: A simple to complex framework for weakly-supervised semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 39(11):2314--2320, 2017.

Digital Library

[38]

Xiang Wang, Shaodi You, Xi Li, and Huimin Ma. Weakly-supervised semantic segmentation by iteratively mining common object features. In Proc. Conf. Comput. Vision Pattern Recognit., pages 1354--1362, 2018.

[39]

Yong Jae Lee, Joydeep Ghosh, and Kristen Grauman. Discovering important people and objects for egocentric video summarization. In Proc. Conf. Comput. Vision Pattern Recognit., pages 1346--1353, 2012.

[40]

Qing-Ge Ji, Zhi-Dang Fang, Zhen-Hua Xie, and Zhe-Ming Lu. Video abstraction based on the visual attention model and online clustering. Signal Process. Image Commun., 28(3):241--253, 2013.

Digital Library

[41]

Andrej Karpathy, Stephen Miller, and Li Fei-Fei. Object discovery in 3d scenes via shape analysis. In Proc. Int. Conf. Robot. Automat, pages 2088--2095. IEEE, 2013.

[42]

Simone Frintrop, Germán Martin García, and Armin B Cremers. A cognitive approach for object discovery. In Proc. Int. Conf. Pattern Recognit, pages 2329--2334. IEEE, 2014.

Digital Library

[43]

Wangjiang Zhu, Shuang Liang, Yichen Wei, and Jian Sun. Saliency optimization from robust background detection. In Proc. Conf. Comput. Vision Pattern Recognit., pages 2814--2821, 2014.

Digital Library

[44]

Md Jahidul Islam, Peigen Luo, and Junaed Sattar. Simultaneous Enhancement and Super-Resolution of Underwater Imagery for Improved Visual Perception. In Robot. Sci. Syst., July 2020.

[45]

Federico Perazzi, Philipp Krähenbühl, Yael Pritch, and Alexander Hornung. Saliency filters: Contrast based filtering for salient region detection. In Proc. Conf. Comput. Vision Pattern Recognit., pages 733--740, 2012.

Cited By

Hao JDuan WLi GChen SWu WLi H(2023)Multi-View Super Resolution for Underwater Images Utilizing Atmospheric Light Scattering Model2023 IEEE 29th International Conference on Parallel and Distributed Systems (ICPADS)10.1109/ICPADS60453.2023.00356(2690-2697)Online publication date: 17-Dec-2023
https://doi.org/10.1109/ICPADS60453.2023.00356
Liu YLiang HQiu RZhao S(2023)LDVNet: Lightweight and Detail-Aware Vision Network for Image Recognition Tasks in Resource-Constrained Environments2023 IEEE 29th International Conference on Parallel and Distributed Systems (ICPADS)10.1109/ICPADS60453.2023.00354(2673-2681)Online publication date: 17-Dec-2023
https://doi.org/10.1109/ICPADS60453.2023.00354

Index Terms

FSNet: Frequency Domain Guided Superpixel Segmentation Network for Complex Scenes
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems

Recommendations

Automatic brain tumor segmentation from magnetic resonance images using superpixel-based approach
Abstract
Cancer is the second leading cause of deaths worldwide, reported by World Health Organization (WHO). The abnormal growth of cells, which should die at the time but they remained in body organ which makes tumor and brain tumor is one of them. ...
Content-sensitive superpixel segmentation via self-organization-map neural network
Highlights
- Present a novel metric for the content-sensitiveness of superpixel.
- Proposed a ...
Abstract
Content-sensitive superpixel segmentation generates small superpixels in content-dense regions and large superpixels in content-sparse regions. It achieves higher segmentation accuracy than traditional superpixels. In this paper, we ...
A Fast Region Growing Based Superpixel Segmentation for Hyperspectral Image Classification
Pattern Recognition and Computer Vision
Abstract
In recent studies, superpixel segmentation has been integrated into hyperspectral (HS) image classification methods. However, the existing superpixel-based classification methods usually suffer from two serious problems. First, the accuracy and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

October 2023

9913 pages

ISBN:9798400701085

DOI:10.1145/3581783

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada & MBZUAI, UAE
,
Tao Mei
HiDream.ai, China
,
Rita Cucchiara
University of Modena and Reggio Emilia, Italy
,
Program Chairs:
Marco Bertini
University of Florence, Italy
,
Diana Patricia Tobon Vallejo
Unversidad de Medellin, Colombia
,
Pradeep K. Atrey
University at Albany, State University of New York, USA
,
M. Shamim Hossain
M. Shamim Hossain (King Saud University, KSA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '23

Sponsor:

SIGMM

MM '23: The 31st ACM International Conference on Multimedia

October 29 - November 3, 2023

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
227
Total Downloads

Downloads (Last 12 months)111
Downloads (Last 6 weeks)12

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hao JDuan WLi GChen SWu WLi H(2023)Multi-View Super Resolution for Underwater Images Utilizing Atmospheric Light Scattering Model2023 IEEE 29th International Conference on Parallel and Distributed Systems (ICPADS)10.1109/ICPADS60453.2023.00356(2690-2697)Online publication date: 17-Dec-2023
https://doi.org/10.1109/ICPADS60453.2023.00356
Liu YLiang HQiu RZhao S(2023)LDVNet: Lightweight and Detail-Aware Vision Network for Image Recognition Tasks in Resource-Constrained Environments2023 IEEE 29th International Conference on Parallel and Distributed Systems (ICPADS)10.1109/ICPADS60453.2023.00354(2673-2681)Online publication date: 17-Dec-2023
https://doi.org/10.1109/ICPADS60453.2023.00354

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten