skip to main content
10.1145/3581783.3612049acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

A Reference-free Self-supervised Domain Adaptation Framework for Low-quality Fundus Image Enhancement

Published: 27 October 2023 Publication History

Abstract

Retinal fundus images have been applied for the diagnosis and screening of eye diseases, such as Diabetic Retinopathy (DR) or Diabetic Macular Edema (DME). However, both low-quality fundus images and style inconsistency potentially increase uncertainty in the diagnosis of fundus disease and even lead to misdiagnosis by ophthalmologists. Most of the existing fundus image enhancement methods mainly focus on improving the image quality by leveraging the guidance of high-quality images, which is difficult to be collected in medical applications. In this paper, we tackle image quality enhancement in a fully unsupervised setting, i.e., neither paired images nor high-quality images. To this end, we explore the potential of the self-supervised task for improving the quality of fundus images without the requirement of high-quality reference images, and proposed a Domain Adaptation Self-supervised Quality Enhancement framework, named DASQE. Specifically, we construct multiple patch-wise domains via a well-designed rule-based quality assessment scheme and style clustering. To achieve robust low-quality image enhancement and address style inconsistency, we formulate two self-supervised domain adaptation tasks to disentangle the features of image content, low-quality factors and style information by exploring intrinsic supervision signals within the low-quality images. Extensive experiments are conducted on four benchmark datasets, and results show that our DASQE method achieves new state-of-the-art performance when only low-quality images are available.

References

[1]
Shane Barratt and Rishi Sharma. 2018. A note on the inception score. arXiv preprint arXiv:1801.01973 (2018).
[2]
Peng Cao, Qingshan Hou, Ruoxian Song, Haonan Wang, and Osmar Zaiane. 2022. Collaborative learning of weakly-supervised domain adaptation for diabetic retinopathy grading on retinal images. Computers in Biology and Medicine, Vol. 144 (2022), 105341.
[3]
Chenghao Chen and Hao Li. 2021. Robust representation learning with feedback for single image deraining. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 7742--7751.
[4]
Xiang Chen, Zhentao Fan, Pengpeng Li, Longgang Dai, Caihua Kong, Zhuoran Zheng, Yufeng Huang, and Yufeng Li. 2022. Unpaired Deep Image Dehazing Using Contrastive Disentanglement Learning. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XVII. Springer, 632--648.
[5]
Pujin Cheng, Li Lin, Yijin Huang, Junyan Lyu, and Xiaoying Tang. 2021. I-secret: Importance-guided fundus image enhancement via semi-supervised contrastive constraining. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 87--96.
[6]
Sung-Jin Cho, Seo-Won Ji, Jun-Pyo Hong, Seung-Won Jung, and Sung-Jea Ko. 2021. Rethinking coarse-to-fine approach in single image deblurring. In Proceedings of the IEEE/CVF international conference on computer vision. 4641--4650.
[7]
Ling Dai, Liang Wu, Huating Li, Chun Cai, Qiang Wu, Hongyu Kong, Ruhan Liu, Xiangning Wang, Xuhong Hou, Yuexing Liu, et al. 2021. A deep learning system for detecting diabetic retinopathy across the disease spectrum. Nature communications, Vol. 12, 1 (2021), 1--11.
[8]
Etienne Decencière, Xiwei Zhang, Guy Cazuguel, Bruno Lay, Béatrice Cochener, Caroline Trone, Philippe Gain, Richard Ordonez, Pascale Massin, Ali Erginay, et al. 2014. Feedback on a publicly distributed image database: the Messidor database. Image Analysis & Stereology, Vol. 33, 3 (2014), 231--234.
[9]
Huazhu Fu, Boyang Wang, Jianbing Shen, Shanshan Cui, Yanwu Xu, Jiang Liu, and Ling Shao. 2019. Evaluation of retinal image quality assessment networks in different color-spaces. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 48--56.
[10]
Xueyang Fu, Peixian Zhuang, Yue Huang, Yinghao Liao, Xiao-Ping Zhang, and Xinghao Ding. 2014. A retinex-based enhancing approach for single underwater image. In 2014 IEEE international conference on image processing (ICIP). IEEE, 4572--4576.
[11]
Zaiwang Gu, Jun Cheng, Huazhu Fu, Kang Zhou, Huaying Hao, Yitian Zhao, Tianyang Zhang, Shenghua Gao, and Jiang Liu. 2019. Ce-net: Context encoder network for 2d medical image segmentation. IEEE transactions on medical imaging, Vol. 38, 10 (2019), 2281--2292.
[12]
Jie Gui, Zhenan Sun, Yonggang Wen, Dacheng Tao, and Jieping Ye. 2021. A review on generative adversarial networks: Algorithms, theory, and applications. IEEE Transactions on Knowledge and Data Engineering (2021).
[13]
Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, and Runmin Cong. 2020a. Zero-reference deep curve estimation for low-light image enhancement. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 1780--1789.
[14]
Jifeng Guo, Zhiqi Pang, Fan Yang, Jiayou Shen, and Jian Zhang. 2020b. Study on the method of fundus image generation based on improved GAN. Mathematical Problems in Engineering, Vol. 2020 (2020), 1--13.
[15]
Xiaojie Guo, Yu Li, and Haibin Ling. 2016. LIME: Low-light image enhancement via illumination map estimation. IEEE Transactions on image processing, Vol. 26, 2 (2016), 982--993.
[16]
Kaiming He, Jian Sun, and Xiaoou Tang. 2010. Single image haze removal using dark channel prior. IEEE transactions on pattern analysis and machine intelligence, Vol. 33, 12 (2010), 2341--2353.
[17]
Alain Hore and Djemel Ziou. 2010. Image quality metrics: PSNR vs. SSIM. In 2010 20th international conference on pattern recognition. IEEE, 2366--2369.
[18]
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4700--4708.
[19]
Huimin Huang, Lanfen Lin, Ruofeng Tong, Hongjie Hu, Qiaowei Zhang, Yutaro Iwamoto, Xianhua Han, Yen-Wei Chen, and Jian Wu. 2020. Unet 3: A full-scale connected unet for medical image segmentation. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1055--1059.
[20]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1125--1134.
[21]
Shruti Jadon. 2020. A survey of loss functions for semantic segmentation. In 2020 IEEE conference on computational intelligence in bioinformatics and computational biology (CIBCB). IEEE, 1--7.
[22]
Qiangguo Jin, Zhaopeng Meng, Tuan D Pham, Qi Chen, Leyi Wei, and Ran Su. 2019. DUNet: A deformable network for retinal vessel segmentation. Knowledge-Based Systems, Vol. 178 (2019), 149--162.
[23]
Guisik Kim and Junseok Kwon. 2022. Self-Parameter Distillation Dehazing. IEEE Transactions on Image Processing (2022).
[24]
Orest Kupyn, Tetiana Martyniuk, Junru Wu, and Zhangyang Wang. 2019. Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. In Proceedings of the IEEE/CVF international conference on computer vision. 8878--8887.
[25]
Tao Li, Yingqi Gao, Kai Wang, Song Guo, Hanruo Liu, and Hong Kang. 2019a. Diagnostic assessment of deep learning algorithms for diabetic retinopathy screening. Information Sciences, Vol. 501 (2019), 511--522.
[26]
Xiaomeng Li, Xiaowei Hu, Lequan Yu, Lei Zhu, Chi-Wing Fu, and Pheng-Ann Heng. 2019b. CANet: cross-disease attention network for joint diabetic retinopathy and diabetic macular edema grading. IEEE transactions on medical imaging, Vol. 39, 5 (2019), 1483--1493.
[27]
Ke Liang, Yue Liu, Sihang Zhou, Wenxuan Tu, Yi Wen, Xihong Yang, Xiangjun Dong, and Xinwang Liu. 2023. Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure. IEEE Transactions on Knowledge and Data Engineering (2023).
[28]
Ke Liang, Lingyuan Meng, Meng Liu, Yue Liu, Wenxuan Tu, Siwei Wang, Sihang Zhou, Xinwang Liu, and Fuchun Sun. 2022. Reasoning over Different Types of Knowledge Graphs: Static, Temporal and Multi-Modal. arXiv preprint arXiv:2212.05767 (2022).
[29]
Ming-Yu Liu, Thomas Breuel, and Jan Kautz. 2017. Unsupervised image-to-image translation networks. Advances in neural information processing systems, Vol. 30 (2017).
[30]
Long Ma, Tengyu Ma, Risheng Liu, Xin Fan, and Zhongxuan Luo. 2022. Toward fast, flexible, and robust low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5637--5646.
[31]
Yuhui Ma, Jiang Liu, Yonghuai Liu, Huazhu Fu, Yan Hu, Jun Cheng, Hong Qi, Yufei Wu, Jiong Zhang, and Yitian Zhao. 2021. Structure and illumination constrained GAN for medical image enhancement. IEEE Transactions on Medical Imaging, Vol. 40, 12 (2021), 3955--3967.
[32]
Taesung Park, Alexei A Efros, Richard Zhang, and Jun-Yan Zhu. 2020. Contrastive learning for unpaired image-to-image translation. In European conference on computer vision. Springer, 319--345.
[33]
Prasanna Porwal, Samiksha Pachade, Ravi Kamble, Manesh Kokare, Girish Deshmukh, Vivek Sahasrabuddhe, and Fabrice Meriaudeau. 2018. Indian diabetic retinopathy image dataset (IDRiD): a database for diabetic retinopathy screening research. Data, Vol. 3, 3 (2018), 25.
[34]
Ziyi Shen, Huazhu Fu, Jianbing Shen, and Ling Shao. 2020. Modeling and enhancing low-quality retinal fundus images. IEEE transactions on medical imaging, Vol. 40, 3 (2020), 996--1006.
[35]
Saikat Kumar Shome and Siva Ram Krishna Vadali. 2011. Enhancement of diabetic retinopathy imagery using contrast limited adaptive histogram equalization. International Journal of Computer Science and Information Technologies, Vol. 2, 6 (2011), 2694--2699.
[36]
Zhen Ling Teo, Yih-Chung Tham, Marco Yu, Miao Li Chee, Tyler Hyungtaek Rim, Ning Cheung, Mukharram M Bikbov, Ya Xing Wang, Yating Tang, Yi Lu, et al. 2021. Global prevalence of diabetic retinopathy and projection of burden through 2045: systematic review and meta-analysis. Ophthalmology, Vol. 128, 11 (2021), 1580--1591.
[37]
Qi-Chong Tian and Laurent D Cohen. 2017. Global and local contrast adaptive enhancement for non-uniform illumination color images. In Proceedings of the IEEE International Conference on Computer Vision Workshops. 3023--3030.
[38]
Xinhang Wan, Jiyuan Liu, Weixuan Liang, Xinwang Liu, Yi Wen, and En Zhu. 2022. Continual Multi-View Clustering. In Proceedings of the 30th ACM International Conference on Multimedia (Lisboa, Portugal) (MM '22). Association for Computing Machinery, New York, NY, USA, 3676--3684. https://doi.org/10.1145/3503161.3547864
[39]
Xinhang Wan, Xinwang Liu, Jiyuan Liu, Siwei Wang, Yi Wen, Weixuan Liang, En Zhu, Zhe Liu, and Lu Zhou. 2023. Auto-weighted Multi-view Clustering for Large-scale Data. arxiv: 2303.01983 [cs.LG]
[40]
Renzhen Wang, Benzhi Chen, Deyu Meng, and Lisheng Wang. 2018. Weakly supervised lesion detection from fundus images. IEEE transactions on medical imaging, Vol. 38, 6 (2018), 1501--1512.
[41]
Xiaofei Wang, Mai Xu, Jicong Zhang, Lai Jiang, Liu Li, Mengxian He, Ningli Wang, Hanruo Liu, and Zulin Wang. 2021. Joint Learning of Multi-Level Tasks for Diabetic Retinopathy Grading on Low-Resolution Fundus Images. IEEE Journal of Biomedical and Health Informatics, Vol. 26, 5 (2021), 2216--2227.
[42]
Yufei Wang, Renjie Wan, Wenhan Yang, Haoliang Li, Lap-Pui Chau, and Alex Kot. 2022. Low-light image enhancement with normalizing flow. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 2604--2612.
[43]
Wenhui Wu, Jian Weng, Pingping Zhang, Xu Wang, Wenhan Yang, and Jianmin Jiang. 2022. Uretinex-net: Retinex-based deep unfolding network for low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5901--5910.
[44]
Xihong Yang, Xiaochang Hu, Sihang Zhou, Xinwang Liu, and En Zhu. 2022. Interpolation-based contrastive learning for few-label semi-supervised learning. IEEE Transactions on Neural Networks and Learning Systems (2022).
[45]
Xihong Yang, Yue Liu, Sihang Zhou, Siwei Wang, Wenxuan Tu, Qun Zheng, Xinwang Liu, Liming Fang, and En Zhu. 2023. Cluster-guided Contrastive Graph Clustering Network. In Proceedings of the AAAI conference on artificial intelligence, Vol. 37. 10834--10842.
[46]
Qijing You, Cheng Wan, Jing Sun, Jianxin Shen, Hui Ye, and Qiuli Yu. 2019. Fundus image enhancement method based on CycleGAN. In 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, 4500--4503.
[47]
He Zhao, Bingyu Yang, Lvchen Cao, and Huiqi Li. 2019. Data-driven enhancement of blurry retinal images via generative adversarial networks. In Medical Image Computing and Computer Assisted Intervention-MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13-17, 2019, Proceedings, Part I 22. Springer, 75--83.
[48]
Mei Zhou, Kai Jin, Shaoze Wang, Juan Ye, and Dahong Qian. 2017. Color retinal image enhancement based on luminosity and contrast adjustment. IEEE Transactions on Biomedical engineering, Vol. 65, 3 (2017), 521--527.
[49]
Yi Zhou, Boyang Wang, Lei Huang, Shanshan Cui, and Ling Shao. 2020. A benchmark for studying diabetic retinopathy: segmentation, grading, and transferability. IEEE Transactions on Medical Imaging, Vol. 40, 3 (2020), 818--828.
[50]
Yuqian Zhou, Hanchao Yu, and Humphrey Shi. 2021. Study group learning: Improving retinal vessel segmentation trained with noisy labels. In Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part I 24. Springer, 57--67.

Cited By

View all
  • (2024)A comprehensive review of artificial intelligence models for screening major retinal diseasesArtificial Intelligence Review10.1007/s10462-024-10736-z57:5Online publication date: 5-Apr-2024
  • (2024)A Clinical-Oriented Lightweight Network for High-Resolution Medical Image EnhancementMedical Image Computing and Computer Assisted Intervention – MICCAI 202410.1007/978-3-031-72384-1_1(3-12)Online publication date: 3-Oct-2024
  • (2023)A Diffusion Model-Based Joint Dual-Task Network for Low-Quality Retinal Image Enhancement and Vessel Segmentation2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM58861.2023.10385623(2107-2110)Online publication date: 5-Dec-2023

Index Terms

  1. A Reference-free Self-supervised Domain Adaptation Framework for Low-quality Fundus Image Enhancement

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      MM '23: Proceedings of the 31st ACM International Conference on Multimedia
      October 2023
      9913 pages
      ISBN:9798400701085
      DOI:10.1145/3581783
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 27 October 2023

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. domain adaptation
      2. fundus image quality enhancement
      3. reference-free
      4. self-supervised

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      MM '23
      Sponsor:
      MM '23: The 31st ACM International Conference on Multimedia
      October 29 - November 3, 2023
      Ottawa ON, Canada

      Acceptance Rates

      Overall Acceptance Rate 2,090 of 8,270 submissions, 25%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)117
      • Downloads (Last 6 weeks)6
      Reflects downloads up to 22 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)A comprehensive review of artificial intelligence models for screening major retinal diseasesArtificial Intelligence Review10.1007/s10462-024-10736-z57:5Online publication date: 5-Apr-2024
      • (2024)A Clinical-Oriented Lightweight Network for High-Resolution Medical Image EnhancementMedical Image Computing and Computer Assisted Intervention – MICCAI 202410.1007/978-3-031-72384-1_1(3-12)Online publication date: 3-Oct-2024
      • (2023)A Diffusion Model-Based Joint Dual-Task Network for Low-Quality Retinal Image Enhancement and Vessel Segmentation2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM58861.2023.10385623(2107-2110)Online publication date: 5-Dec-2023

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media