skip to main content
10.1145/3581783.3612049acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

A Reference-free Self-supervised Domain Adaptation Framework for Low-quality Fundus Image Enhancement

Published:27 October 2023Publication History

ABSTRACT

Retinal fundus images have been applied for the diagnosis and screening of eye diseases, such as Diabetic Retinopathy (DR) or Diabetic Macular Edema (DME). However, both low-quality fundus images and style inconsistency potentially increase uncertainty in the diagnosis of fundus disease and even lead to misdiagnosis by ophthalmologists. Most of the existing fundus image enhancement methods mainly focus on improving the image quality by leveraging the guidance of high-quality images, which is difficult to be collected in medical applications. In this paper, we tackle image quality enhancement in a fully unsupervised setting, i.e., neither paired images nor high-quality images. To this end, we explore the potential of the self-supervised task for improving the quality of fundus images without the requirement of high-quality reference images, and proposed a Domain Adaptation Self-supervised Quality Enhancement framework, named DASQE. Specifically, we construct multiple patch-wise domains via a well-designed rule-based quality assessment scheme and style clustering. To achieve robust low-quality image enhancement and address style inconsistency, we formulate two self-supervised domain adaptation tasks to disentangle the features of image content, low-quality factors and style information by exploring intrinsic supervision signals within the low-quality images. Extensive experiments are conducted on four benchmark datasets, and results show that our DASQE method achieves new state-of-the-art performance when only low-quality images are available.

References

  1. Shane Barratt and Rishi Sharma. 2018. A note on the inception score. arXiv preprint arXiv:1801.01973 (2018).Google ScholarGoogle Scholar
  2. Peng Cao, Qingshan Hou, Ruoxian Song, Haonan Wang, and Osmar Zaiane. 2022. Collaborative learning of weakly-supervised domain adaptation for diabetic retinopathy grading on retinal images. Computers in Biology and Medicine, Vol. 144 (2022), 105341.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Chenghao Chen and Hao Li. 2021. Robust representation learning with feedback for single image deraining. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 7742--7751.Google ScholarGoogle ScholarCross RefCross Ref
  4. Xiang Chen, Zhentao Fan, Pengpeng Li, Longgang Dai, Caihua Kong, Zhuoran Zheng, Yufeng Huang, and Yufeng Li. 2022. Unpaired Deep Image Dehazing Using Contrastive Disentanglement Learning. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XVII. Springer, 632--648.Google ScholarGoogle Scholar
  5. Pujin Cheng, Li Lin, Yijin Huang, Junyan Lyu, and Xiaoying Tang. 2021. I-secret: Importance-guided fundus image enhancement via semi-supervised contrastive constraining. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 87--96.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Sung-Jin Cho, Seo-Won Ji, Jun-Pyo Hong, Seung-Won Jung, and Sung-Jea Ko. 2021. Rethinking coarse-to-fine approach in single image deblurring. In Proceedings of the IEEE/CVF international conference on computer vision. 4641--4650.Google ScholarGoogle ScholarCross RefCross Ref
  7. Ling Dai, Liang Wu, Huating Li, Chun Cai, Qiang Wu, Hongyu Kong, Ruhan Liu, Xiangning Wang, Xuhong Hou, Yuexing Liu, et al. 2021. A deep learning system for detecting diabetic retinopathy across the disease spectrum. Nature communications, Vol. 12, 1 (2021), 1--11.Google ScholarGoogle Scholar
  8. Etienne Decencière, Xiwei Zhang, Guy Cazuguel, Bruno Lay, Béatrice Cochener, Caroline Trone, Philippe Gain, Richard Ordonez, Pascale Massin, Ali Erginay, et al. 2014. Feedback on a publicly distributed image database: the Messidor database. Image Analysis & Stereology, Vol. 33, 3 (2014), 231--234.Google ScholarGoogle ScholarCross RefCross Ref
  9. Huazhu Fu, Boyang Wang, Jianbing Shen, Shanshan Cui, Yanwu Xu, Jiang Liu, and Ling Shao. 2019. Evaluation of retinal image quality assessment networks in different color-spaces. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 48--56.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Xueyang Fu, Peixian Zhuang, Yue Huang, Yinghao Liao, Xiao-Ping Zhang, and Xinghao Ding. 2014. A retinex-based enhancing approach for single underwater image. In 2014 IEEE international conference on image processing (ICIP). IEEE, 4572--4576.Google ScholarGoogle ScholarCross RefCross Ref
  11. Zaiwang Gu, Jun Cheng, Huazhu Fu, Kang Zhou, Huaying Hao, Yitian Zhao, Tianyang Zhang, Shenghua Gao, and Jiang Liu. 2019. Ce-net: Context encoder network for 2d medical image segmentation. IEEE transactions on medical imaging, Vol. 38, 10 (2019), 2281--2292.Google ScholarGoogle Scholar
  12. Jie Gui, Zhenan Sun, Yonggang Wen, Dacheng Tao, and Jieping Ye. 2021. A review on generative adversarial networks: Algorithms, theory, and applications. IEEE Transactions on Knowledge and Data Engineering (2021).Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, and Runmin Cong. 2020a. Zero-reference deep curve estimation for low-light image enhancement. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 1780--1789.Google ScholarGoogle ScholarCross RefCross Ref
  14. Jifeng Guo, Zhiqi Pang, Fan Yang, Jiayou Shen, and Jian Zhang. 2020b. Study on the method of fundus image generation based on improved GAN. Mathematical Problems in Engineering, Vol. 2020 (2020), 1--13.Google ScholarGoogle Scholar
  15. Xiaojie Guo, Yu Li, and Haibin Ling. 2016. LIME: Low-light image enhancement via illumination map estimation. IEEE Transactions on image processing, Vol. 26, 2 (2016), 982--993.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Kaiming He, Jian Sun, and Xiaoou Tang. 2010. Single image haze removal using dark channel prior. IEEE transactions on pattern analysis and machine intelligence, Vol. 33, 12 (2010), 2341--2353.Google ScholarGoogle Scholar
  17. Alain Hore and Djemel Ziou. 2010. Image quality metrics: PSNR vs. SSIM. In 2010 20th international conference on pattern recognition. IEEE, 2366--2369.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4700--4708.Google ScholarGoogle ScholarCross RefCross Ref
  19. Huimin Huang, Lanfen Lin, Ruofeng Tong, Hongjie Hu, Qiaowei Zhang, Yutaro Iwamoto, Xianhua Han, Yen-Wei Chen, and Jian Wu. 2020. Unet 3: A full-scale connected unet for medical image segmentation. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1055--1059.Google ScholarGoogle ScholarCross RefCross Ref
  20. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1125--1134.Google ScholarGoogle ScholarCross RefCross Ref
  21. Shruti Jadon. 2020. A survey of loss functions for semantic segmentation. In 2020 IEEE conference on computational intelligence in bioinformatics and computational biology (CIBCB). IEEE, 1--7.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Qiangguo Jin, Zhaopeng Meng, Tuan D Pham, Qi Chen, Leyi Wei, and Ran Su. 2019. DUNet: A deformable network for retinal vessel segmentation. Knowledge-Based Systems, Vol. 178 (2019), 149--162.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Guisik Kim and Junseok Kwon. 2022. Self-Parameter Distillation Dehazing. IEEE Transactions on Image Processing (2022).Google ScholarGoogle Scholar
  24. Orest Kupyn, Tetiana Martyniuk, Junru Wu, and Zhangyang Wang. 2019. Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. In Proceedings of the IEEE/CVF international conference on computer vision. 8878--8887.Google ScholarGoogle ScholarCross RefCross Ref
  25. Tao Li, Yingqi Gao, Kai Wang, Song Guo, Hanruo Liu, and Hong Kang. 2019a. Diagnostic assessment of deep learning algorithms for diabetic retinopathy screening. Information Sciences, Vol. 501 (2019), 511--522.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Xiaomeng Li, Xiaowei Hu, Lequan Yu, Lei Zhu, Chi-Wing Fu, and Pheng-Ann Heng. 2019b. CANet: cross-disease attention network for joint diabetic retinopathy and diabetic macular edema grading. IEEE transactions on medical imaging, Vol. 39, 5 (2019), 1483--1493.Google ScholarGoogle Scholar
  27. Ke Liang, Yue Liu, Sihang Zhou, Wenxuan Tu, Yi Wen, Xihong Yang, Xiangjun Dong, and Xinwang Liu. 2023. Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure. IEEE Transactions on Knowledge and Data Engineering (2023).Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Ke Liang, Lingyuan Meng, Meng Liu, Yue Liu, Wenxuan Tu, Siwei Wang, Sihang Zhou, Xinwang Liu, and Fuchun Sun. 2022. Reasoning over Different Types of Knowledge Graphs: Static, Temporal and Multi-Modal. arXiv preprint arXiv:2212.05767 (2022).Google ScholarGoogle Scholar
  29. Ming-Yu Liu, Thomas Breuel, and Jan Kautz. 2017. Unsupervised image-to-image translation networks. Advances in neural information processing systems, Vol. 30 (2017).Google ScholarGoogle Scholar
  30. Long Ma, Tengyu Ma, Risheng Liu, Xin Fan, and Zhongxuan Luo. 2022. Toward fast, flexible, and robust low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5637--5646.Google ScholarGoogle ScholarCross RefCross Ref
  31. Yuhui Ma, Jiang Liu, Yonghuai Liu, Huazhu Fu, Yan Hu, Jun Cheng, Hong Qi, Yufei Wu, Jiong Zhang, and Yitian Zhao. 2021. Structure and illumination constrained GAN for medical image enhancement. IEEE Transactions on Medical Imaging, Vol. 40, 12 (2021), 3955--3967.Google ScholarGoogle ScholarCross RefCross Ref
  32. Taesung Park, Alexei A Efros, Richard Zhang, and Jun-Yan Zhu. 2020. Contrastive learning for unpaired image-to-image translation. In European conference on computer vision. Springer, 319--345.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Prasanna Porwal, Samiksha Pachade, Ravi Kamble, Manesh Kokare, Girish Deshmukh, Vivek Sahasrabuddhe, and Fabrice Meriaudeau. 2018. Indian diabetic retinopathy image dataset (IDRiD): a database for diabetic retinopathy screening research. Data, Vol. 3, 3 (2018), 25.Google ScholarGoogle ScholarCross RefCross Ref
  34. Ziyi Shen, Huazhu Fu, Jianbing Shen, and Ling Shao. 2020. Modeling and enhancing low-quality retinal fundus images. IEEE transactions on medical imaging, Vol. 40, 3 (2020), 996--1006.Google ScholarGoogle Scholar
  35. Saikat Kumar Shome and Siva Ram Krishna Vadali. 2011. Enhancement of diabetic retinopathy imagery using contrast limited adaptive histogram equalization. International Journal of Computer Science and Information Technologies, Vol. 2, 6 (2011), 2694--2699.Google ScholarGoogle Scholar
  36. Zhen Ling Teo, Yih-Chung Tham, Marco Yu, Miao Li Chee, Tyler Hyungtaek Rim, Ning Cheung, Mukharram M Bikbov, Ya Xing Wang, Yating Tang, Yi Lu, et al. 2021. Global prevalence of diabetic retinopathy and projection of burden through 2045: systematic review and meta-analysis. Ophthalmology, Vol. 128, 11 (2021), 1580--1591.Google ScholarGoogle ScholarCross RefCross Ref
  37. Qi-Chong Tian and Laurent D Cohen. 2017. Global and local contrast adaptive enhancement for non-uniform illumination color images. In Proceedings of the IEEE International Conference on Computer Vision Workshops. 3023--3030.Google ScholarGoogle ScholarCross RefCross Ref
  38. Xinhang Wan, Jiyuan Liu, Weixuan Liang, Xinwang Liu, Yi Wen, and En Zhu. 2022. Continual Multi-View Clustering. In Proceedings of the 30th ACM International Conference on Multimedia (Lisboa, Portugal) (MM '22). Association for Computing Machinery, New York, NY, USA, 3676--3684. https://doi.org/10.1145/3503161.3547864Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Xinhang Wan, Xinwang Liu, Jiyuan Liu, Siwei Wang, Yi Wen, Weixuan Liang, En Zhu, Zhe Liu, and Lu Zhou. 2023. Auto-weighted Multi-view Clustering for Large-scale Data. arxiv: 2303.01983 [cs.LG]Google ScholarGoogle Scholar
  40. Renzhen Wang, Benzhi Chen, Deyu Meng, and Lisheng Wang. 2018. Weakly supervised lesion detection from fundus images. IEEE transactions on medical imaging, Vol. 38, 6 (2018), 1501--1512.Google ScholarGoogle Scholar
  41. Xiaofei Wang, Mai Xu, Jicong Zhang, Lai Jiang, Liu Li, Mengxian He, Ningli Wang, Hanruo Liu, and Zulin Wang. 2021. Joint Learning of Multi-Level Tasks for Diabetic Retinopathy Grading on Low-Resolution Fundus Images. IEEE Journal of Biomedical and Health Informatics, Vol. 26, 5 (2021), 2216--2227.Google ScholarGoogle ScholarCross RefCross Ref
  42. Yufei Wang, Renjie Wan, Wenhan Yang, Haoliang Li, Lap-Pui Chau, and Alex Kot. 2022. Low-light image enhancement with normalizing flow. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 2604--2612.Google ScholarGoogle ScholarCross RefCross Ref
  43. Wenhui Wu, Jian Weng, Pingping Zhang, Xu Wang, Wenhan Yang, and Jianmin Jiang. 2022. Uretinex-net: Retinex-based deep unfolding network for low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5901--5910.Google ScholarGoogle ScholarCross RefCross Ref
  44. Xihong Yang, Xiaochang Hu, Sihang Zhou, Xinwang Liu, and En Zhu. 2022. Interpolation-based contrastive learning for few-label semi-supervised learning. IEEE Transactions on Neural Networks and Learning Systems (2022).Google ScholarGoogle ScholarCross RefCross Ref
  45. Xihong Yang, Yue Liu, Sihang Zhou, Siwei Wang, Wenxuan Tu, Qun Zheng, Xinwang Liu, Liming Fang, and En Zhu. 2023. Cluster-guided Contrastive Graph Clustering Network. In Proceedings of the AAAI conference on artificial intelligence, Vol. 37. 10834--10842.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Qijing You, Cheng Wan, Jing Sun, Jianxin Shen, Hui Ye, and Qiuli Yu. 2019. Fundus image enhancement method based on CycleGAN. In 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, 4500--4503.Google ScholarGoogle Scholar
  47. He Zhao, Bingyu Yang, Lvchen Cao, and Huiqi Li. 2019. Data-driven enhancement of blurry retinal images via generative adversarial networks. In Medical Image Computing and Computer Assisted Intervention-MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13-17, 2019, Proceedings, Part I 22. Springer, 75--83.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Mei Zhou, Kai Jin, Shaoze Wang, Juan Ye, and Dahong Qian. 2017. Color retinal image enhancement based on luminosity and contrast adjustment. IEEE Transactions on Biomedical engineering, Vol. 65, 3 (2017), 521--527.Google ScholarGoogle ScholarCross RefCross Ref
  49. Yi Zhou, Boyang Wang, Lei Huang, Shanshan Cui, and Ling Shao. 2020. A benchmark for studying diabetic retinopathy: segmentation, grading, and transferability. IEEE Transactions on Medical Imaging, Vol. 40, 3 (2020), 818--828.Google ScholarGoogle ScholarCross RefCross Ref
  50. Yuqian Zhou, Hanchao Yu, and Humphrey Shi. 2021. Study group learning: Improving retinal vessel segmentation trained with noisy labels. In Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part I 24. Springer, 57--67.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A Reference-free Self-supervised Domain Adaptation Framework for Low-quality Fundus Image Enhancement

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        MM '23: Proceedings of the 31st ACM International Conference on Multimedia
        October 2023
        9913 pages
        ISBN:9798400701085
        DOI:10.1145/3581783

        Copyright © 2023 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 27 October 2023

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

        Acceptance Rates

        Overall Acceptance Rate995of4,171submissions,24%

        Upcoming Conference

        MM '24
        MM '24: The 32nd ACM International Conference on Multimedia
        October 28 - November 1, 2024
        Melbourne , VIC , Australia
      • Article Metrics

        • Downloads (Last 12 months)147
        • Downloads (Last 6 weeks)28

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader