Chengfang Zhang
ABSTRACT
In contrast to ”synthetic” models such as convolutional sparse representation(CSR)/convolutional dictionary learning(CDL), convolutional analytic operator learning(CAOL) based on convergent block proximal extrapolation gradient with majorizer(BPEG-M), which uses unsupervised learning to train automatically encoded CNNs to compensate for large memory requirements of patch-based learning and to solve block multinonconvex problems more accurately, is the latest optimization framework for solving block multinonconvex problems. A learning model named CAOL is introduced into medical image fusion to overcome patch-based defects in this paper. In our work, the high-frequency component is fused by using the BPEG-M approach, while the low-pass component uses the ”choose-max” strategy. In experiments, we used 5 types of medical brain images and 4 popular fusion methods to verify the effectiveness of the proposed method. The experimental results show that our approach is successful in terms of qualitative and quantitative analysis.
- [1] Deepika, T., and G. K. Kannan, Gambhir D, ”A Novel adaptive optimization of Dual-Tree Complex Wavelet Transform for Medical Image Fusion”, arXiv,2020.Google Scholar
- [2] KUNAL KISHORE PANDEY, SHEKHAR R SURALKAR, ”Medical image fusion using curvelet transform.” International Journal of Electronics Communication Engineering Technology.,2013.Google Scholar
- [3] Z. Zhu, M. Zheng, G. Qi, D. Wang and Y. Xiang, ”A Phase Congruency and Local Laplacian Energy Based Multi-Modality Medical Image Fusion Method in NSCT Domain,” IEEE Access7, 20811–20824, 2019.Google ScholarCross Ref
- [4] Huang C, Tian G, Lan Y, Peng Y, Ng EYK, Hao Y, Cheng Y and Che W, ”A New Pulse Coupled Neural Network (PCNN) for Brain Medical Image Fusion Empowered by Shuffled Frog Leaping Algorithm,” Front. Neurosci., 2019.Google Scholar
- [5] Ming Yin, Xiaoning Liu, Yu Liu, Xun Chen, ”Medical Image Fusion With Parameter-Adaptive Pulse Coupled Neural Network in Nonsubsampled Shearlet Transform Domain”, IEEE Transactions on Instrumentation and Measurement, 68(1),49-64, 2019.Google ScholarCross Ref
- [6] Maqsood, Sarmad and Javed, Umer, ”Multi-modal Medical Image Fusion based on Two-scale Image Decomposition and Sparse Representation”, Biomedical Signal Processing and Control, vol. 57, 2020.Google Scholar
- [7] Liu,Yu and Liu,Shuping and Wang,Zengfu, ”A general framework for image fusion based on multi-scale transform and sparse representation,” Information Fusion,vol. 24, pp. 147-164, 2015.Google ScholarDigital Library
- [8] Yin, Haitao and Li, Shutao, ”Multimodal image fusion with joint sparsity model,” Optical Engineering, vol. 50, 2011.Google Scholar
- [9] Yang, Bin and Li, Shutao, ”Visual attention guided image fusion with sparse representation,” Optik - International Journal for Light and Electron Optics, vol. 125, pp.4881-4888,2014.Google ScholarCross Ref
- [10] Gao, Zhisheng and Zhang, Chengfang, ”Texture clear multi-modal image fusion with joint sparsity model,” Optik - International Journal for Light and Electron Optics, vol. 121305, pp.S0030402616310725,2016.Google Scholar
- [11] ZHANG C F,Yi L Z, ”Multimodal image fusion with adaptive joint sparsity model,” J. Electron. Imaging, vol. 28, pp.013043,2019.Google Scholar
- [12] Chengfang Zhang, Ziliang Feng, Zhisheng Gao, Xin Jin, Dan Yan, Liangzhong Yi, ”Salient feature multimodal image fusion with a joint sparse model and multiscale dictionary learning,” Opt. Eng, vol. 59, pp.051402,2020.Google Scholar
- [13] Zong Jingjing,Qiu Tianshuang, ”Brain Image Fusion Based on Online Dictionary Learning and Pulse Coupled Neural Network,” Chinese Journal of Biomedical Engineering,34(5): 540-547,2015.Google Scholar
- [14] B. Wohlberg, ”Efficient algorithms for convolutional sparse representations,” IEEE Trans. Image Process,25, pp.301–315,2016.Google ScholarDigital Library
- [15] Garcia-Cardona, Cristina and Wohlberg, Brendt, ”Convolutional Dictionary Learning: A Comparative Review and New Algorithms,” IEEE Transactions on Computational Imaging,vol. 4,no.3, pp.366–381,2018.Google ScholarCross Ref
- [16] Il and Yong and Chun and Jeffrey and A and Fessler,”Convolutional Dictionary Learning: Acceleration and Convergence”,IEEE Transactions on Image Processing,VOL. 27, NO. 4, pp.1697-1712, 2018.Google ScholarDigital Library
- [17] Chun, Il Yong and Fessler, Jeffrey A,”Convergent convolutional dictionary learning using Adaptive Contrast Enhancement (CDL-ACE): Application of CDL to image denoising”,2017 International Conference on Sampling Theory and Applications (SampTA),2017.Google Scholar
- [18] Y. Liu, X. Chen, and R. K. Ward, ”Image fusion with convolutional sparse representation,”IEEE Signal Process. Lett.vol. 23, pp.1882–1886,2016.Google ScholarCross Ref
- [19] Liu and Yu and Chen and Xun and Ward and Rabab and K. and Wang and Z. and Jane,”Medical Image Fusion via Convolutional Sparsity Based Morphological Component Analysis,”IEEE Signal Processing Letters,26(3),485-489,2019.Google Scholar
- [20] Jingming Xia, Yi Lu, and Ling Tan,”Research of Multimodal Medical Image Fusion Based on Parameter-Adaptive Pulse-Coupled Neural Network and Convolutional Sparse Representation,”1-13,2020.Google Scholar
- [21] www.med.harvard.edu/aanlib/home.html.Google Scholar
- [22] F. Heide, W. Heidrich, and G. Wetzstein, ”Fast and flexible convolutional sparse coding,” in Proc. IEEE CVPR, Boston, MA, Jun. 2015, pp.5135–5143.Google ScholarCross Ref
- [23] M. D. Zeiler, D. Krishnan, G. W. Taylor, and R. Fergus, ”Deconvolutional networks,”in Proc. IEEE CVPR, San Francisco, CA, Jun. 2010,pp. 2528–2535.Google ScholarCross Ref
- [24] Z. Liu et al., ”Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative study,” IEEE Trans. Pattern Anal. Machine Intell.34(1), 94–109 (2011). pp.5135–5143.Google ScholarDigital Library
- [25] Xydeas, C. S., and P. V. V.. ”Objective image fusion performance measure.” Military Technical Courier 56.4(2000):181-193.Google Scholar
- [26] Piella, Gemma, and H. Heijmans. ”A new quality metric for image fusion.” International Conference on Image Processing IEEE, 2003.Google Scholar
- [27] Zhao, Jiying, R. Laganiere, and Z. Liu. ”Performance assessment of combinative pixel-level image fusion based on an absolute feature measurement.” International Journal of Innovative Computing Information Control Ijicic 3.6(2006).Google Scholar
- [28] https://sites.google.com/site/yuliu316316Google Scholar
- [29] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, ”Distributed optimization and statistical learning via the alternating direction method of multipliers,”Foundations and Trends in Machine Learning, vol. 3,no. 1, pp. 1C122, 2010.Google ScholarDigital Library
- [30] Il Yong Chun and Jeffrey A. Fessler, ”Convolutional analysis operator learning: Acceleration and convergence,”IEEE Trans. Image Process., 29(1): 2108-2122, 2020.Google ScholarCross Ref
Recommendations
Visible and infrared Image Fusion via Convolution Analysis Operator
ICITEE '21: Proceedings of the 4th International Conference on Information Technologies and Electrical EngineeringAs a synthetic model, conventional convolutional sparse representation (CSR)/convolutional dictionary learning (CDL) suffers from model mismatch. This deficiency leads to loss of details and smoothing in infrared-visible fusion results with CDL/CSR ...
Convolutional analysis operator learning for multifocus image fusion
AbstractSparse representation (SR), convolutional sparse representation (CSR) and convolutional dictionary learning (CDL) are synthetic-based priors that have proven to be successful in signal inverse problems (such as multifocus image fusion)...
Highlights- A new multi-focus image fusion framework CAOL-based is proposed.
- The different ...
Deep learning methods for medical image fusion: A review
AbstractThe image fusion methods based on deep learning has become a research hotspot in the field of computer vision in recent years. This paper reviews these methods from five aspects: Firstly, the principle and advantages of image fusion methods based ...
Graphical abstractDisplay Omitted
Highlights- Summarizing various deep learning models of medical image fusion.
- Discussing the applications of deep learning in medical image fusion field.
- Summarizing medical image fusion methods based on deep learning and datasets used in ...
Comments