Abstract
Diffusion-weighted and Spectroscopic MR images are found to be very helpful for diagnostic purposes as they provide complementary information to that provided by conventional MRI. These images are also acquired at a faster rate, but with low signal-to-noise ratio. This limitation can be overcome by applying image super-resolution techniques. In this paper, we propose a single-image super-resolution (SISR) technique via sparse representation for diffusion-weighted (DW) and spectroscopic MR (MRS) images. It is based on non-local total variation approach to regularize an ill-posed inverse problem of SISR. Experiments are conducted for both DW and MRS test images and the results are compared with other recent regularization-based methods using sparse representation. The comparison also validates the potential of the proposed method for clinical applications.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Diffusion-weighted imaging (DWI) and magnetic resonance spectroscopy imaging (MRSI) are important techniques for brain imaging. DWI is a specific MR imaging method based on mapping of diffusion process of water molecules in tissues [1]. It is an effective technique, which provides functional information of the brain tissues. DWI is a faster acquisition technique as compared to conventional MRI and it does not require any contrast agent as well. Most of the obtained images are acquired using high speed protocols with a low spatial resolution, reducing the patient stress but also producing low quality images [1]. On the other hand, MRSI is a non-invasive technique which gives information about the biochemical components within the tissues [7]. MRSI is particularly of great help in early diagnosis of brain lesions on the basis of the spectra obtained from different metabolite concentrations. MRSI also has the same advantage of fast scan time as DWI. In spite of advantages of DWI and MRSI, the rise in the cost of scans and poor signal-to-noise ratio limit their clinical use. These limits can be overcome by image super-resolution (SR) methods [2]. SR methods are categorized as either single-image SR (SISR) or multiple-image SR. SISR is more preferable in medical imaging as multiple images are difficult to be acquired in a particular cross-section, taking considerable scan time. Sparse representation approach has proved to be very effective in case of single-image SR. But the stabilization for the solution of inverse problems is a major issue in sparsity approach, which can be overcome by regularization. In this paper, we focus to develop a SISR method for MR images using sparse representation along with non-local total variation (NLTV) regularization.
The remaining part of the paper is written in following sections: In Sect. 2, a brief literature about of the regularization based SR methods is provided. Section 3 will describe the proposed methodology. Experimental results on different datasets of diffusion-weighted and spectroscopic MR images are explained in Sect. 4. Lastly, Sect. 5 gives a conclusion about the paper.
2 Background
Owing to the resolution limitations, there is a need to create high resolution images in a short acquisition time using post-processing techniques. Even though, basic interpolation methods are available, but these methods introduce blurring artifacts. Taking the disadvantages of the interpolation based techniques into consideration, SR had been developed, which has shown very influential and efficient results in HR image reconstruction. The SR methods tries to perform mapping between the LR and HR images via learning some prior information regarding similarity of the two images with their own image space [9]. Earlier works apply SR algorithm on medical imaging to generate HR images using multiple LR images with different angular views, but because of the insufficient number of low resolution images and unknown blurring operators, it becomes highly ill-posed in nature.
For solving the ill-posed problem of image SR, many recent regularization techniques are developed to get a more stabilized solution [6, 8]. Recently, sparse representation based SR reconstruction came into picture as a powerful tool for image restoration [4, 5]. Sparse representation is mainly concerned with the solution of inverse problems. In medical image processing, total variation (TV) regularization methods are successfully used as it shows excellent edge preservations. Despite its high effectiveness, problems like over-smoothing of image textures and odd artifacts may limit such methods. Lately, the non local means (NLM) based regularization approaches are also introduced for solving inverse problems like, image restoration, super-resolution and denoising, etc.,
3 Proposed Method
The performances of sparse representation algorithms for image SR are related to several phenomenons, like, quality of the dictionary trained, effectiveness of the constraint term selected for regularization, etc. The proposed method for SR reconstruction from a set of LR MR images is discussed in the following subsections. It consist of two parts: first, learning of LR and HR dictionaries and second, reconstruction of HR output image utilizing the learned dictionaries. The reconstruction algorithm is again can be divided into the following sub-tasks: first, extraction of high-frequency features of the patches, then solving a sparse prior based regularization and secondly, a non-local total variation regularization to restore the textural details remove the undesirable staircase artifact to recover the fine details and textures. Finally, a global image regularization is done that helps in incorporating the given LR image’s point spread function into the reconstructed HR images by utilizing the image acquisition model constraint.
3.1 Sparsity Based Image Super-Resolution
In the beginning, overlapping patches of size \(k \times k\) are extracted from the input LR image Y. The sparse coefficients \(\alpha \) corresponding to each low-resolution patch y is found with respect to the trained dictionaries \(D_{l}\) and \(D_{h}\). Next, these sparse coefficients are combined with high-resolution dictionary \(D_{h}\) to find high-resolution patches x.
The solution of the following sparse regularization problem gives the sparse coefficients corresponding to each low-resolution patch y:
where \({{ D^{'}}} = \left[ \begin{gathered} \,\,{{D}}_\ell \\ T{{D}}_h \\ \end{gathered} \right] \), \({{\tilde{y}}} = \left[ \begin{gathered} {{y}} \\ w \\ \end{gathered} \right] \), and \(\lambda \) is the regularization parameter. T is the overlap region extraction operator which finds the region which is common to both the presently reconstructing patch and the latest HR patch generated; w represents the overlapped pixels contained in the previously reconstructed HR image.
Following the computation of sparse coefficients \(\alpha ^{*}\) using Eq. 1, HR patches x are obtained by solving the following relation which supports the fact that the HR and LR dictionaries shares the same sparse representation.
Arranging all the individual HR patches reconstructed by Eq. 2 on a single grid will yield to an intermediate HR image \(X_{0}\). Before finding the initial HR reconstructed image \(X_{0}\) by the minimization problem, non local total variation regularization is performed so that the patches to be reconstructed fit properly in the above minimization formulation. Again due to measurement errors, \(X_{0}\) may not fit the generalized model, \(Y=WX\), where Y is the input LR image, X is the desired HR image and W is the image sampling operator. For overcoming these limitations due to noise, a global reconstruction constraint is imposed by solving a minimization problem:
The above Eq. 4 is the gradient descent method, which is minimized iteratively to find the final reconstructed image \(X_{0}\).
3.2 NLTV Regularization
The non-local means (NLM) filtering implies the weighted average of the surrounding pixels within a search window for the computation of the new filtered pixel value. Two blocks of the HR image \(X_{0}\) having central pixels at \(x_{i}\) and \(x_{j}\) contributes to weight \(w_{ij}\) which is the gaussian distance \(l_{2}\) between the blocks [12]. Consider \(x_{i}\) and \(x_{j}\) denote the pixel at the center of \(b_s \times b_s\) blocks and it is assumed that \(x_{j}\) lies in the search window of \(x_{i}\). Weight \(w_{ij}\) is computed by:
where f and \(c_{i}\) are controlling parameter and normalization factors, respectively. The new filtered pixel value is denoted by NLM (\(x_{i}\)), and this approach of filtering has lead to a new approach of regularization, known as nonlocal regularization. Consider all the pixels in the center organized as a column vector, represented as r and all the weights are also organized as column vector, represented as w. Mathematically, this nonlocal regularization can be represented as:
These weights are updated iteratively and before the implementation of gradient descent method for final reconstruction, the nonlocal total variation (NLTV) regularization is implemented, the formulation of this approach is as:
The solution of the above formulation is the basis of the NLTV regularization. The HR image \(X_{0}\) obtained after the regularization is used in Eq. 4, which undergoes minimization iteratively to obtain the final SR image \(X^{*}\).
3.3 Dictionary Learning
Two training image patch pairs, set of high-resolution patches is represented by \(X^{h}={x_{1},x_{2},....,x_{k}}\) and set of low-resolution patches is represented by \(Y^{l}={y_{1},y_{2},....,y_{k}}\). These two dictionaries are jointly trained with the condition that both HR and LR image patches have a common sparse representations among them. A joint sparse representation regularization can be formulated involving the LR and HR image patches simultaneously. Mathematically,
where LR and HR patches in vector form have dimensions S and R respectively. \(||Z||_{1}\) is a \({\ell _1}\)-norm term that enforces sparsity into both the dictionaries. Equation 7 is solved iteratively for three parameters simultaneously to obtain the HR and LR dictionaries \(D_{h}\) and \(D_{l}\).
4 Experiments and Evaluations
Simulations of the proposed work is carried out using MATLAB (R2015b) environment on PC having configurations as follows: OS- Windows 7, Processor: Intel core i5 (2.2 GHz), and RAM: 8Â GB. The diffusion-weighted MRI data has been acquired from a GE HDx 1.5Â T with the following parameters: TR/TE: 4225/76.6 ms; Slice thickness: 5Â mm, spacing between scans: 5Â mm; Field of view (FOV): \(100\times 100\); Flip angle: \(90^{0}\). The spectroscopic MRI images have also been acquired from a GE HDx 1.5Â T with the following parameters: TR/TE: 150/1.372 ms; Slice thickness: 8Â mm; spacing between scans: 5Â mm; Field of view (FOV): \(100\times 100\); Flip angle: \(70^{0}\).
4.1 Simulations
First, the LR dictionary \(D_{l}\) and the HR dictionary \(D_{h}\) are trained jointly where both consist of 512 atoms in each. For training, a number of 1,00,000 LR/HR patch pairs are selected from about 30 standard MR images. The regularization parameter for the dictionary has been considered as \(\lambda =0.15\). This dictionary has been trained as per the approach proposed by Yang et al. [11].
Next, for the super-resolution reconstruction, two upscale factors have been considered, i.e., 2 and 3. For both the upscale factors, size of the LR input is \(128\times 128\). Size of the output HR image is \(256\times 256\) and \(384\times 384\) for upscale factor 2 and 3 respectively. The results of the proposed method and some other SR based methods has been given with their magnified view. In Tables 1 and 2, DWI results represent the SR results of diffusion-weighted MRI images and MRSI results represent the SR results of Spectroscopic MRI images.
4.2 Evaluations
The simulation results obtained are evaluated both visually and quantitatively. In Figs. 1, 2, 3, 4, 5, 6, 7 and 8, it is clearly seen that the fine details such as edges have been preserved efficiently in the proposed method. From Tables 1 and 2, it can be seen that evaluation parameters obtained for the proposed method are better in terms of peak signal-to-noise ratio (PSNR) as well as mean structural similarity index (MSSIM) compared to the traditional bicubic interpolation techniques. The proposed method has also shown better results as compared to ScSR proposed by Yang et al. [11] and LRTV proposed in [10]. Compared to the ground truth, bi-cubic interpolation, and LRTV methods produce blurry results. As far as ScSR is concerned, it does not produce blurring artifacts, but in comparison to the proposed method, it does not preserve equivalent edge details. We have compared the image quality using two more metrics mean-square error (MSE) and mutual information (MI) [3] between the ground truth and the SR result image. For better image quality, MSE should be less and MI should be more. All the images used in the experiments are brain images. The magnified views of the results clearly show the zoomed view of a specified portion of the results. It can be seen that fine details are recovered by the proposed method.
5 Conclusion
In this paper, we have shown that the implementation of non-local TV regularization for solving the regularization issues of the sparsity based approach can be a viable solution to the issues. This combination provides better consistency of patches, thereby giving better results. Quantitative comparisons show that the proposed method outperforms the existing regularization based approaches. Proposed method is computationally expensive due to the iterative process of regularization. As a future work, this can be extended to multi-core processing for computationally efficient results.
References
Altay, C., Balci, P.: The efficiency of diffusion weighted MRI and MR spectroscopy on breast MR imaging. J. Breast Health 10(4), 197–200 (2014)
Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1167–1183 (2002)
Ceccarelli, M., di Bisceglie, M., Galdi, C., Giangregorio, G., Ullo, S.L.: Image registration using non-linear diffusion. In: IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2008 (2008)
Datta, S., Deka, B., Mullah, H.U., Kumar, S.: An efficient interpolated compressed sensing method for highly correlated 2D multi-slice MRI. In: 2016 International Conference on Accessibility to Digital World (ICADW), pp. 187–192, December 2016
Deka, B., Datta, S.: Compressed Sensing Magnetic Resonance Image Reconstruction Algorithms: A Convex Optimization Approach. Springer Series on Bio- and Neurosystems. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-3597-6
Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: 12th International Conference on Computer Vision, pp. 349–356. IEEE, Japan (2009)
Jain, S., et al.: Patch based super-resolution of MR spectroscopic images. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI) (2016)
Mullah, H.U., Deka, B.: A fast satellite image super-resolution technique using multicore processing. In: Abraham, A., Muhuri, P.K., Muda, A.K., Gandhi, N. (eds.) HIS 2017. AISC, vol. 734, pp. 51–60. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-76351-4_6
Park, S., Park, M., Kang, M.: Super-resolution image reconstruction: a technical overview. IEEE Sig. Process. Mag. 20, 21–36 (2003)
Shi, F., Cheng, J., Wang, L., Yap, P., Shen, D.: LRTV: MR image super-resolution with low-rank and total variation regularizations. IEEE Trans. Med. Imaging 34, 2459–2466 (2015)
Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)
Zhang, J., Liu, S., Xiong, R., Ma, S., Zhao, D.: Improved total variation based image compressive sensing recovery by nonlocal regularization. In: IEEE International Symposium on Circuits and Systems, ISCAS 2013, Beijing, China (2013)
Acknowledgements
Authors would like to thank All India Council for Technical Education (AICTE) for providing funds under the project (File No. 8-15/RIFD/RPS/POLICY-1/2016-17) and Ministry of Electronics and Information Technology (MeiTY), GoI for providing financial support under the Visvesvaraya Ph.D. Scheme for Electronics & IT (Ph.D./MLA/ 04(41)/2015-16/01) which helped in smooth conduction of the above research work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Deka, B., Mullah, H.U., Datta, S., Lakshmi, V., Ganesan, R. (2019). Sparse Representation Based Super-Resolution of MRI Images with Non-Local Total Variation Regularization. In: Deka, B., Maji, P., Mitra, S., Bhattacharyya, D., Bora, P., Pal, S. (eds) Pattern Recognition and Machine Intelligence. PReMI 2019. Lecture Notes in Computer Science(), vol 11942. Springer, Cham. https://doi.org/10.1007/978-3-030-34872-4_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-34872-4_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34871-7
Online ISBN: 978-3-030-34872-4
eBook Packages: Computer ScienceComputer Science (R0)