Accurate MR image super-resolution via lightweight lateral inhibition network

Highlights

• A high-precision and fast lightweight CNN model is presented for efficient MR image SR tasks.

• An inhibition unit is devised to impose explicit inhibitory regulation on intermediate features.

• A shallow feature fusion strategy is proposed to provide more effective evidence for SR inference.

Abstract

In recent years, convolutional neural networks (CNNs) have shown their advantages on MR image super-resolution (SR) tasks. Many current SR models, however, have heavy demands on computation and memory, which are not friendly to magnetic resonance imaging (MRI) where computing resource is usually constrained. On the other hand, a basic consideration in most MRI experiments is how to reduce scanning time to improve patient comfort and reduce motion artifacts. In this work, we ease the problem by presenting an effective and lightweight model that supports fast training and accurate SR inference. The proposed network is inspired by the lateral inhibition mechanism, which assumes that there exist inhibitory effects between adjacent neurons. The backbone of our network consists of several lateral inhibition blocks, where the inhibitory effect is explicitly implemented by a battery of cascaded local inhibition units. When model scale is small, explicitly inhibiting feature activations is expected to further explore model representational capacity. For more effective feature extraction, several parallel dilated convolutions are also used to extract shallow features directly from the input image. Extensive experiments on typical MR images demonstrate that our lateral inhibition network (LIN) achieves better SR performance than other lightweight models with similar model scale.

Introduction

Magnetic resonance imaging (MRI) is a commonly-used and versatile non-invasive imaging modality with the advantages of multi-contrast and no ionizing radiation etc. Spatial resolution is one of the most important imaging parameters in most MRI experiments. In general, high-resolution (HR) images usually provide rich structural details and benefit more accurate image postprocessing, hence promoting effective subsequent analysis and early clinic diagnosis (Greenspan et al., 2001, Greenspan et al., 2002, Reeth et al., 2012, Shi et al., 2019). However, the spatial resolution of magnetic resonance (MR) images are typically constrained by various physical and physiological limitations, e.g., hardware device, imaging time, signal-to-noise ratio (SNR), and motion artifacts etc. (Reeth et al., 2012, Plenge et al., 2012). Increasing the spatial resolution of MR images typically reduces SNR, and increase imaging time and thus patient discomfort, indicating that these imaging parameters are highly interdependent to each other (Plenge et al., 2012).

Image super-resolution (SR) provides an effective alternative to enhance the resolution of MR images from the perspective of postprocessing (Zhao et al., 2018c), which aims at recovering a HR image from one or more low-resolution (LR) images. As a postprocessing method, image SR is an active research field that can substantially break through the limitations of hardware device and improve image resolution (Park et al., 2003, Plenge et al., 2012). In recent years, deep learning techniques (LeCun et al., 2015), especially convolutional neural networks (CNNs) (LeCun et al., 1989), have greatly promoted the development of this field, resulting in the emergence of many advanced SR methods, such as SRCNN (Dong et al., 2016a), DRCN (Kim et al., 2016b), DRRN (Tai et al., 2017a), MemNet (Tai et al., 2017b), VDSR (Kim et al., 2016a), EDSR/MDSR (Lim et al.), RDN (Zhang et al., 2018b), RCAN (Zhang et al., 2018a) and CSN (Zhao et al., 2018c) etc. Although these models have excellent performance, most of them are mainly aimed at the SR tasks on natural images, instead of MR images.

In medical image processing community, there are also some deep CNN-based medical image SR methods, e.g., Pham et al., Chen et al., 2018b, Chen et al., 2018a, Zhao et al., 2018a, Zhao et al., 2018c etc. The primary intention of these methods, to some extent, is to improve the performance of MR image SR tasks. However, a fundamental consideration in many MRI experiments is how to reduce imaging time to improve patient comfort and avoid motion artifacts as much as possible. Therefore, high-efficiency HR image reconstruction is also of significance in practical applications. On the other hand, an important problem in medical image processing with deep learning techniques is the degradation of training samples (Litjens et al., 2017, Zhao et al., 2018c). As the model scale (e.g., model parameters, network depth/width etc.) increases, it will be more difficult to train larger models with these degraded medical training samples (Zhao et al., 2018c, Zhao et al., 2019), and more tricks are needed for successful model training (Li et al., 2018). In this regard, lightweight models may be more appropriate for practical applications of medical image SR tasks.

With these considerations, we aim at efficient MR image SR reconstruction by introducing a lightweight CNN model in this paper. The proposed model, which we term as lateral inhibition network (LIN), is well-motivated and inspired by the biological lateral inhibition mechanism that assumes there exists explicit inhibitory regulation between adjacent neurons. The building module of our LIN network is local inhibition unit (LIU) that takes residual block (Lim et al., Zhang et al., 2018b, Zhang et al., 2018a) as the backbone and a inhibition tail (IT) is attached to integrate lateral inhibition mechanism into feature mapping. A series of cascaded LIUs construct a local nonlinear mapping block, i.e., lateral inhibition block (LIB), as shown in Fig. 2. Then multiple LIBs are stacked together to build the nonlinear subnet of the proposed LIN model. Besides, to extract shallow features with different receptive field sizes, we use a group of 3 × 3 dilated conv layers with different dilation rates in the feature extraction subnet, as shown in Fig. 1. Like Lim et al., Zhang et al., 2018b, Zhao et al., 2018c, we only apply one 3 × 3 conv layer to reconstruct the final output.

Deep CNN models are generally built upon the convolution operation that extracts informative local features by integrating spatial and channel information together within local receptive fields. In fact, a lot of work demonstrates that careful structural design helps to improve the representational capacity of deep models substantially (Simonyan and Zisserman, 2014, He et al., 2016a, He et al., 2016b, Kim et al., 2016a, Huang et al., 2017, Hu et al., 2017, Zhao et al., 2018b). The proposed LIU follows this point and serves as a feature regulator that simulates the Hartline–Ratliff equation and explicitly adjusts the hierarchical features of deep models. The explicit adjustment of the hierarchical features is considered beneficial to alleviate the representational burden of deep models and therefore improve SR performance (Hu et al., 2017, Zhao et al., 2018c). The main contributions of this work are as following:

• A lightweight CNN model, LIN, is proposed for efficient and accurate MR image SR tasks. With moderate model parameters and computational overhead, our LIN achieves high-precision and fast SR reconstruction.

• Motivated by the lateral inhibition mechanism, we design a local inhibition unit (LIU) to explicitly impose inhibitory regulation on feature maps, alleviating the representational burden of the model.

• We propose to integrate the shallow features with different receptive field sizes to boost model performance. Through this strategy, we can increase the diversity of the extracted features and provide more effective evidence for nonlinear inference and image reconstruction.

• We experimentally and analytically verify that combining the lateral inhibition mechanism with our shallow feature extraction strategy favors to improving the performance of deep models.

Extensive experiments on various MR images show that our model achieves competitive SR performance with much less model parameters and higher efficiency. The remainder of this paper is organized as follows: in Section 2, we introduce some previous work related to this work. The proposed LIN model is illustrated in Section 3. The experimental results and analyses are given in Section 4, and the conclusion is in Section 5.