Exploiting prior knowledge in compressed sensing to design robust systems for endoscopy image recovery

https://doi.org/10.1016/j.jfranklin.2022.02.005Get rights and content

Abstract

In this work, we investigate compressed sensing (CS) techniques based on the exploitation of prior knowledge to support telemedicine. In particular, prior knowledge is obtained by computing the probability of appearance of non-zero elements in each row of a sparse matrix, which is then employed in sensing matrix design and recovery algorithms for CS systems. A robust sensing matrix is designed by jointly reducing the average mutual coherence and the projection of the sparse representation error. A Probability-Driven Normalized Iterative Hard Thresholding (PD-NIHT) algorithm is developed as the recovery method, which also exploits the prior knowledge of the probability of appearance of non-zero elements and can bring performance benefits. Simulations for synthetic data and different organs of endoscopy image are carried out, where the proposed sensing matrix and PD-NIHT algorithm achieve a better performance than previously reported algorithms.

Introduction

Telemedicine has become the most promoted form of medicine, especially during the COVID-19, which can avoid cross infection and balance the medical resource. Reports with a huge amount of medical data and images need to be saved and then transmitted through computers and the internet, which brings great challenges on the storage space and transmission efficiency [18], [27]. Compressed sensing (CS) is a cost-effective technique that can reconstruct data from lower dimensional measurements to original ones [9], [12], [20], [32], [47].The measurements can be obtained as:y=Φx,where xN×1 is the original data, yM×1 is the measurement vector, and ΦM×N is named sensing matrix with MN. Infinite solutions can be obtained by solving the underdetermined problem Eq. (1). A possible way to obtain a unique solution is to include the constraint of sparsity on x [42], [46], which can be written as a form of sparse representation:x=k=1Kα(k)Ψ(:,k)=Ψα,in which the dictionary ΨN×K consists of its atoms {Ψ(:,k)}k=1K and α is the sparse vector with α0S, where ·0 counts the number of non-zero elements and S is the so called sparsity. α(k) represents the kth element in the vector α. It is assumed that the signal x is compressible in Ψ.

A generic CS system considers both Eqs. (1) and (2) which results in the following measurement:y=ΦΨα=Dα,where the matrix DM×K is called equivalent dictionary. The estimated data x^ can be obtained by computing x^=Ψα^, in which the estimated sparse vector α^ can be obtained by solving the following problem:α^=argminαyDα22s.t.α0S.The traditional methods to solve this problem can be structured into three categories. Firstly, greedy algorithms, such as Orthogonal Matching Pursuit (OMP) [38], Normalized Iterative Hard Thresholding (NIHT) [7]. The second category is composed of convex algorithms to solve the problem with the 1-norm replacing the 0-norm, for instance, the Basis Pursuit Denoising (BPDN) [8] and the Least absolute Shrinkage and Selection Operator (LASSO) [43]. The third kind of methods includes sparse Bayesian learning [44], and smoothed 0-norm minimization [35]. Recently, some knowledge-aided based methods [22], [24], [26], [36], [40] are developed to further improve the system performance by extending those traditional methods. Prior knowledge in [24], [40] is extracted from the sparse coefficient to reflect the probability distribution of its non-zero elements. The work in [36] describes how to obtain the prior knowledge from the similar signal which has been reconstructed beforehand. All methods utilize the similarity of the successive signals.

In order to guarantee the successful recovery of α^ via (4), the equivalent dictionary can be designed according to a measure called mutual coherence [13], [15], which is defined as:μ(D)max1ijK|(D(:,i))TD(:,j)|D(:,i)2D(:,j)2,where T is the transpose operator. Mutual coherence measures the worst-case coherence between any two atoms of D which will influence the range of sparsity S in an exact recovery process [13], which can be concluded as:S<12[1+1μ(D)].Eq. (6) indicates that a smaller mutual coherence will allow a larger range of S. This motivates us to design D such that μ(D) is as small as possible. In other words, if the dictionary Ψ is given, the sensing matrix Φ can be optimized to obtain a minimum μ(D).

In general, the optimization for sensing matrix design [4], [14], [15], [31] can be expressed as:argminΦ,GtGtGF2,where ·F denotes the Frobenius norm, G=DTD is the Gram of equivalent dictionary D, and Gt is a target Gram with certain desired properties. Observing the Gram matrix G, if the dictionary D is normalized, the largest off-diagonal elements in G is the mutual coherence to be minimized. As introduced in [15], the inequation of Eq. (6) indicates that the sensing matrix is designed such that μ(D) is as small as possible, which allows a larger range of S to guarantee the exact recovery. While this conclusion is true from the view of a worst-case. However, it turns out that the mutual coherence does not justice to the actual behavior of sparse representations. Thus, if the restriction is relaxed, the average mutual coherence should be considered which describes the true behavior. The average mutual coherence is defined by:μav(D)(i,j)Sav|G¯(i,j)|Nav,where G¯(i,j) are the off-diagonal elements for the matrix G¯ which is the Gram of the column-normalized equivalent dictionary D, Sav is a set as Sav={(i,j):μ¯|G¯(i,j)|<1,ij} with 0μ¯<1, noting that the author in [15] gives the parameter μ¯=KMM(K1). Nav counts the number of elements in set Sav.

In some early work, the target Gram adopted for sensing matrix design is the identity matrix IK with dimension K [2], [14], [31], [45], which approaches the off-diagonal elements in Gram G to zero to minimize the average mutual coherence. The works in [23], [25] design the sensing matrix in such a way that the off-diagonal elements in the Gram of a normalized equivalent dictionary approaches a relaxed equiangular tight frame (ETF) target Gram [6], [31]. This kind of target Gram satisfies the relaxed ETF structure as {GtK×K:Gt=GtT,Gt(k,k)=1,maxij|Gt(i,j)|μ} with KMM(K1)μ1. In such strategies, the mutual coherence of D is smaller than those using the target Gram of the identity matrix, but it is sensitive to the noise [5], [29]. Recently, the work in [28] has exploited prior knowledge to extend the above problem that improves the recovery results further. The prior knowledge is about the magnitude of the previous recovery sparse vector, which means different vectors create different types of prior knowledge. Hence, many sensing matrices have to been redesigned because vectors representing the data are different from each other, increasing the computational burden in both compressive stage and recovery stage. In our work, the prior knowledge is extracted from statistical knowledge of the sparse matrix and then it is adopted to design one sensing matrix for data and image recovery, which brings research significance in strategy of telemedicine.

In addition, the framework in [28] just considers the design performance under the condition that the signal x is absolutely sparse, which is hard to apply in practical scenarios. The sparse representation error (SRE) [29] is defined as:e=xΨαwhich cannot be ignored in speech, images and video [10], [11], [17], [41]. The block based sensing matrix (BCS) is designed in [10], [17] which can improve the processing efficiency of images. Among the real applications, the endoscopy images have many interference due to the complex gastrointestinal environment, hence compressed sensing can be adopted as a robust algorithm to deal with the compression and recovery for endoscopy images [34], [39].

In our work, prior knowledge is extracted by computing the ratio of non-zero elements in each row of a sparse matrix, which provides the information to design both the sensing matrix and recovery algorithm. In the stage of sensing matrix design, this kind knowledge aims to reduce the average mutual coherence for the columns of equivalent dictionary with high utilization ratio. Prior knowledge is arranged into a diagonal matrix, which is denoted as a weighted matrix. To build more robust systems, the identity matrix is selected as the target Gram against the sparse representation error and the influence of SRE to CS systems is also considered. Hence, an analytical solution of sensing matrix can be obtained by solving a weighted cost function, which minimizes the difference between the identity matrix and the Gram of the equivalent dictionary, and minimizing the projection of the sparse representation error simultaneously. This designed sensing matrix is robust to systems against the sparse representation error which will exist in the real application scenarios, such as speech, images, video. In the stage of recovery, prior knowledge can help locate the nonzero elements more accurately. A novel recovery algorithm named Probability Driven Normalized Iterative Hard Thresholding (PD-NIHT) is designed based on the NIHT algorithm. The CS system in which the sensing matrix and recovery algorithm are optimized with prior knowledge can achieve a better recovery performance than the state-of-the-art algorithms.

The main contributions of this work can be summarized as follows:

  • Prior knowledge is extracted from the previous recovery sparse matrix by counting the proportion of non-zero elements in each row. This knowledge will be employed both in sensing matrix design and recovery algorithm.

  • An analytical solution is derived for the sensing matrix design through solving a cost function with the assistance of prior knowledge. The proposed sensing matrix design algorithm is robust, which is named Knowledge-aided Sensing Matrix Design considering SRE (KASMD-SRE). A NIHT-based recovery algorithm named Probability Driven NIHT (PD-NIHT) is also developed.

  • Compared with some existing CS systems, the proposed CS system within the sensing matrix of KASMD-SRE algorithm and the PD-NIHT recovery algorithm can achieve the best performance on the experiments of synthetic data and different organs of endoscopy images, which provide the technical support for telemedicine.

The rest of the paper is arranged as follows. The existing algorithms for sensing matrix design and recovery are introduced in Section 2, some necessary background for the proposed algorithms are also presented in Section 2. Section 3 details the proposed approach for knowledge-aided sensing matrix design with consideration of SRE. In Section 4, the proposed PD-NIHT algorithm is detailed. In addition, the computational complexity for the proposed and existing algorithms are analyzed. Simulations for synthetic data and an application on different organs of endoscopy images are carried out in Section 5 to support the theoretical argument. Section 6 draws the conclusions.

Section snippets

Preliminaries

This section reviews existing approaches for the design of sensing matrices and reconstruction algorithms.

Proposed KASMD-SRE sensing matrix design

In this section, prior knowledge based method KASMD-SRE is proposed to design a sensing matrix assuming that the dictionary is well trained through the samples. The sparse representation error (SRE) computed from the samples under this dictionary is also considered in the design procedure.

A matrix of weights ΣWK×K is set by the prior knowledge ξK×1 given by Eq. (19). ΣW is a diagonal matrix with its kth diagonal element expressed as:ΣW(k,k)=τ+(1τ)ξ(k),where τ is a positive scalar with τ1.

Design of recovery algorithm with prior knowledge

In this section, the proposed recovery algorithm named PD-NIHT is detailed in the Section 4.1. The Section 4.2 summarizes the proposed CS system and analyzes the computational complexity of above mentioned sensing matrices and recovery algorithms.

Simulations

The simulations of synthetic data and application of endoscopy images are carried out in this section. In Section 5.1, the process of generating synthetic data with prior knowledge are detailed. The evaluation indices for algorithm performance are also introduced. Simulations for the proposed sensing matrix compared with existing algorithms are carried out in Section 5.2. The proposed recovery algorithm compared with existing algorithms are executed in Section 5.3. Section 5.4 presents the

Conclusion

This paper has studied the problem of designing a CS system in which the sensing matrix and the recovery algorithm are jointly designed with the aid of prior knowledge. We have considered the problem of sensing matrix design with prior information, where the measures of average mutual coherence of the equivalent dictionary as well as the sparse representation error are taken into account. A robust sensing matrix design has been obtained as an analytical solution by solving a constrained

CRediT authorship contribution statement

Qianru Jiang: Conceptualization, Methodology, Software, Formal analysis, Writing – original draft, Writing – review & editing. Sheng Li: Conceptualization, Methodology, Validation, Writing – review & editing, Project administration, Funding acquisition. Liping Chang: Software, Investigation. Xiongxiong He: Validation, Supervision, Project administration, Funding acquisition. Rodrigo C. de Lamare: Conceptualization, Writing – review & editing, Funding acquisition.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was partially supported by National Natural Science Foundation of PR China (Grant:61873239 and 61675183). Zhejiang Provincial Key Research and Development Program of PR China (Grant:2020C03074). Zhejiang Provincial Natural Science Foundation of PR China (Grant:LY18F010023). CNPq Foundation. FAPERJ Foundation.

References (48)

  • T. Blumensath et al.

    Normalized iterative hard thresholding: guaranteed stability and performance

    IEEE J. Sel. Top. Signal Process.

    (2010)
  • E.J. Candes et al.

    Decoding by linear programming

    IEEE Trans. Inf. Theory

    (2005)
  • E.J. Candès et al.

    An introduction to compressive sampling

    IEEE Signal Process. Mag.

    (2008)
  • C. Chen et al.

    Compressed-sensing recovery of images and video using multihypothesis predictions

    2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR)

    (2011)
  • C. Deng et al.

    Robust image coding based upon compressive sensing

    IEEE Trans. Multimedia

    (2012)
  • D.L. Donoho

    Compressed sensing

    IEEE Trans. Inf. Theory

    (2006)
  • D.L. Donoho et al.

    Optimally sparse representation in general (nonorthogonal) dictionaries via 1 minimization

    Proc. Nat. Acad. Sci.

    (2003)
  • J.M. Duarte-Carvajalino et al.

    Learning to sense sparse signals: simultaneous sensing matrix and sparsifying dictionary optimization

    IEEE Trans. Image Process.

    (2009)
  • M. Elad

    Optimized projections for compressed sensing

    IEEE Trans. Signal Process.

    (2007)
  • K. Engan et al.

    Method of optimal directions for frame design

    IEEE Int. Conf. Acoust., Speech, Signal Process.

    (1999)
  • N. Eslahi et al.

    Compressive sensing image restoration using adaptive curvelet thresholding and nonlocal sparse regularization

    IEEE Trans. Image Process.

    (2016)
  • A.C. Gilbert et al.

    Recent developments in the sparse fourier transform: a compressed fourier transform for big data

    IEEE Signal Process. Mag.

    (2014)
  • G.H. Golub, C.F. Van Loan, Matrix computations (3rd ed.) 20(5–6) (1996)...
  • S. Hu et al.

    Performance analysis of joint-sparse recovery from multiple measurement vectors via convex optimization: which prior information is better?

    IEEE Access

    (2018)
  • View full text