Exploiting prior knowledge in compressed sensing to design robust systems for endoscopy image recovery
Introduction
Telemedicine has become the most promoted form of medicine, especially during the COVID-19, which can avoid cross infection and balance the medical resource. Reports with a huge amount of medical data and images need to be saved and then transmitted through computers and the internet, which brings great challenges on the storage space and transmission efficiency [18], [27]. Compressed sensing (CS) is a cost-effective technique that can reconstruct data from lower dimensional measurements to original ones [9], [12], [20], [32], [47].The measurements can be obtained as:where is the original data, is the measurement vector, and is named sensing matrix with . Infinite solutions can be obtained by solving the underdetermined problem Eq. (1). A possible way to obtain a unique solution is to include the constraint of sparsity on [42], [46], which can be written as a form of sparse representation:in which the dictionary consists of its atoms and is the sparse vector with , where counts the number of non-zero elements and is the so called sparsity. represents the th element in the vector . It is assumed that the signal is compressible in .
A generic CS system considers both Eqs. (1) and (2) which results in the following measurement:where the matrix is called equivalent dictionary. The estimated data can be obtained by computing , in which the estimated sparse vector can be obtained by solving the following problem:The traditional methods to solve this problem can be structured into three categories. Firstly, greedy algorithms, such as Orthogonal Matching Pursuit (OMP) [38], Normalized Iterative Hard Thresholding (NIHT) [7]. The second category is composed of convex algorithms to solve the problem with the -norm replacing the -norm, for instance, the Basis Pursuit Denoising (BPDN) [8] and the Least absolute Shrinkage and Selection Operator (LASSO) [43]. The third kind of methods includes sparse Bayesian learning [44], and smoothed -norm minimization [35]. Recently, some knowledge-aided based methods [22], [24], [26], [36], [40] are developed to further improve the system performance by extending those traditional methods. Prior knowledge in [24], [40] is extracted from the sparse coefficient to reflect the probability distribution of its non-zero elements. The work in [36] describes how to obtain the prior knowledge from the similar signal which has been reconstructed beforehand. All methods utilize the similarity of the successive signals.
In order to guarantee the successful recovery of via (4), the equivalent dictionary can be designed according to a measure called mutual coherence [13], [15], which is defined as:where is the transpose operator. Mutual coherence measures the worst-case coherence between any two atoms of which will influence the range of sparsity in an exact recovery process [13], which can be concluded as:Eq. (6) indicates that a smaller mutual coherence will allow a larger range of . This motivates us to design such that is as small as possible. In other words, if the dictionary is given, the sensing matrix can be optimized to obtain a minimum .
In general, the optimization for sensing matrix design [4], [14], [15], [31] can be expressed as:where denotes the Frobenius norm, is the Gram of equivalent dictionary , and is a target Gram with certain desired properties. Observing the Gram matrix , if the dictionary is normalized, the largest off-diagonal elements in is the mutual coherence to be minimized. As introduced in [15], the inequation of Eq. (6) indicates that the sensing matrix is designed such that is as small as possible, which allows a larger range of to guarantee the exact recovery. While this conclusion is true from the view of a worst-case. However, it turns out that the mutual coherence does not justice to the actual behavior of sparse representations. Thus, if the restriction is relaxed, the average mutual coherence should be considered which describes the true behavior. The average mutual coherence is defined by:where are the off-diagonal elements for the matrix which is the Gram of the column-normalized equivalent dictionary , is a set as with , noting that the author in [15] gives the parameter . counts the number of elements in set .
In some early work, the target Gram adopted for sensing matrix design is the identity matrix with dimension [2], [14], [31], [45], which approaches the off-diagonal elements in Gram to zero to minimize the average mutual coherence. The works in [23], [25] design the sensing matrix in such a way that the off-diagonal elements in the Gram of a normalized equivalent dictionary approaches a relaxed equiangular tight frame (ETF) target Gram [6], [31]. This kind of target Gram satisfies the relaxed ETF structure as with . In such strategies, the mutual coherence of is smaller than those using the target Gram of the identity matrix, but it is sensitive to the noise [5], [29]. Recently, the work in [28] has exploited prior knowledge to extend the above problem that improves the recovery results further. The prior knowledge is about the magnitude of the previous recovery sparse vector, which means different vectors create different types of prior knowledge. Hence, many sensing matrices have to been redesigned because vectors representing the data are different from each other, increasing the computational burden in both compressive stage and recovery stage. In our work, the prior knowledge is extracted from statistical knowledge of the sparse matrix and then it is adopted to design one sensing matrix for data and image recovery, which brings research significance in strategy of telemedicine.
In addition, the framework in [28] just considers the design performance under the condition that the signal is absolutely sparse, which is hard to apply in practical scenarios. The sparse representation error (SRE) [29] is defined as:which cannot be ignored in speech, images and video [10], [11], [17], [41]. The block based sensing matrix (BCS) is designed in [10], [17] which can improve the processing efficiency of images. Among the real applications, the endoscopy images have many interference due to the complex gastrointestinal environment, hence compressed sensing can be adopted as a robust algorithm to deal with the compression and recovery for endoscopy images [34], [39].
In our work, prior knowledge is extracted by computing the ratio of non-zero elements in each row of a sparse matrix, which provides the information to design both the sensing matrix and recovery algorithm. In the stage of sensing matrix design, this kind knowledge aims to reduce the average mutual coherence for the columns of equivalent dictionary with high utilization ratio. Prior knowledge is arranged into a diagonal matrix, which is denoted as a weighted matrix. To build more robust systems, the identity matrix is selected as the target Gram against the sparse representation error and the influence of SRE to CS systems is also considered. Hence, an analytical solution of sensing matrix can be obtained by solving a weighted cost function, which minimizes the difference between the identity matrix and the Gram of the equivalent dictionary, and minimizing the projection of the sparse representation error simultaneously. This designed sensing matrix is robust to systems against the sparse representation error which will exist in the real application scenarios, such as speech, images, video. In the stage of recovery, prior knowledge can help locate the nonzero elements more accurately. A novel recovery algorithm named Probability Driven Normalized Iterative Hard Thresholding (PD-NIHT) is designed based on the NIHT algorithm. The CS system in which the sensing matrix and recovery algorithm are optimized with prior knowledge can achieve a better recovery performance than the state-of-the-art algorithms.
The main contributions of this work can be summarized as follows:
- •
Prior knowledge is extracted from the previous recovery sparse matrix by counting the proportion of non-zero elements in each row. This knowledge will be employed both in sensing matrix design and recovery algorithm.
- •
An analytical solution is derived for the sensing matrix design through solving a cost function with the assistance of prior knowledge. The proposed sensing matrix design algorithm is robust, which is named Knowledge-aided Sensing Matrix Design considering SRE (KASMD-SRE). A NIHT-based recovery algorithm named Probability Driven NIHT (PD-NIHT) is also developed.
- •
Compared with some existing CS systems, the proposed CS system within the sensing matrix of KASMD-SRE algorithm and the PD-NIHT recovery algorithm can achieve the best performance on the experiments of synthetic data and different organs of endoscopy images, which provide the technical support for telemedicine.
The rest of the paper is arranged as follows. The existing algorithms for sensing matrix design and recovery are introduced in Section 2, some necessary background for the proposed algorithms are also presented in Section 2. Section 3 details the proposed approach for knowledge-aided sensing matrix design with consideration of SRE. In Section 4, the proposed PD-NIHT algorithm is detailed. In addition, the computational complexity for the proposed and existing algorithms are analyzed. Simulations for synthetic data and an application on different organs of endoscopy images are carried out in Section 5 to support the theoretical argument. Section 6 draws the conclusions.
Section snippets
Preliminaries
This section reviews existing approaches for the design of sensing matrices and reconstruction algorithms.
Proposed KASMD-SRE sensing matrix design
In this section, prior knowledge based method KASMD-SRE is proposed to design a sensing matrix assuming that the dictionary is well trained through the samples. The sparse representation error (SRE) computed from the samples under this dictionary is also considered in the design procedure.
A matrix of weights is set by the prior knowledge given by Eq. (19). is a diagonal matrix with its th diagonal element expressed as:where is a positive scalar with .
Design of recovery algorithm with prior knowledge
In this section, the proposed recovery algorithm named PD-NIHT is detailed in the Section 4.1. The Section 4.2 summarizes the proposed CS system and analyzes the computational complexity of above mentioned sensing matrices and recovery algorithms.
Simulations
The simulations of synthetic data and application of endoscopy images are carried out in this section. In Section 5.1, the process of generating synthetic data with prior knowledge are detailed. The evaluation indices for algorithm performance are also introduced. Simulations for the proposed sensing matrix compared with existing algorithms are carried out in Section 5.2. The proposed recovery algorithm compared with existing algorithms are executed in Section 5.3. Section 5.4 presents the
Conclusion
This paper has studied the problem of designing a CS system in which the sensing matrix and the recovery algorithm are jointly designed with the aid of prior knowledge. We have considered the problem of sensing matrix design with prior information, where the measures of average mutual coherence of the equivalent dictionary as well as the sparse representation error are taken into account. A robust sensing matrix design has been obtained as an analytical solution by solving a constrained
CRediT authorship contribution statement
Qianru Jiang: Conceptualization, Methodology, Software, Formal analysis, Writing – original draft, Writing – review & editing. Sheng Li: Conceptualization, Methodology, Validation, Writing – review & editing, Project administration, Funding acquisition. Liping Chang: Software, Investigation. Xiongxiong He: Validation, Supervision, Project administration, Funding acquisition. Rodrigo C. de Lamare: Conceptualization, Writing – review & editing, Funding acquisition.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was partially supported by National Natural Science Foundation of PR China (Grant:61873239 and 61675183). Zhejiang Provincial Key Research and Development Program of PR China (Grant:2020C03074). Zhejiang Provincial Natural Science Foundation of PR China (Grant:LY18F010023). CNPq Foundation. FAPERJ Foundation.
References (48)
- et al.
A gradient-based alternating minimization approach for optimization of the measurement matrix in compressive sensing
Signal Process.
(2012) - et al.
Compressed sensing for real measurements of quaternion signals
J. Franklin Inst.
(2017) - et al.
Effective zero-norm minimization algorithms for noisy compressed sensing
J. Franklin Inst.
(2020) - et al.
An efficient method for robust projection matrix design
Signal Process.
(2018) - et al.
Projection matrix design using prior information in compressive sensing
Signal Process.
(2017) - et al.
Nonconvex nonsmooth low-rank minimization for generalized image compressed sensing via group sparse representation
J. Franklin Inst.
(2020) - The Kvasir Dataset[Online]: https://datasets.simula.no//kvasir/, [Accessed]: Jul. 23,...
- et al.
K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation
IEEE Trans. Signal Process.
(2006) - et al.
Alternating optimization of sensing matrix and sparsifying dictionary for compressed sensing
IEEE Trans. Signal Process.
(2015) - et al.
Sensing matrix optimization based on equiangular tight frames with consideration of sparse representation error
IEEE Trans. Multimedia
(2016)
Normalized iterative hard thresholding: guaranteed stability and performance
IEEE J. Sel. Top. Signal Process.
Decoding by linear programming
IEEE Trans. Inf. Theory
An introduction to compressive sampling
IEEE Signal Process. Mag.
Compressed-sensing recovery of images and video using multihypothesis predictions
2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR)
Robust image coding based upon compressive sensing
IEEE Trans. Multimedia
Compressed sensing
IEEE Trans. Inf. Theory
Optimally sparse representation in general (nonorthogonal) dictionaries via minimization
Proc. Nat. Acad. Sci.
Learning to sense sparse signals: simultaneous sensing matrix and sparsifying dictionary optimization
IEEE Trans. Image Process.
Optimized projections for compressed sensing
IEEE Trans. Signal Process.
Method of optimal directions for frame design
IEEE Int. Conf. Acoust., Speech, Signal Process.
Compressive sensing image restoration using adaptive curvelet thresholding and nonlocal sparse regularization
IEEE Trans. Image Process.
Recent developments in the sparse fourier transform: a compressed fourier transform for big data
IEEE Signal Process. Mag.
Performance analysis of joint-sparse recovery from multiple measurement vectors via convex optimization: which prior information is better?
IEEE Access
Cited by (2)
Design of Robust Sensing Matrix for UAV Images Encryption and Compression
2023, Applied Sciences (Switzerland)