Exploring the complementarity of THz pulse imaging and DCE-MRIs: Toward a unified multi-channel classification and a deep learning framework

https://doi.org/10.1016/j.cmpb.2016.08.026Get rights and content

Highlights

  • •

    The complementarity of THz transient imaging spectrometry and DCE MRI datasets for assessing disease proliferation is explained.

  • •

    Multi-channel signal processing for de-noising, feature extraction and selection, as well as fusion are discussed.

  • •

    Recent advances in capturing textural information for both sensing modalities are placed in context.

  • •

    The general structure of multi-dimensional classifiers using complex extensions of SVM and ELM as well as Clifford algebras are explained.

  • •

    Multi-layer deep-learning architectures and the use of geometric neurons are proposed for the assessment of disease proliferation in the future.

Abstract

We provide a comprehensive account of recent advances in biomedical image analysis and classification from two complementary imaging modalities: terahertz (THz) pulse imaging and dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI). The work aims to highlight underlining commonalities in both data structures so that a common multi-channel data fusion framework can be developed. Signal pre-processing in both datasets is discussed briefly taking into consideration advances in multi-resolution analysis and model based fractional order calculus system identification. Developments in statistical signal processing using principal component and independent component analysis are also considered. These algorithms have been developed independently by the THz-pulse imaging and DCE-MRI communities, and there is scope to place them in a common multi-channel framework to provide better software standardization at the pre-processing de-noising stage. A comprehensive discussion of feature selection strategies is also provided and the importance of preserving textural information is highlighted. Feature extraction and classification methods taking into consideration recent advances in support vector machine (SVM) and extreme learning machine (ELM) classifiers and their complex extensions are presented. An outlook on Clifford algebra classifiers and deep learning techniques suitable to both types of datasets is also provided. The work points toward the direction of developing a new unified multi-channel signal processing framework for biomedical image analysis that will explore synergies from both sensing modalities for inferring disease proliferation.

Introduction

This work aims to establish first the application of THz pulse imaging and DCE-MRI imaging as complementary modalities to identify and assess disease proliferation. Examples from the literature are used to show the unique advantages but also limitations associated with each imaging technique. Second, the work aims to provide an account of recent advances in time series analysis and imaging, for both THz pulse and DCE-MRI imaging modalities, this establishes that both communities are facing similar problems at the pre-processing de-noising stage so that software standardization may be possible through the universal application of some of these algorithms. Once the commonalities in both data structures are established, a unified analysis framework for multi-channel data fusion is proposed. A third aim of the work is to highlight recent progress in both subjects and suggest some future directions for multi-channel machine learning techniques that would lead to a generic framework for the automated quantitative assessment of disease proliferation using both sensing modalities. To clarify these concepts, the paper is structured as follows: Section 2 provides an introduction to THz-transient spectrometry as this is the more well-established modality for building THz imaging systems. This discussion focuses on recent advances of relevance to biomedical applications. The general structure of these datasets is also explained. This section also provides an account of recent advances in MRI, and places both sensing modalities under a common data acquisition and signal processing framework. The following sections take the view that the problem of developing automated classifier solutions for disease proliferation should be seen as the tuning of three different modules that may be individually optimized for particular samples and data sets: 1) the data acquisition imaging module, 2) the data de-noising pre-processing and feature extraction module and 3) the classifier module. Depending on the type of sample, tuning may be tailored for each module to optimize the learning process. The necessity for adopting a multi-channel measurement modality for both dataset types is also discussed. Section 3 discusses generic signal de-noising methodologies applicable to both systems using linear transforms and windowing, apodization, parametric model fitting, and multiresolution feature extraction methods with wavelets or adaptive wavelets. The above discussions are focused on robust feature extraction and selection from a single pixel perspective. A discussion of feature selection strategies relevant to both modalities is also provided and a multi-channel data fusion methodology is proposed for integrating the complementary information from the two imaging modalities. In addition, artefact removal in multi-channel intensity based segmentation and image reconstruction is also considered as such errors can significantly compromise classifier performance. This section also provides a discussion of recent advancements in texture information retrieval; this is an emergent topic in the computer science literature. A multi-channel feature selection approach that incorporates textural information is then proposed as the preferred way to preserve information before this is presented to a classifier.

Section 4 discusses recent advances in different classifier methodologies, with an emphasis in complex support vector machine (CSVM) and extreme learning machine (ELM) approaches. These classifiers may be adopted for binary or multiclass classification tasks. The methodology may be naturally extended to multi-pixel or voxel images. Section 5 provides an outlook of multi-channel classifiers incorporating multiple features in their input space. Such approaches are also suitable for classifying tensorial datasets. The discussion focuses on Clifford algebra based feature classification. The multi-channel approach also enables the fusion of information acquired from multiple images at different time stamps, thus potentially elucidating disease proliferation. Section 6 discusses recent advances in deep learning as related to MRI and THz imaging datasets. This is currently most relevant to MRI in clinical practice, but can benefit the THz-pulse imaging community, especially if such systems are to undergo clinical trials. Section 7 provides concluding remarks related to both systems. A general discussion that draws attention to different aspects of the unified framework proposed is provided in Section 7. Concluding remarks focusing on the important findings of the work are then provided in Section 8.

Section snippets

Introduction to THz pulse imaging

Investigations at the terahertz (THz) part of the electromagnetic (EM) spectrum, loosely defined between 100 GHz and 10 THz, are of much relevance to the biological sciences, because THz radiation interacts strongly with polar molecules [1], [2], [3], [4]. Biological tissue is generally composed of polar liquids so discrimination between tissue types can be made on the basis of water content. THz spectroscopic studies also provide complementary information on low-frequency bond vibrations,

Data windowing, model fitting parametric approaches and statistical techniques

In the case of THz pulse imaging datasets, pre-processing aims to improve measurement precision of the complex insertion loss function from the effects of amplitude and phase noise. This noise originates from pulse-to-pulse variation, THz emitter output instability, temperature dependent laser beam pointing stability, limitations in translation stage movement uniformity and detector shot noise. When this noise is not Gaussian, it cannot be removed through repeated measurements, and artefacts(a

Support vector machine classifiers (SVMs)

Kernel based learning and support vector machine (SVM) methodologies reside at the core of a range of interdisciplinary challenges. Their formulation shares concepts from different disciplines such as: linear algebra, mathematics, statistics, signal processing, systems and control theory, optimization, machine learning, pattern recognition, data mining and neural networks. The idea of the SVM is to map data from the input space into a high-dimensional feature space, in which an optimal

Outlook for Clifford algebra based feature classification

The necessity for a multi-channel framework that captures a multitude of features extracted using the signal processing routines discussed earlier leads to a need for developing alternative classifiers capable of accommodating a large number of input vectors. Multi-dimensional classifiers do not require a dimensionality reduction of the datasets, thus preserving the information presented at the input stage of the classifier which could otherwise be lost in a conventional dataset fusion

An outlook for applying deep learning AI architectures in MRI and THz-pulse imaging datasets

Within the context of approximation theory, any given continuous function g(x) representing MRI or THz-pulse imaging datasets can be seen as a superposition of weighted functions:y(x)=∑j=1Nωjσj(wjTx+θj)where σ(.) is a continuous discriminatory function like a sigmoid, ωj ∈ R and x,θj​,w∈ℜn. The finite sums of the form of Eq. (26) are dense if |g(x)−y(x)|<ε for a given ε > 0 and all x∈[0,1]n.The above expression represents a generalization of the well-known density theorem which is

General discussion

This work considers commonalities and differences in THz pulse imaging and DCE-MRI datasets and proposes a unified signal processing framework. THz pulse imaging is currently being explored as a viable alternative imaging modality to assess disease proliferation in a non-invasive manner. DCE-MRI is well established and as such it is regularly used in clinical environments. In contrast, THz-pulse imaging has yet to gain popularity although there is a general recognition of its potential to

Conclusion

After reviewing the relevant literature of THz pulse imaging and the different MRI modalities as applied to biomedical applications, commonalities in both data structures are identified so that a multi-channel data acquisition and signal processing framework suitable to both measurement techniques could be established. By contrasting the progress in each field, further directions for efficient de-noising and parsimonious feature extraction could be established. The two communities have been

Acknowledgement

We would like to thank Dr. J. Bowen from Reading and Dr. B. Fischer and Prof. D. Abbott from Adelaide for providing us with the THz spectrometric datasets presented in this study and Prof. R.K.H. Galvão from ITA, Brazil for his valuable discussions on the formulation of multiresolution algorithms.

References (264)

  • T. Chen et al.

    Identification of biomolecules by terahertz spectroscopy and fuzzy pattern recognition

    Spectrochim. Acta A. Mol. Biomol. Spectrosc

    (2013)
  • A. Menikh et al.

    Label-free amplified bioaffinity detection using terahertz wave technology

    Biosens. Bioelectron

    (2004)
  • M. Walther et al.

    Far-infrared vibrational spectra of all-trans, 9-cis and 13-cis retinal measured by THz time-domain spectroscopy

    Chem. Phys. Lett

    (2000)
  • I. Jones et al.

    Towards T-ray spectroscopy of retinal isomers: a review of methods and modelling

    Vib. Spectrosc

    (2006)
  • D. Martin et al.

    Polarised interferometric spectrometry for the millimetre and submillimetre spectrum

    Infrared Phys

    (1970)
  • X.-X. Yin et al.

    Wavelet based local tomographic image using terahertz techniques

    Digital Signal Process

    (2009)
  • M. Bergamino et al.

    A review of technical aspects of T1-weighted dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) in human brain tumors

    Phys. Med

    (2014)
  • C.F. Beckmann et al.

    Tensorial extensions of independent component analysis for multisubject FMRI analysis

    Neuroimage

    (2005)
  • M. Neeman et al.

    Pulsed-gradient spin-echo studies in NMR imaging. effects of the imaging gradients on the determination of diffusion coefficients

    J. Magn. Reson

    (1990)
  • G. Cleveland et al.

    Nuclear magnetic resonance measurement of skeletal muscle: anisotropy of the diffusion coefficient of the intracellular water

    Biophys. J.

    (1976)
  • J. Tanner

    Self diffusion of water in frog muscle

    Circ. Res

    (1979)
  • P. Basser et al.

    Estimation of the effective self-diffusion tensor from the NMR spin echo

    J. Magn. Reson. B

    (1994)
  • P. Basser et al.

    Estimation of the effective self-diffusion tensor from the NMR spin echo

    Biophys. J.

    (1994)
  • J. Mattiello et al.

    Analytical expression for the b matrix in NMR diffusion imaging and spectroscopy

    J. Magn. Reson. A

    (1994)
  • M. Scheller et al.

    Analyzing sub-100 nm samples with transmission terahertz time domain spectroscopy

    Opt. Commun

    (2009)
  • G.K. Aguirre et al.

    Perfusion fMRI for functional neuroimaging

    Int. Rev. Neurobiol. Neuroimag. Part A

    (2005)
  • E. Özarslan et al.

    MR diffusion-“diffraction” phenomenon in multi-pulse-field-gradient experiments

    J. Magn. Reson

    (2007)
  • E. Pickwell et al.

    Biomedical applications of terahertz technology

    J. Phys. D Appl. Phys

    (2006)
  • D. Mittleman et al.

    Recent advances in terahertz imaging

    Appl. Phys. B Laser. Opt

    (1999)
  • R.M. Woodward et al.

    Terahertz pulse imaging in reflection geometry of human skin cancer and skin tissue

    J. Invest. Dermatol

    (2002)
  • S. Smye et al.

    The interaction between terahertz radiation and biological tissue

    Phys. Med. Biol

    (2001)
  • B. Fischer et al.

    Chemical recognition in terahertz time-domain spectroscopy and imaging, semiconductor science and technology

    Semicond. Sci. Technol

    (2005)
  • P. Siegel

    Terahertz technology in biology and medicine

    IEEE Trans. Microwave Theory Techniq

    (2004)
  • S. Hadjiloucas et al.

    Measurements of leaf water content using terahertz radiation

    IEEE Trans. Microwave Theory Techniq

    (1999)
  • S. Hadjiloucas et al.

    Precision of quasioptical null-balanced bridge techniques for transmission and reflection coefficient measurements

    Rev. Sci. Instr

    (1999)
  • V.P. Wallace et al.

    Terahertz pulsed imaging of basal cell carcinoma ex vivo and in vivo

    J. Invest. Dermatol

    (2004)
  • C.J. Strachan et al.

    Terahertz pulsed spectroscopy and imaging in the pharmaceutical setting—a review

    J. Pharm. Pharmacol

    (2005)
  • C. Rønne et al.

    THz spectroscopy of liquid H2O and D2O

    Phys. Rev. Lett

    (1999)
  • O. Sushko et al.

    Sub-terahertz spectroscopy reveals that proteins influence the properties of water at greater distances than previously detected

    J. Chem. Phys

    (2015)
  • O. Sushko et al.

    Terahertz spectral domain computational analysis of hydration shell of proteins with increasingly complex tertiary structure

    J. Phys. Chem. B

    (2013)
  • D.M. Mittleman et al.

    T-Ray imaging

    IEEE J. Select. Topics Quant. Electron

    (1996)
  • M.K. Choi et al.

    Broadband 10–300 GHz stimulus-response sensing for chemical and biological entities

    Phys. Med. Biol

    (2002)
  • P. Martel et al.

    Vibrational modes of hemoglobin in red blood cells

    Biophys. J.

    (1991)
  • P. Siegel

    Terahertz technology

    IEEE Trans. Microwave Theory Techniq

    (2002)
  • M. Herrmann et al.

    Display modes in time-resolved terahertz imaging

    Japanese J. Appl. Phys. Part 1 Regular Papers Short Notes Rev. Papers

    (2000)
  • B. Fischer et al.

    Terahertz time-domain spectroscopy and imaging of artificial RNA

    Opt. Express

    (2005)
  • M. Brucherseifer et al.

    Label-free probing of the bindingstate of DNA by time-domain terahertz sensing

    Appl. Phys. Lett

    (2000)
  • A. Mazhorova et al.

    Label-free bacteria detection using evanescent mode of a suspended core terahertz fiber

    Opt. Express

    (2012)
  • A. Menikh et al.

    Terahertz biosensing technology: frontiers and progress

    Chemphyschem

    (2002)
  • H. Frölich

    The biological effects of microwaves and related questions

    Adv. Electronics Electron Phys

    (1980)

    • Cited by (16)

    • MRI radiogenomics for intelligent diagnosis of breast tumors and accurate prediction of neoadjuvant chemotherapy responses-a review

      2022, Computer Methods and Programs in Biomedicine
      Citation Excerpt :

      By decomposing high-dimensional images into simple geometric elements such as points, lines, areas, and volumes, and combining geometric algebra to analyze the characteristics of multidimensional consistency of spatiotemporal data, a multi-dimensional, multi-scale deep learning classification system can be designed [87–89]. As discussed in [90], a scalar product can be conveniently extended to Clifford or geometric product to account for errors due to a patient's movement, to extract non-linear features from images, or to account for observed scaling changes across the dimensions of a tumour, as illustrated in Fig. 5. Time domain terahertz spectroscopy and MRI which are complementary sensing modalities may be used as part of a diagnostics protocol and in that case time-space unified deep learning schemes that combines all these heterogeneous datasets needs to be adopted.

    • Terahertz Data Extraction and Analysis Based on Deep Learning Techniques for Emerging Applications

      2024, IEEE Access

    • Digital Filtering Techniques Using Fuzzy-Rules Based Logic Control

      2023, Journal of Imaging

    • Nondestructive Evaluation of Thermal Barrier Coatings’ Porosity Based on Terahertz Multi-Feature Fusion and a Machine Learning Approach

      2023, Applied Sciences (Switzerland)

    • Pretreatment to terahertz absorption curves by narrow undulation constraint and Its quick implementation suggested by convex hull

      2022, Scientific Reports

    • Artificial neural networks for material parameter extraction in terahertz time-domain spectroscopy

      2022, Optics Express
    View all citing articles on Scopus

    This work was supported in part by the National Natural Science Foundation of China (No. 61332013 and No. 81371526).

    View full text
    © 2016 Elsevier Ireland Ltd. All rights reserved.