Residual Wavelon Convolutional Networks for Characterization of Disease Response on MRI

Abstract

Wavelets have shown significant promise for medical image decomposition and artifact pre-processing by representing inputs via shifted and scaled components of a specified mother wavelet function. However, wavelets could also be leveraged within deep neural networks as activation functions for neurons (called wavelons) in the hidden layer. Integrating wavelons into a convolutional neural network architecture (termed a “wavelon network” (WN)) offers additional flexibility and stability during optimization, but the resulting model complexity has caused it to be limited to low-dimensional applications. Towards addressing these issues, we present the Residual Wavelon Convolutional Network (RWCN), a novel integrated WN architecture that employs weighted skip connections (to enable residual learning) together with image convolutions and wavelet activation functions to more efficiently capture high-dimensional disease response-specific patterns from medical imaging data. In addition to developing the analytical basis for wavelet activation functions as used in this work, we implemented RWCNs by adapting the popular VGG and ResNet architectures. Evaluation was conducted within three different challenging clinical problems: (a) predicting pathologic complete response (pCR) to neoadjuvant chemoradiation via 153 pre-treatment T2-weighted (T2w) MRI scans in rectal cancers, (b) evaluating pCR after chemoradiation via 100 post-treatment T2w MRIs in rectal cancers, as well as (c) risk stratifying patients who will or will not require surgery after aggressive medication in Crohn’s disease using 73 baseline MRI scans. In comparison to 4 state-of-the-art alternative models (VGG-16, VGG-19, ResNet-18, ResNet-50), RWCN architectures yielded significantly improved and more efficient classifier performance on unseen data in multi-institutional validation cohorts (hold-out accuracies of 0.82, 0.85, and 0.88, respectively).