Ensemble Image Super-Resolution CNNs for Small Data and Diverse Compressive Models

Abstract

Convolutional neural networks (CNN) have become some of the most powerful tools for image reconstruction problems thanks to the availability of very large data sets. Implementations of deep residual structures, adversarial generation networks and attention mechanisms have made great accomplishment. However, the good performance from complex and deep network architecture is not guaranteed when the training data set is small and not well preventative for the entire population. There are many real-world image reconstruction tasks where large and diverse training data is unavailable, such as problems in the physical sciences and engineering for which the data set generation process is complicated and large data sets are expensive to construct. For example, herein we discuss the application of deep-learning to challenging problems in material science. Inspired by compressive sensing and ensemble learning, we propose a method using ensemble image super-resolution CNNs in transform domains to overcome the challenges of small training data in image reconstruction problems. Ensemble methods provide a more robust approach when CNNs are trained with less representative data. Transform domains could support the CNNs with multiple sparse representations of the original image data which enrich the information so that the CNNs can be sufficiently trained even using small data sets. Particularly, we report here a successful application of CNN ensembles to the reconstruction of areal density maps of carbon nano-tube sheet materials. We show that applying the ensemble CNNs in transform domains can reveal finer details in the material texture and help to improve the quality control capabilities for carbon nano-tube sheet production with only a small collection of training data.