Digital Heritage Reconstruction Using Deep Learning-Based Super-Resolution

Abstract

Heritage sites and archival monuments have a great cultural significance. However, they suffer degradation due to several reasons. As a result, in order to preserve the cultural heritage, one has seen increased interest in research on digitally restoring the photographs of vandalized monuments. One may think of recreating the historical monuments by super-resolving the heritage images, an algorithmic approach to increase the spatial resolution of an image. This chapter presents a single image super-resolution (SR) method based on deep learning to obtain higher resolution photographs of the digitally reconstructed monuments. The resulting images can serve as the input to walkthrough systems. Given a low spatial resolution test image and a database consisting of low and high spatial resolution (LR-HR) images, we obtain super-resolution for the test image. We use the idea proposed in Dong et al. (Computer Vision–ECCV 2014, Springer, 2014, [5]) to represent the mapping between LR and HR images by using a deep convolutional neural network (CNN). CNN filters are learned by standard backpropagation and stochastic gradient descent method. The novelty of our approach lies in the elimination of interpolation during the training phase. Our method directly learns the end-to-end mapping between LR and HR images. The advantage of our approach is that once the network is trained for a magnification factor of 2, the learned parameters can be used to obtain SR for higher magnification factors also. We demonstrate the effectiveness of the proposed approach by conducting experiments using the images of heritage monuments as well as natural scene. Our results are compared with the standard interpolation technique and existing learning-based approaches. Visual and quantitative comparisons confirm the effectiveness of the proposed method.