Shaolei Zhang
ABSTRACT
The Spectral super-resolution methods based on deep learning have achieved promising performance when the network models are well trained, most of which require sufficient training dataset. However, capturing enough paired color image and hyperspectral images is expensive or complex under existing conditions. Thus, the available training dataset is still small to some extent, which is prone to over-fitting as the network becomes deeper. To solve this problem, a novel method based on deep network is proposed for spectral super-resolution, which is named as multi-scale recursive residual attention network. Without increasing network parameters, the recursive learning can increase the depth and receptive field of network to strengthen nonlinear modeling capabilities. In addition, multi-scale residual attention block is proposed to exploit contextual information in different scale and adjust channel-wise features adaptively. Experiments on public hyperspectral datasets demonstrates that our method can achieve better reconstruction effects compared with other spectral reconstruction methods when there is small training dataset.
- Arad, B., Ben-Shahar, O.: Sparse recovery of hyperspectral signal from natural RGB images. In: European Conference on Computer Vision 2016, pp. 19-34. SpringerGoogle Scholar
Cross Ref
- Fu, Y., Zheng, Y., Zhang, L., Huang, H.: Spectral reflectance recovery from a single rgb image. IEEE Transactions on Computational Imaging 4(3), 382-394 (2018).Google ScholarCross Ref
- Nguyen, R.M., Prasad, D.K., Brown, M.S.: Training-based spectral reconstruction from a single RGB image. In: European Conference on Computer Vision 2014, pp. 186-201. SpringerGoogle ScholarCross Ref
- Galliani, S., Lanaras, C., Marmanis, D., Baltsavias, E., Schindler, K.: Learned spectral super-resolution. arXiv preprint arXiv:1703.09470 (2017).Google Scholar
- Shi, Z., Chen, C., Xiong, Z., Liu, D., Wu, F.: Hscnn+: Advanced cnn-based hyperspectral recovery from rgb images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops 2018, pp. 939-947Google ScholarCross Ref
- Zhang, L., Lang, Z., Wang, P., Wei, W., Liao, S., Shao, L., Zhang, Y.: Pixel-Aware Deep Function-Mixture Network for Spectral Super-Resolution. In: AAAI 2020, pp. 12821-12828Google Scholar
- Li, J., Wu, C., Song, R., Li, Y., Liu, F.: Adaptive weighted attention network with camera spectral sensitivity prior for spectral reconstruction from rgb images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops 2020, pp. 462-463Google ScholarCross Ref
- Zhao, Y., Po, L.-M., Yan, Q., Liu, W., Lin, T.: Hierarchical regression network for spectral reconstruction from rgb images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops 2020, pp. 422-423Google ScholarCross Ref
- Tai, Y., Yang, J., Liu, X.: Image Super-Resolution via Deep Recursive Residual Network. In: IEEE Conference on Computer Vision and Pattern Recognition 2017, pp. 2790-2798Google ScholarCross Ref
- Shocher, A., Cohen, N., Irani, M.: “zero-shot” super-resolution using deep internal learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2018, pp. 3118-3126Google ScholarCross Ref
- Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition 2018, pp. 7132-7141Google ScholarCross Ref
- Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV) 2018, pp. 286-301Google ScholarDigital Library
- Shi, Z., Chen, C., Xiong, Z., Liu, D., Zha, Z.-J., Wu, F.: Deep residual attention network for spectral image super-resolution. In: European Conference on Computer Vision 2018, pp. 214-229. SpringerGoogle Scholar
- Yasuma, F., Mitsunaga, T., Iso, D., Nayar, S.K.: Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum. IEEE transactions on image processing 19(9), 2241-2253 (2010). doi:10.1109/TIP.2010.2046811Google ScholarDigital Library
- Chakrabarti, A., Zickler, T.: Statistics of real-world hyperspectral images. In: CVPR 2011 2011, pp. 193-200. IEEEGoogle ScholarDigital Library
Index Terms
- Deep Multi-scale Recursive Residual Attention Network for Spectral Super Resolution
Recommendations
Spatial-Spectral Deep Residual Network for Hyperspectral Image Super-Resolution
AbstractRecently, single hyperspectral image super-resolution (SR) methods based on deep learning have been extensively studied. However, there has been limited technical development focusing on single hyperspectral image super-resolution due to the high-...
Deep Residual Attention Network for Spectral Image Super-Resolution
Computer Vision – ECCV 2018 WorkshopsAbstractSpectral imaging sensors often suffer from low spatial resolution, as there exists an essential tradeoff between the spectral and spatial resolutions that can be simultaneously achieved, especially when the temporal resolution needs to be ...
Spectral super-resolution meets deep learning: Achievements and challenges
AbstractSpectral super-resolution (sSR) is a very important technique to obtain hyperspectral images from only RGB images, which can effectively overcome the high acquisition cost and low spatial resolution of hyperspectral imaging. From ...
Highlights- Deep learning-based spectral super-resolution methods are reviewed.
- Three ...
Comments