Abstract
Deep learning has been one promising tool for compressive imaging whose task is to reconstruct latent images from their compressive measurements. Aiming at addressing the limitations of supervised deep learning-based methods caused by their prerequisite on the ground truths of latent images, this paper proposes an unsupervised approach that trains a deep image reconstruction model using only a set of compressive measurements. The training is self-supervised in the domain of measurements and the domain of images, using a double-head noise-injected loss with a sign-flipping-based noise generator. In addition, the proposed scheme can also be used for efficiently adapting a trained model to a test sample for further improvement, with much less overhead than existing internal learning methods. Extensive experiments show that the proposed approach provides noticeable performance gain over existing unsupervised methods and competes well against the supervised ones.
Yuhui Quan is also with Pazhou Lab, Guangzhou 510335, China. He would like to thank the support in part by National Natural Science Foundation of China under Grant 61872151 and in part by Natural Science Foundation of Guangdong Province under Grant 2022A1515011755.
Hui Ji would like to thank the support in part by Singapore MOE AcRF under Grant R-146-000-315-114.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, D., Davies, M.E.: Deep decomposition learning for inverse imaging problems. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12373, pp. 510–526. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58604-1_31
Chen, D., Tachella, J., Davies, M.E.: Equivariant imaging: learning beyond the range space. In: Proceedings of International Conference on Computer Vision (2021)
Chen, D., Tachella, J., Davies, M.E.: Robust equivariant imaging: a fully unsupervised framework for learning to image from noisy and partial measurements. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2022)
Chen, J., Sun, Y., Liu, Q., Huang, R.: Learning memory augmented cascading network for compressed sensing of images. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12367, pp. 513–529. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58542-6_31
Clark, K., et al.: The cancer imaging archive (TCIA): maintaining and operating a public information repository. J. Digit. Imaging 26(6), 1045–1057 (2013)
Cole, E.K., Pauly, J.M., Vasanawala, S.S., Ong, F.: Unsupervised MRI reconstruction with generative adversarial networks. arXiv preprint arXiv:2008.13065 (2020)
Diamond, S., Sitzmann, V., Heide, F., Wetzstein, G.: Unrolled optimization with deep priors. arXiv preprint arXiv:1705.08041 (2017)
Ding, Q., Chen, G., Zhang, X., Huang, Q., Ji, H., Gao, H.: Low-dose CT with deep learning regularization via proximal forward-backward splitting. Phys. Med. Biol. 65(12), 125009 (2020)
Dong, W., Shi, G., Li, X., Ma, Y., Huang, F.: Compressive sensing via nonlocal low-rank regularization. IEEE Trans. Image Process. 23(8), 3618–3632 (2014)
Eldar, Y.C.: Generalized sure for exponential families: applications to regularization. IEEE Trans. Signal Process. 57(2), 471–481 (2008)
Feng, C.M., Yang, Z., Chen, G., Xu, Y., Shao, L.: Dual-octave convolution for accelerated parallel MR image reconstruction. In: Proceedings of AAAI Conference on Artificial Intelligence (2021)
Jalal, A., Karmalkar, S., Dimakis, A.G., Price, E.: Instance-optimal compressed sensing via posterior sampling. In: Proceedings of International Conference on Machine Learning (2021)
Jin, K.H., McCann, M.T., Froustey, E., Unser, M.A.: Deep convolutional neural network for inverse problems in imaging. IEEE Trans. Image Process. 26, 4509–4522 (2017)
Kabkab, M., Samangouei, P., Chellappa, R.: Task-aware compressed sensing with generative adversarial networks. In: Proceedings of AAAI Conference on Artificial Intelligence (2018)
Kadkhodaie, Z., Simoncelli, E.: Stochastic solutions for linear inverse problems using the prior implicit in a denoiser. In: Proceedings of Advances in Neural Information Processing Systems, vol. 34 (2021)
Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., Ashok, A.: ReconNet: non-iterative reconstruction of images from compressively sensed measurements. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 449–458 (2016)
Liu, J., Kuang, T., Zhang, X.: Image reconstruction by splitting deep learning regularization from iterative inversion. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 224–231. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_26
Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn. Reson. Med. 58(6), 1182–1195 (2007)
Mardani, M., et al.: Neural proximal gradient descent for compressive imaging. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of International Conference on Computer Vision, IEEE, vol. 2, pp. 416–423. (2001)
Metzler, C., Mousavi, A., Heckel, R., Baraniuk, R.: Unsupervised learning with stein’s unbiased risk estimator. arXiv preprint arXiv:1805.10531 (2018)
Metzler, C.A., Maleki, A., Baraniuk, R.: BM3D-AMP: a new image recovery algorithm based on BM3D denoising. In: Proceedings of International Conference on Image Processing, pp. 3116–3120. IEEE (2015)
Metzler, C.A., Maleki, A., Baraniuk, R.: From denoising to compressed sensing. IEEE Trans. Inf. Theory 62(9), 5117–5144 (2016)
Metzler, C.A., Mousavi, A., Baraniuk, R.G.: Learned D-AMP: principled neural network based compressive image recovery. In: Proceedings of Conference on Neural Information Processing Systems (2017)
Pang, T., Quan, Y., Ji, H.: Self-supervised Bayesian deep learning for image recovery with applications to compressive sensing. In: Proceedings of European Conference on Computer Vision (2020)
Pang, T., Zheng, H., Quan, Y., Ji, H.: Recorrupted-to-recorrupted: unsupervised deep learning for image denoising. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2021)
Quan, Y., Chen, M., Pang, T., Ji, H.: Self2self with dropout: learning self-supervised denoising from single image. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1890–1898 (2020)
Quan, Y., Qin, X., Chen, M., Huang, Y.: High-quality self-supervised snapshot hyperspectral imaging. In: IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, pp. 1526–1530. (2022)
Shi, W., Jiang, F., Liu, S., Zhao, D.: Scalable convolutional neural network for image compressed sensing. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 12290–12299 (2019)
Shi, W., Jiang, F., Liu, S., Zhao, D.: Image compressed sensing using convolutional neural network. IEEE Trans. Image Process. 29, 375–388 (2019)
Sun, Y., Chen, J., Liu, Q., Liu, B., Guo, G.: Dual-path attention network for compressed sensing image reconstruction. IEEE Trans. Image Process. 29, 9482–9495 (2020)
Ulyanov, D., Vedaldi, A., Lempitsky, V.: Deep image prior. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 9446–9454 (2018)
Wang, W., Li, J., Ji, H.: Self-supervised deep image restoration via adaptive stochastic gradient langevin dynamics. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1989–1998 (2022)
Wei, K., Aviles-Rivero, A., Liang, J., Fu, Y., Schönlieb, C.B., Huang, H.: Tuning-free plug-and-play proximal algorithm for inverse imaging problems. In: Proceedings of International Conference on Machine Learning. PMLR, pp. 10158–10169 (2020)
Xia, Z., Chakrabarti, A.: Training image estimators without image ground-truth. In: Proceedings of Conference on Neural Information Processing Systems (2019)
Xiang, J., Dong, Y., Yang, Y.: FISTA-Net: learning a fast iterative shrinkage thresholding network for inverse problems in imaging. IEEE Trans. Med. Imaging 40(5), 1329–1339 (2021)
Xin, B., Wang, Y., Gao, W., Wipf, D.: Maximal sparsity with deep networks? In: Proceedings of Conference on Neural Information Processing Systems (2016)
Xu, K., Zhang, Z., Ren, F.: LAPRAN: a scalable Laplacian pyramid reconstructive adversarial network for flexible compressive sensing reconstruction. In: Proceedings of European Conference on Computer Vision, pp. 485–500 (2018)
Yang, Y., Sun, J., Li, H., Xu, Z.: ADMM-CSNet: a deep learning approach for image compressive sensing. IEEE Trans. Pattern Anal. Mach. Intell. 42(3), 521–538 (2019)
You, D., Zhang, J., Xie, J., Chen, B., Ma, S.: Coast: controllable arbitrary-sampling network for compressive sensing. IEEE Trans. Image Process. 30, 6066–6080 (2021)
Zalbagi Darestani, M., Heckel, R.: Accelerated MRI with un-trained neural networks. In: IEEE Transactions on Computational Imaging. vol. 7, pp. 724–733 (2021)
Zhang, J., Ghanem, B.: ISTA-Net: interpretable optimization-inspired deep network for image compressive sensing. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1828–1837 (2018)
Zhang, Z., Liu, Y., Liu, J., Wen, F., Zhu, C.: AMP-Net: denoising-based deep unfolding for compressive image sensing. IEEE Trans. Image Process. 30, 1487–1500 (2021)
Zheng, H., Fang, F., Zhang, G.: Cascaded dilated dense network with two-step data consistency for MRI reconstruction. In: Proceedings of Conference on Neural Information Processing Systems (2019)
Zhou, B., Zhou, S.K.: DuDoRNet: learning a dual-domain recurrent network for fast MRI reconstruction with deep T1 prior. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 4273–4282 (2020)
Zhussip, M., Soltanayev, S., Chun, S.: Training deep learning based image denoisers from undersampled measurements without ground truth and without image prior. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 10255–10264 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Quan, Y., Qin, X., Pang, T., Ji, H. (2022). Dual-Domain Self-supervised Learning and Model Adaption for Deep Compressive Imaging. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13690. Springer, Cham. https://doi.org/10.1007/978-3-031-20056-4_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-20056-4_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20055-7
Online ISBN: 978-3-031-20056-4
eBook Packages: Computer ScienceComputer Science (R0)