ABSTRACT
Depth-of-field (DoF) rendering is an important technique in computational photography that simulates the human visual attention system. Existing DoF rendering methods usually suffer from a high computational cost. The task of DoF rendering can be accelerated by guided upsampling methods. However, the state-of-the-art guided upsampling methods fail to distinguish the focus and defocus areas, resulting in unsatisfying DoF effects. In this paper, we propose a novel deep weighted guided upsampling network (DWGUN) based on a encoder and decoder framework to jointly upsample the low-resolution DoF image under the guidance of the corresponding high-resolution all-in-focus image. Due to the intuitive weight design, the traditional weighted image upsampling is not tailored to DoF image upsampling. We propose a deep refocus-defocus edge-aware module (DREAM) to learn the spatially-varying weights and embed them in the deep weighted guided upsampling block (DWGUB). We have conducted comprehensive experiments to evaluate the proposed method. Rigorous ablation studies are also conducted to validate the rationality of the proposed components.
- Jonathan T. Barron, Andrew Adams, YiChang Shih, and Carlos Hernández. 2015. Fast bilateral-space stereo for synthetic defocus. In IEEE Conference on Computer Vision and Pattern Recognition. 4466--4474.Google ScholarCross Ref
- Benjamin Busam, Matthieu Hog, Steven McDonagh, and Gregory G. Slabaugh. 2019. SteReFo: Efficient Image Refocusing with Stereo Vision. In International Conference on Computer Vision Workshop. 3295--3304.Google Scholar
- Jiawen Chen, Andrew Adams, Neal Wadhwa, and Samuel W. Hasinoff. 2016. Bilateral guided upsampling. ACM Trans. Graph. 35, 6 (2016), 203.Google ScholarDigital Library
- Alireza Esmaeilzehi, M. Omair Ahmad, and M. N. S. Swamy. 2021. SRNSSI: A Deep Light-Weight Network for Single Image Super Resolution Using Spatial and Spectral Information. IEEE Trans. Computational Imaging 7 (2021), 409--421.Google ScholarCross Ref
- Rahul Garg, Neal Wadhwa, Sameer Ansari, and Jonathan T. Barron. 2019. Learning Single Camera Depth Estimation Using Dual-Pixels. In International Conference on Computer Vision. 7627--7636.Google Scholar
- Yong Guo, Jian Chen, Jingdong Wang, Qi Chen, Jiezhang Cao, Zeshuai Deng, Yanwu Xu, and Mingkui Tan. 2020. Closed-Loop Matters: Dual Regression Networks for Single Image Super-Resolution. In IEEE Conference on Computer Vision and Pattern Recognition. 5406--5415.Google Scholar
- Kaiming He, Jian Sun, and Xiaoou Tang. 2013. Guided Image Filtering. IEEE Trans. Pattern Anal. Mach. Intell. 35, 6 (2013), 1397--1409.Google ScholarDigital Library
- Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-Excitation Networks. In IEEE Conference on Computer Vision and Pattern Recognition. 7132--7141.Google Scholar
- Andrey Ignatov, Jagruti Patel, and Radu Timofte. 2020. Rendering Natural Camera Bokeh Effect with Deep Learning. In IEEE Conference on Computer Vision and Pattern Recognition. 1676--1686.Google Scholar
- Jing Jin, Junhui Hou, Jie Chen, Sam Kwong, and Jingyi Yu. 2020. Light Field Super-resolution via Attention-Guided Fusion of Hybrid Lenses. In International Conference on Multimedia. 193--201.Google ScholarDigital Library
- Johannes Kopf, Michael F. Cohen, Dani Lischinski, and Matthew Uyttendaele. 2007. Joint bilateral upsampling. ACM Trans. Graph. 26, 3 (2007), 96.Google ScholarDigital Library
- Fei Kou, Weihai Chen, Changyun Wen, and Zhengguo Li. 2015. Gradient Domain Guided Image Filtering. IEEE Trans. Image Process. 24, 11 (2015), 4528--4539.Google ScholarDigital Library
- Zhen Li, Jinglei Yang, Zheng Liu, Xiaomin Yang, Gwanggil Jeon, and Wei Wu. 2019. Feedback Network for Image Super-Resolution. In IEEE Conference on Computer Vision and Pattern Recognition. 3867--3876.Google Scholar
- Zhengguo Li, Jinghong Zheng, Zijian Zhu, Wei Yao, and Shiqian Wu. 2015. Weighted Guided Image Filtering. IEEE Trans. Image Process. 24, 1 (2015), 120--129.Google ScholarCross Ref
- Shunta Maeda. 2020. Unpaired Image Super-Resolution Using Pseudo-Supervision. In IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- Pingfan Song, Xin Deng, João F. C. Mota, Nikos Deligiannis, Pier Luigi Dragotti, and Miguel R. D. Rodrigues. 2020. Multimodal Image Super-Resolution via Joint Sparse Representations Induced by Coupled Dictionaries. IEEE Trans. Computational Imaging 6 (2020), 57--72.Google ScholarCross Ref
- Carlo Tomasi and Roberto Manduchi. 1998. Bilateral Filtering for Gray and Color Images. In International Conference on Computer Vision. 839--846.Google Scholar
- Neal Wadhwa, Rahul Garg, David E. Jacobs, Bryan E. Feldman, Nori Kanazawa, Robert Carroll, Yair Movshovitz-Attias, Jonathan T. Barron, Yael Pritch, and Marc Levoy. 2018. Synthetic depth-of-field with a single-camera mobile phone. ACM Trans. Graph. 37, 4 (2018), 64.Google ScholarDigital Library
- Lijun Wang, Xiaohui Shen, Jianming Zhang, Oliver Wang, Zhe L. Lin, Chih-Yao Hsieh, Sarah Kong, and Huchuan Lu. 2018. DeepLens: shallow depth of field from a single image. ACM Trans. Graph. 37, 6 (2018), 245:1--245:11.Google ScholarDigital Library
- Yuwang Wang, Yebin Liu, Wolfgang Heidrich, and Qionghai Dai. 2017. The Light Field Attachment: Turning a DSLR into a Light Field Camera Using a Low Budget Camera Ring. IEEE Trans. Vis. Comput. Graph. 23, 10 (2017), 2357--2364.Google ScholarDigital Library
- Huikai Wu, Shuai Zheng, Junge Zhang, and Kaiqi Huang. 2018. Fast End-to-End Trainable Guided Filter. In IEEE Conference on Computer Vision and Pattern Recognition. 1838--1847.Google Scholar
- Fuzhi Yang, Huan Yang, Jianlong Fu, Hongtao Lu, and Baining Guo. 2020. Learning Texture Transformer Network for Image Super-Resolution. In IEEE Conference on Computer Vision and Pattern Recognition. 5790--5799.Google Scholar
- Jaejun Yoo, Namhyuk Ahn, and Kyung-Ah Sohn. 2020. Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Analysis and a New Strategy. In IEEE Conference on Computer Vision and Pattern Recognition. 8372--8381.Google ScholarCross Ref
- Zhifei Zhang, Zhaowen Wang, Zhe L. Lin, and Hairong Qi. 2019. Image Super-Resolution by Neural Texture Transfer. In IEEE Conference on Computer Vision and Pattern Recognition. 7982--7991.Google ScholarCross Ref
- Mandan Zhao, Gaochang Wu, Yipeng Li, Xiangyang Hao, Lu Fang, and Yebin Liu. 2018. Cross-Scale Reference-Based Light Field Super-Resolution. IEEE Trans. Computational Imaging 4, 3 (2018), 406--418.Google ScholarCross Ref
Index Terms
Deep Weighted Guided Upsampling Network for Depth of Field Image Upsampling
Recommendations
RGB-Guided Hyperspectral Image Upsampling
ICCV '15: Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV)Hyperspectral imaging usually lack of spatial resolution due to limitations of hardware design of imaging sensors. On the contrary, latest imaging sensors capture a RGB image with resolution of multiple times larger than a hyperspectral image. In this ...
High Resolution Local Structure-Constrained Image Upsampling
With the development of ultra-high-resolution display devices, the visual perception of fine texture details is becoming more and more important. A method of high-quality image upsampling with a low cost is greatly needed. In this paper, we propose a fast ...
Fast multi-scale joint bilateral texture upsampling
Special Issue: CAD/Graphics 2009We present a new approach using a multi-scale joint bilateral filter for upsampling the synthesized texture generated by optimization-based methods. Our method is based on the following motivation: if the available exemplar texture is used as a priority ...
Comments