Abstract
Multi-exposure High Dynamic Range (HDR) imaging is a challenging task when facing truncated texture and complex motion. Existing deep learning-based methods have achieved great success by either following the alignment and fusion pipeline or utilizing attention mechanism. However, the large computation cost and inference delay hinder them from deploying on resource limited devices. In this paper, to achieve better efficiency, a novel Selective Alignment Fusion Network (SAFNet) for HDR imaging is proposed. After extracting pyramid features, it jointly refines valuable area masks and cross-exposure motion in selected regions with shared decoders, and then fuses high quality HDR image in an explicit way. This approach can focus the model on finding valuable regions while estimating their easily detectable and meaningful motion. For further detail enhancement, a lightweight refine module is introduced which enjoys privileges from previous optical flow, selection masks and initial prediction. Moreover, to facilitate learning on samples with large motion, a new window partition cropping method is presented during training. Experiments on public and newly developed challenging datasets show that proposed SAFNet not only exceeds previous SOTA competitors quantitatively and qualitatively, but also runs order of magnitude faster. Code and dataset is available at https://github.com/ltkong218/SAFNet.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bogoni, L.: Extending dynamic range of monochrome and color images through fusion. In: Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, vol. 3, pp. 7–12 (2000)
Catley-Chandar, S., Tanay, T., Vandroux, L., Leonardis, A., Slabaugh, G., Pérez-Pellitero, E.: FlexHDR: modeling alignment and exposure uncertainties for flexible HDR imaging. IEEE Trans. Image Process. 31, 5923–5935 (2022)
Chen, J., Yang, Z., Chan, T.N., Li, H., Hou, J., Chau, L.P.: Attention-guided progressive neural texture fusion for high dynamic range image restoration. IEEE Trans. Image Process. 31, 2661–2672 (2022)
Chung, H., Cho, N.I.: LAN-HDR: luminance-based alignment network for high dynamic range video reconstruction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 12760–12769 (2023)
Debevec, P.E., Malik, J.: Recovering high dynamic range radiance maps from photographs. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, pp. 369–378. SIGGRAPH 1997 (1997)
Fossum, E.R., Hondongwa, D.B.: A review of the pinned photodiode for CCD and CMOS image sensors. IEEE J. Electron Devices Soc. 2(3), 33–43 (2014)
Froehlich, J., Grandinetti, S., Eberhardt, B., Walter, S., Schilling, A., Brendel, H.: Creating cinematic wide gamut HDR-video for the evaluation of tone mapping operators and HDR-displays. In: Digital Photography X, vol. 9023, p. 90230X (2014)
Gallo, O., Gelfandz, N., Chen, W.C., Tico, M., Pulli, K.: Artifact-free high dynamic range imaging. In: 2009 IEEE International Conference on Computational Photography (ICCP), pp. 1–7 (2009)
Grosch, T.: Fast and robust high dynamic range image generation with camera and object movement. In: IEEE Conference of Vision, Modeling and Visualization (2006)
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: 2015 IEEE International Conference on Computer Vision (ICCV) (2015)
Hu, J., Gallo, O., Pulli, K., Sun, X.: HDR Deghosting: how to deal with saturation? In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1163–1170 (2013)
Jacobs, K., Loscos, C., Ward, G.: Automatic high-dynamic range image generation for dynamic scenes. IEEE Comput. Graphics Appl. 28(2), 84–93 (2008)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Computer Vision – ECCV 2016, pp. 694–711 (2016)
Kalantari, N.K., Ramamoorthi, R.: Deep high dynamic range imaging of dynamic scenes. ACM Trans. Graph. 36(4), 1–12 (2017)
Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R.: High dynamic range video. ACM Trans. Graph. 22(3), 319–325 (2003)
Kearney, J.K., Thompson, W.B., Boley, D.L.: Optical flow estimation: an error analysis of gradient-based methods with local optimization. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-9(2), 229–244 (1987)
Khan, E.A., Akyuz, A.O., Reinhard, E.: Ghost removal in high dynamic range images. In: 2006 International Conference on Image Processing, pp. 2005–2008 (2006)
Kong, L., et al.: IFRNet: intermediate feature refine network for efficient frame interpolation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022)
Kong, L., Shen, C., Yang, J.: FastFlowNet: a lightweight network for fast optical flow estimation. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 10310–10316 (2021)
Kong, L., Yang, J.: FdFlowNet: fast optical flow estimation using a deep lightweight network. In: 2020 IEEE International Conference on Image Processing (ICIP) (2020)
Kong, L., Yang, J.: MDFlow: unsupervised optical flow learning by reliable mutual knowledge distillation. IEEE Trans. Circ. Syst. Video Technol. 33, 677–688 (2022)
Lee, C., Li, Y., Monga, V.: Ghost-free high dynamic range imaging via rank minimization. IEEE Signal Process. Lett. 21(9), 1045–1049 (2014)
Liu, Z., et al.: ADNet: attention-guided deformable convolutional network for high dynamic range imaging. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 463–470 (2021)
Liu, Z., Wang, Y., Zeng, B., Liu, S.: Ghost-free high dynamic range imaging with context-aware transformer. In: Computer Vision – ECCV 2022, pp. 344–360 (2022)
Ma, K., Duanmu, Z., Zhu, H., Fang, Y., Wang, Z.: Deep guided learning for fast multi-exposure image fusion. IEEE Trans. Image Process. 29, 2808–2819 (2020)
Ma, K., Li, H., Yong, H., Wang, Z., Meng, D., Zhang, L.: Robust multi-exposure image fusion: a structural patch decomposition approach. IEEE Trans. Image Process. 26(5), 2519–2532 (2017)
Mantiuk, R., Kim, K.J., Rempel, A.G., Heidrich, W.: HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans. Graph. 30(4), 1–14 (2011)
Meister, S., Hur, J., Roth, S.: UnFlow: unsupervised learning of optical flow with a bidirectional census loss. In: Proceedings of the AAAI Conference on Artificial Intelligence (2018)
Niu, Y., Wu, J., Liu, W., Guo, W., Lau, R.W.H.: HDR-GAN: HDR image reconstruction from multi-exposed LDR images with large motions. IEEE Trans. Image Process. 30, 3885–3896 (2021)
Oh, T.H., Lee, J.Y., Tai, Y.W., Kweon, I.S.: Robust high dynamic range imaging by rank minimization. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1219–1232 (2015)
Pece, F., Kautz, J.: Bitmap movement detection: HDR for dynamic scenes. In: 2010 Conference on Visual Media Production, pp. 1–8 (2010)
Prabhakar, K.R., Agrawal, S., Singh, D.K., Ashwath, B., Babu, R.V.: Towards practical and efficient high-resolution HDR Deghosting with CNN. In: Computer Vision – ECCV 2020, pp. 497–513 (2020)
Prabhakar, K.R., Arora, R., Swaminathan, A., Singh, K.P., Babu, R.V.: A fast, scalable, and reliable Deghosting method for extreme exposure fusion. In: 2019 IEEE International Conference on Computational Photography (ICCP), pp. 1–8 (2019)
Ram Prabhakar, K., Sai Srikar, V., Venkatesh Babu, R.: DeepFuse: a deep unsupervised approach for exposure fusion with extreme exposure image pairs. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017)
Ranjan, A., Black, M.J.: Optical flow estimation using a spatial pyramid network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Sen, P., Kalantari, N.K., Yaesoubi, M., Darabi, S., Goldman, D.B., Shechtman, E.: Robust patch-based HDR reconstruction of dynamic scenes. ACM Trans. Graph. 31(6), 1–11 (2012)
Song, J.W., Park, Y.I., Kong, K., Kwak, J., Kang, S.J.: Selective TransHDR: transformer-based selective HDR imaging using ghost region mask. In: Computer Vision – ECCV 2022, pp. 288–304 (2022)
Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2432–2439 (2010)
Sun, D., Yang, X., Liu, M.Y., Kautz, J.: PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Tel, S., et al.: Alignment-free HDR Deghosting with semantics consistent transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 12836–12845 (2023)
Wu, S., Xu, J., Tai, Y.W., Tang, C.K.: Deep high dynamic range imaging with large foreground motions. In: Computer Vision – ECCV 2018, pp. 120–135 (2018)
Xiong, P., Chen, Y.: Hierarchical fusion for practical ghost-free high dynamic range imaging. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 4025–4033 (2021)
Yan, Q., Chen, W., Zhang, S., Zhu, Y., Sun, J., Zhang, Y.: A unified HDR imaging method with pixel and patch level. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22211–22220 (2023)
Yan, Q., et al.: Attention-guided network for ghost-free high dynamic range imaging. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Yan, Q., et al.: Deep HDR imaging via a non-local network. IEEE Trans. Image Process. 29, 4308–4322 (2020)
Ye, Q., Xiao, J., Lam, K.-M., Okatani, T.: Progressive and selective fusion network for high dynamic range imaging. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 5290–5297 (2021)
Zhang, W., Cham, W.K.: Gradient-directed multiexposure composition. IEEE Trans. Image Process. 21(4), 2318–2323 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kong, L., Li, B., Xiong, Y., Zhang, H., Gu, H., Chen, J. (2025). SAFNet: Selective Alignment Fusion Network for Efficient HDR Imaging. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15084. Springer, Cham. https://doi.org/10.1007/978-3-031-73347-5_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-73347-5_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-73346-8
Online ISBN: 978-3-031-73347-5
eBook Packages: Computer ScienceComputer Science (R0)