Skip to main content
Springer Nature Link
Log in

Real-World Thermal Image Super-Resolution

  • Conference paper
  • First Online:
Advances in Visual Computing (ISVC 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 13017))

Included in the following conference series:

  • 1497 Accesses

Abstract

Thermal cameras are used in various domains where the vision of RGB cameras is limited. Thermographic imaging enables the visualizations of objects beyond the visible range, which enables its use in many applications like autonomous cars, nightly footage, military, or surveillance. However, the high cost of manufacturing this type of camera limits the spatial resolution that it can provide. Real-World Super-Resolution (RWSR) is a topic that can be used to solve this problem by using image processing techniques that enhance the quality of a real-world image by reconstructing lost high-frequency information. This work adapts an existing RWSR framework that is designed to super-resolve real-world RGB images. This framework estimates the degradation parameters needed to generate realistic Low-resolution (LR) and High-resolution (HR) image pairs, then the SR model learns the mapping between the LR and HR domains using the constructed image pairs and applies this mapping to new LR thermal images. The experiments results show a clear improvement in the perceptual quality in terms of clarity and sharpness, which surpasses the performance of the current SotA method for thermal image SR.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Agustsson, E., Timofte, R.: NTIRE 2017 challenge on single image super-resolution: dataset and study. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017

    Google Scholar 

  2. Ahn, N., Kang, B., Sohn, K.A.: Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 252–268 (2018)

    Google Scholar 

  3. Bell-Kligler, S., Shocher, A., Irani, M.: Blind super-resolution kernel estimation using an internal-gan (2020)

    Google Scholar 

  4. Cho, Y., Bianchi-Berthouze, N., Marquardt, N., Julier, S.J.: Deep thermal imaging. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, April 2018. https://doi.org/10.1145/3173574.3173576

  5. Choi, Y., Kim, N., Hwang, S., Kweon, I.S.: Thermal image enhancement using convolutional neural network. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 223–230. IEEE (2016)

    Google Scholar 

  6. Chudasama, V., et al.: Therisurnet - a computationally efficient thermal image super-resolution network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020

    Google Scholar 

  7. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2015)

    Google Scholar 

  8. Emad, M., Peemen, M., Corporaal, H.: Dualsr: Zero-shot dual learning for real-world super-resolution. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 1630–1639, January 2021

    Google Scholar 

  9. Fritsche, M., Gu, S., Timofte, R.: Frequency separation for real-world super-resolution. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3599–3608 (2019)

    Google Scholar 

  10. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2018)

    Google Scholar 

  11. Ji, X., Cao, Y., Tai, Y., Wang, C., Li, J., Huang, F.: Real-world super-resolution via kernel estimation and noise injection. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020

    Google Scholar 

  12. Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard GAN (2018)

    Google Scholar 

  13. Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4681–4690 (2017)

    Google Scholar 

  14. Lee, K., Lee, J., Lee, J., Hwang, S., Lee, S.: Brightness-based convolutional neural network for thermal image enhancement. IEEE Access 5, 26867–26879 (2017). https://doi.org/10.1109/ACCESS.2017.2769687

    Article  Google Scholar 

  15. Li, J., Fang, F., Mei, K., Zhang, G.: Multi-scale residual network for image super-resolution. In: The European Conference on Computer Vision (ECCV), September 2018

    Google Scholar 

  16. Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 136–144 (2017)

    Google Scholar 

  17. Lugmayr, A., et al.: NTIRE 2020 challenge on real-world image super-resolution: Methods and results. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 494–495 (2020)

    Google Scholar 

  18. Park, S.J., Son, H., Cho, S., Hong, K.S., Lee, S.: Srfeat: single image super-resolution with feature discrimination. In: Proceedings of the European Conference on Computer Vision (ECCV), September 2018

    Google Scholar 

  19. Rivadeneira, R.E.: Thermal image super-resolution challenge - PBVS 2020. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 432–439 (2020). https://doi.org/10.1109/CVPRW50498.2020.00056

    Google Scholar 

  20. Rivadeneira, R.E., Sappa, A.D., Vintimilla, B.X.: Thermal image super-resolution: a novel architecture and dataset. In: International Conference on Computer Vision Theory and Applications, pp. 1–2 (2020)

    Google Scholar 

  21. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: Orb: An efficient alternative to sift or surf. In: 2011 International Conference on Computer Vision, pp. 2564–2571 (2011). https://doi.org/10.1109/ICCV.2011.6126544

    Google Scholar 

  22. Shocher, A., Bagon, S., Isola, P., Irani, M.: Ingan: capturing and remapping the DNA of a natural image (2019)

    Google Scholar 

  23. Shocher, A., Cohen, N., Irani, M.: Zero-Shot super-resolution using deep internal learning (2017)

    Google Scholar 

  24. Sierra-Olympic: Vayu HD feature specification. https://sierraolympic.com/wp-content/uploads/2020/06/2020_VayuHD_Sell-Sheet_FINAL.pdf. Accessed 02 June 2021

  25. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)

    Google Scholar 

  26. Soh, J.W., Cho, S., Cho, N.I.: Meta-transfer learning for zero-shot super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3516–3525 (2020)

    Google Scholar 

  27. Wang, X., et al.: Esrgan: enhanced super-resolution generative adversarial networks. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops (2018)

    Google Scholar 

  28. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 286–301 (2018)

    Google Scholar 

  29. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Visual Analysis and Perception Laboratory, Aalborg University, Aalborg, Denmark

    Moaaz Allahham, Andreas Aakerberg, Kamal Nasrollahi & Thomas B. Moeslund

  2. Research Department, Milestone Systems A/S, Milestone Systems, Brøndby, Denmark

    Kamal Nasrollahi

Authors
  1. Moaaz Allahham
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andreas Aakerberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kamal Nasrollahi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas B. Moeslund
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Nevada, Reno, NV, USA

    George Bebis

  2. University of Texas at Arlington, Arlington, TX, USA

    Vassilis Athitsos

  3. University of South Carolina, Columbia, SC, USA

    Tong Yan

  4. City University of Hong Kong, Kowloon, Hong Kong

    Manfred Lau

  5. School of Engineering and Computing, University of Durham, Durham, Durham, UK

    Frederick Li

  6. Airbnb, New York, NY, USA

    Conglei Shi

  7. Peking University, Beijing, China

    Xiaoru Yuan

  8. Purdue University, West Lafayette, IN, USA

    Christos Mousas

  9. IST, School of Modeling, Simulation, and Training, Orlando, FL, USA

    Gerd Bruder

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Allahham, M., Aakerberg, A., Nasrollahi, K., Moeslund, T.B. (2021). Real-World Thermal Image Super-Resolution. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2021. Lecture Notes in Computer Science(), vol 13017. Springer, Cham. https://doi.org/10.1007/978-3-030-90439-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-90439-5_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-90438-8

  • Online ISBN: 978-3-030-90439-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics