Skip to main content
Springer Nature Link
Log in

Shape from Heat Conduction

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15096))

Included in the following conference series:

  • 350 Accesses

Abstract

Thermal cameras measure the temperature of objects based on radiation emitted in the infrared spectrum. In this work, we propose a novel shape recovery approach that exploits the properties of heat transport, specifically heat conduction, induced on objects when illuminated using simple light bulbs. Although heat transport occurs in the entirety of an object’s volume, we show a surface approximation that enables shape recovery and empirically analyze its validity for objects with varying thicknesses. We develop an algorithm that solves a linear system of equations to estimate the intrinsic shape Laplacian from thermal videos along with several properties including heat capacity, convection coefficient, and absorbed heat flux under uncalibrated lighting of arbitrary shapes. Further, we propose a novel shape from Laplacian objective that aims to resolve the inherent shape ambiguities by drawing insights from absorbed heat flux images using two unknown lights sources. Finally, we devise a coarse-to-fine refinement strategy that faithfully recovers both low- and high-frequency shape details. We validate our method by showing accurate reconstructions, to within an error of 1–2 mm (object size \(\le \) 13.5 cm), in both simulations and from noisy thermal videos of real-world objects with complex shapes and material properties including those that are transparent and translucent to visible light. We believe leveraging heat transport as a novel cue for vision can enable new imaging capabilities.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Ideally, four light sources are required to resolve the per-pixel ambiguity but we found that using two light sources to correct a few normals is sufficient for the optimization.

References

  1. Adato, Y., Vasilyev, Y., Zickler, T., Ben-Shahar, O.: Shape from specular flow. IEEE Trans. Pattern Anal. Mach. Intell. 32(11), 2054–2070 (2010). https://doi.org/10.1109/TPAMI.2010.126

    Article  Google Scholar 

  2. Alexa, M., Herholz, P., Kohlbrenner, M., Sorkine-Hornung, O.: Properties of laplace operators for tetrahedral meshes. In: Computer Graphics Forum (2020). https://doi.org/10.1111/cgf.14068

  3. Bergman, T.L.: Introduction to Heat Transfer. Wiley, Hoboken (2011)

    Google Scholar 

  4. Boscaini, D., Eynard, D., Kourounis, D., Bronstein, M.M.: Shape-from-operator: recovering shapes from intrinsic operators. In: Computer Graphics Forum, no. 2, pp. 265–274. https://doi.org/10.1111/cgf.12558

  5. Brahmbhatt, S., Ham, C., Kemp, C.C., Hays, J.: ContactDB: analyzing and predicting grasp contact via thermal imaging. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  6. Calcagni, G., Oriti, D., Thürigen, J.: Laplacians on discrete and quantum geometries. Class. Quantum Gravity 30(12), 125006 (2013). https://doi.org/10.1088/0264-9381/30/12/125006

    Article  MathSciNet  Google Scholar 

  7. Chen, I.C., Wang, C.J., Wen, C.K., Tzou, S.J.: Multi-person pose estimation using thermal images. IEEE Access 8, 174964–174971 (2020). https://doi.org/10.1109/ACCESS.2020.3025413

    Article  Google Scholar 

  8. Chern, A., Knöppel, F., Pinkall, U., Schröder, P.: Shape from metric. ACM Trans. Graph. 37(4) (2018). https://doi.org/10.1145/3197517.3201276

  9. Dashpute, A., et al.: Thermal spread functions (TSF): physics-guided material classification. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023). https://doi.org/10.1109/cvpr52729.2023.00164

  10. Dellaert, F., Seitz, S., Thorpe, C., Thrun, S.: Structure from motion without correspondence. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662), vol. 2, pp. 557–564 (2000). https://doi.org/10.1109/CVPR.2000.854916

  11. Eren, G., et al.: Scanning from heating: 3D shape estimation of transparent objects from local surface heating. Opt. Express 17(14), 11457–11468 (2009). https://doi.org/10.1364/OE.17.011457

    Article  Google Scholar 

  12. Forsyth, D.A.: Shape from texture without boundaries. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 225–239. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-47977-5_15

    Chapter  Google Scholar 

  13. Frankot, R., Chellappa, R.: A method for enforcing integrability in shape from shading algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 10(4), 439–451 (1988). https://doi.org/10.1109/34.3909

    Article  Google Scholar 

  14. Fu, C., Hu, Y., Wu, X., Shi, H., Mei, T., He, R.: CM-NAS: cross-modality neural architecture search for visible-infrared person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 11823–11832 (2021)

    Google Scholar 

  15. Gan, L., Lee, C., Chung, S.J.: Unsupervised RGB-to-thermal domain adaptation via multi-domain attention network. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE (2023). https://doi.org/10.1109/icra48891.2023.10160872

  16. Garrido, I., Lagüela, S., Otero, R., Arias, P.: Thermographic methodologies used in infrastructure inspection: a review–post-processing procedures. Appl. Energy 266, 114857 (2020). https://doi.org/10.1016/j.apenergy.2020.114857

    Article  Google Scholar 

  17. Goes, F.D., Memari, P., Mullen, P., Desbrun, M.: Weighted triangulations for geometry processing. ACM Trans. Graph. 33(3) (2014). https://doi.org/10.1145/2602143

  18. Gordon, C., Webb, D.L., Wolpert, S.: One cannot hear the shape of a drum. Bull. Am. Math. Soc. 27(1), 134–138 (1992). https://doi.org/10.1090/s0273-0979-1992-00289-6

    Article  MathSciNet  Google Scholar 

  19. Gray, A.: Modern Differential Geometry of Curves and Surfaces with Mathematica, 1st edn. CRC Press Inc, Boca Raton (1996)

    Google Scholar 

  20. Hildebrandt, C., Raschner, C., Ammer, K.: An overview of recent application of medical infrared thermography in sports medicine in Austria. Sensors 10(5), 4700–4715 (2010). https://doi.org/10.3390/s100504700

    Article  Google Scholar 

  21. Huo, D., Wang, J., Qian, Y., Yang, Y.H.: Glass segmentation with RGB-thermal image pairs. Trans. Img. Proc. 32, 1911–1926 (2023). https://doi.org/10.1109/TIP.2023.3256762

  22. Iwasawa, S., Ebihara, K., Ohya, J., Morishima, S.: Real-time estimation of human body posture from monocular thermal images. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 15–20 (1997). https://doi.org/10.1109/CVPR.1997.609290

  23. Kac, M.: Can one hear the shape of a drum? Am. Math. Mon. 73(4P2), 1–23 (1966). https://doi.org/10.1080/00029890.1966.11970915

  24. Kadambi, A., Taamazyan, V., Shi, B., Raskar, R.: Polarized 3D: high-quality depth sensing with polarization cues. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 3370–3378 (2015). https://doi.org/10.1109/ICCV.2015.385

  25. Kerr, E., McGinnity, T., Coleman, S.: Material classification based on thermal properties - a robot and human evaluation. In: 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1048–1053 (2013). https://doi.org/10.1109/ROBIO.2013.6739602

  26. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014)

    Google Scholar 

  27. Kütük, Z., Algan, G.: Semantic segmentation for thermal images: a comparative survey. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 286–295 (2022)

    Google Scholar 

  28. Liu, R., Vondrick, C.: Humans as light bulbs: 3D human reconstruction from thermal reflection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12531–12542 (2023)

    Google Scholar 

  29. Maeda, T., Wang, Y., Raskar, R., Kadambi, A.: Thermal non-line-of-sight imaging. In: 2019 IEEE International Conference on Computational Photography (ICCP), pp. 1–11 (2019). https://doi.org/10.1109/ICCPHOT.2019.8747343

  30. Meyer, M., Desbrun, M., Schröder, P., Barr, A.H.: Discrete differential-geometry operators for triangulated 2-manifolds. In: Hege, H.C., Polthier, K. (eds.) Visualization and Mathematics III, pp. 35–57. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-662-05105-4_2

    Chapter  Google Scholar 

  31. Miyazaki, D., Tan, R.T., Hara, K., Ikeuchi, K.: Polarization-based inverse rendering from a single view. In: Proceedings Ninth IEEE International Conference on Computer Vision, vol. 2, pp. 982–987 (2003). https://doi.org/10.1109/ICCV.2003.1238455

  32. Miyazaki, D., Saito, M., Sato, Y., Ikeuchi, K.: Determining surface orientations of transparent objects based on polarization degrees in visible and infrared wavelengths. J. Opt. Soc. Am. A 19(4), 687–694 (2002). https://doi.org/10.1364/JOSAA.19.000687

    Article  Google Scholar 

  33. Nagase, Y., Kushida, T., Tanaka, K., Funatomi, T., Mukaigawa, Y.: Shape from thermal radiation: passive ranging using multi-spectral LWIR measurements. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12651–12661 (2022). https://doi.org/10.1109/CVPR52688.2022.01233

  34. Nicolet, B., Jacobson, A., Jakob, W.: Large steps in inverse rendering of geometry. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 40(6) (2021). https://doi.org/10.1145/3478513.3480501

  35. Park, H., Lee, S., Lee, J., Ham, B.: Learning by aligning: visible-infrared person re-identification using cross-modal correspondences. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 12046–12055 (2021)

    Google Scholar 

  36. Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library (2019)

    Google Scholar 

  37. Pinkall, U., Polthier, K.: Computing discrete minimal surfaces and their conjugates. Exp. Math. 2(1), 15–36 (1993)

    Article  MathSciNet  Google Scholar 

  38. Ramanagopal, M., Narayanan, S., Sankaranarayanan, A.C., Narasimhan, S.G.: A theory of joint light and heat transport for lambertian scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11924–11933 (2024)

    Google Scholar 

  39. Ravi, N., et al.: Accelerating 3D deep learning with PyTorch3D. arXiv:2007.08501 (2020)

  40. Riba, J.R., Canals, T., Cantero, R.: Recovered paperboard samples identification by means of mid-infrared sensors. IEEE Sens. J. 13(7), 2763–2770 (2013). https://doi.org/10.1109/JSEN.2013.2257943

    Article  Google Scholar 

  41. Rivadeneira, R.E., et al.: Thermal image super-resolution challenge results - PBVS 2022. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 417–425 (2022). https://doi.org/10.1109/CVPRW56347.2022.00057

  42. Salamati, N., Fredembach, C., Süsstrunk, S.: Material classification using color and NIR images. In: Color and Imaging Conference, vol. 17 (2009). https://doi.org/10.2352/CIC.2009.17.1.art00040

  43. Salvi, J., Fernandez, S., Pribanic, T., Llado, X.: A state of the art in structured light patterns for surface profilometry. Pattern Recogn. 43(8), 2666–2680 (2010). https://doi.org/10.1016/j.patcog.2010.03.004

    Article  Google Scholar 

  44. Saponaro, P., Sorensen, S., Kolagunda, A., Kambhamettu, C.: Material classification with thermal imagery. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4649–4656 (2015). https://doi.org/10.1109/CVPR.2015.7299096

  45. Sheinin, M., Sankaranarayanan, A., Narasimhan, S.G.: Projecting trackable thermal patterns for dynamic computer vision. In: Proceedings of IEEE/CVF CVPR (2024)

    Google Scholar 

  46. Shin, U., Park, J., Kweon, I.S.: Deep depth estimation from thermal image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1043–1053 (2023)

    Google Scholar 

  47. Solomon, J., Crane, K., Vouga, E.: Laplace-beltrami: the swiss army knife of geometry processing. In: Symposium on Geometry Processing Graduate School, Cardiff, UK (2014)

    Google Scholar 

  48. Srinivasan Ramanagopal, M., Zhang, Z., Vasudevan, R., Johnson Roberson, M.: Pixel-wise motion deblurring of thermal videos. In: Robotics: Science and Systems XVI. RSS2020, Robotics: Science and Systems Foundation (2020). https://doi.org/10.15607/rss.2020.xvi.022

  49. Tanaka, K., et al.: Time-resolved far infrared light transport decomposition for thermal photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 43(6), 2075–2085 (2021). https://doi.org/10.1109/TPAMI.2019.2959304

    Article  Google Scholar 

  50. Tang, Z., Ye, W., Ma, W.C., Zhao, H.: What happened 3 seconds ago? Inferring the past with thermal imaging. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17111–17120 (2023)

    Google Scholar 

  51. Treptow, A., Cielniak, G., Duckett, T.: Real-time people tracking for mobile robots using thermal vision. Robot. Auton. Syst. 54(9), 729–739 (2006). https://doi.org/10.1016/j.robot.2006.04.013. Selected papers from the 2nd European Conference on Mobile Robots (ECMR ’05)

  52. Vollmer, M., Mllmann, K.P.: Some Basic Concepts in Heat Transfer, chap. 4, pp. 351–392. Wiley (2017). https://doi.org/10.1002/9783527693306.ch4

  53. Wei, Z., Yang, X., Wang, N., Gao, X.: Syncretic modality collaborative learning for visible infrared person re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 225–234 (2021)

    Google Scholar 

  54. Woodham, R.J.: Photometric Method for Determining Surface Orientation from Multiple Images, pp. 513–531. MIT Press, Cambridge (1989)

    Google Scholar 

  55. Zeng, W., Guo, R., Luo, F., Gu, X.: Discrete heat kernel determines discrete riemannian metric. Graph. Models 74(4), 121–129 (2012). https://doi.org/10.1016/j.gmod.2012.03.009

  56. Zhang, P., Zhao, J., Wang, D., Lu, H., Ruan, X.: Visible-thermal UAV tracking: a large-scale benchmark and new baseline. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8886–8895 (2022)

    Google Scholar 

Download references

Acknowledgements

This work was partly supported by NSF grants IIS-2107236, CCF-1730147, and NSF-NIFA AI Institute for Resilient Agriculture. Thanks to Keenan Crane for useful discussions.

Author information

Authors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, USA

    Sriram Narayanan, Mani Ramanagopal, Aswin C. Sankaranarayanan & Srinivasa G. Narasimhan

  2. Weizmann Institute of Science, Rehovot, Israel

    Mark Sheinin

Authors
  1. Sriram Narayanan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mani Ramanagopal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mark Sheinin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aswin C. Sankaranarayanan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Srinivasa G. Narasimhan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sriram Narayanan .

Editor information

Editors and Affiliations

  1. University of Birmingham, Birmingham, UK

    Aleš Leonardis

  2. University of Trento, Trento, Italy

    Elisa Ricci

  3. Technical University of Darmstadt, Darmstadt, Hessen, Germany

    Stefan Roth

  4. Princeton University, Palo Alto, CA, USA

    Olga Russakovsky

  5. Czech Technical University in Prague, Prague, Czech Republic

    Torsten Sattler

  6. École des Ponts ParisTech, Marne-la-Vallée, France

    Gül Varol

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 663 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Narayanan, S., Ramanagopal, M., Sheinin, M., Sankaranarayanan, A.C., Narasimhan, S.G. (2025). Shape from Heat Conduction. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15096. Springer, Cham. https://doi.org/10.1007/978-3-031-72920-1_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-72920-1_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-72919-5

  • Online ISBN: 978-3-031-72920-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics