Skip to main content
Springer Nature Link
Log in

Superpixel-Informed Implicit Neural Representation for Multi-dimensional Data

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15060))

Included in the following conference series:

Abstract

Recently, implicit neural representations (INRs) have attracted increasing attention for multi-dimensional data recovery. However, INRs simply map coordinates via a multi-layer perceptron (MLP) to corresponding values, ignoring the inherent semantic information of the data. To leverage semantic priors from the data, we propose a novel Superpixel-informed INR (S-INR). Specifically, we suggest utilizing generalized superpixel instead of pixel as an alternative basic unit of INR for multi-dimensional data (e.g., images and weather data). The coordinates of generalized superpixels are first fed into exclusive attention-based MLPs, and then the intermediate results interact with a shared dictionary matrix. The elaborately designed modules in S-INR allow us to ingenuously exploit the semantic information within and across generalized superpixels. Extensive experiments on various applications validate the effectiveness and efficacy of our S-INR compared to state-of-the-art INR methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    For point data, pixels correspond to a feature value vector.

  2. 2.

    https://r0k.us/graphics/kodak/.

  3. 3.

    https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes.

  4. 4.

    https://icvl.cs.bgu.ac.il/hyperspectral/.

  5. 5.

    http://www.vision.deis.unibo.it/research/80-shot.

  6. 6.

    https://daac.ornl.gov/cgi-bin/dsviewer.pl?ds id=2130.

  7. 7.

    Specifically, the superpixel-based INR is based on using K INRs to represent K superpixels, where K is the number of generalized superpixels.

References

  1. Altman, N.S.: An introduction to kernel and nearest-neighbor nonparametric regression. Am. Stat. 46(3), 175–185 (1992)

    Article  MathSciNet  Google Scholar 

  2. Arthur, D., Vassilvitskii, S.: k-means++: the advantages of careful seeding. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1027–1035 (2007)

    Google Scholar 

  3. Ben-Shabat, Y., Koneputugodage, C.H., Gould, S.: DiGS: divergence guided shape implicit neural representation for unoriented point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 19323–19332 (2022)

    Google Scholar 

  4. Cervantes, P., Sekikawa, Y., Sato, I., Shinoda, K.: Implicit neural representations for variable length human motion generation. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision, ECCV 2022. LNCS, vol. 13677, pp. 356–372. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19790-1_22

  5. Chen, H., He, B., Wang, H., Ren, Y., Lim, S.N., Shrivastava, A.: NeRV: neural representations for videos. In: Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), vol. 34, pp. 21557–21568 (2021)

    Google Scholar 

  6. Chen, Y., Liu, S., Wang, X.: Learning continuous image representation with local implicit image function. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2021, pp. 8628–8638 (2021)

    Google Scholar 

  7. Davis, J., Nehab, D., Ramamoorthi, R., Rusinkiewicz, S.: Spacetime stereo: a unifying framework for depth from triangulation. IEEE Trans. Pattern Anal. Mach. Intell. 27(2), 296–302 (2005)

    Article  Google Scholar 

  8. Fathony, R., Sahu, A.K., Willmott, D., Kolter, J.Z.: Multiplicative filter networks. In: Proceedings of the International Conference on Learning Representations (ICLR) (2021)

    Google Scholar 

  9. Fujiwara, K., Hashimoto, T.: Neural implicit embedding for point cloud analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020, pp. 11734–11743 (2020)

    Google Scholar 

  10. Henzler, P., Mitra, N.J., Ritschel, T.: Learning a neural 3D texture space from 2D exemplars. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020, pp. 8353–8361 (2020)

    Google Scholar 

  11. Ho, T.K.: Random decision forests. In: Proceedings of 3rd International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 278–282 (1995)

    Google Scholar 

  12. Hofherr, F., Koestler, L., Bernard, F., Cremers, D.: Neural implicit representations for physical parameter inference from a single video. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), January 2023, pp. 2093–2103 (2023)

    Google Scholar 

  13. Hong, S., et al.: Neural matching fields: implicit representation of matching fields for visual correspondence. In: Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), vol. 35, pp. 13512–13526 (2022)

    Google Scholar 

  14. Hu, J., Shen, L., Albanie, S., Sun, G., Wu, E.: Squeeze-and-Excitation networks. IEEE Trans. Pattern Anal. Mach. Intell. 42(8), 2011–2023 (2020)

    Article  Google Scholar 

  15. Hunt, E.B., Marin, J., Stone, P.J.: Experiments in induction (1966)

    Google Scholar 

  16. Huynh-Thu, Q., Ghanbari, M.: Scope of validity of PSNR in image/video quality assessment. Electron. Lett. 44, 800–801 (2008)

    Article  Google Scholar 

  17. Jacot, A., Gabriel, F., Hongler, C.: Neural tangent kernel: convergence and generalization in neural networks. In: Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), vol. 31, p. 6 (2018)

    Google Scholar 

  18. Jayasundara, V., Agrawal, A., Heron, N., Shrivastava, A., Davis, L.S.: FlexNeRF: photorealistic free-viewpoint rendering of moving humans from sparse views. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 21118–21127 (2023)

    Google Scholar 

  19. Kaneko, T.: MIMO-NeRF: fast neural rendering with multi-input multi-output neural radiance fields. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2023, pp. 3273–3283 (2023)

    Google Scholar 

  20. Kazerouni, A., Azad, R., Hosseini, A., Merhof, D., Bagci, U.: INCODE: implicit neural conditioning with prior knowledge embeddings. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), January 2024, pp. 1298–1307 (2024)

    Google Scholar 

  21. Khan, M.O., Fang, Y.: Implicit neural representations for medical imaging segmentation. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) Medical Image Computing and Computer Assisted Intervention, MICCAI 2022. LNCS, vol. 13435, pp. 433–443. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16443-9_42

  22. Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the International Conference on Learning Representations (ICLR), May 2015, p. 13 (2015)

    Google Scholar 

  23. Lindell, D.B., Van Veen, D., Park, J.J., Wetzstein, G.: BACON: band-limited coordinate networks for multiscale scene representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 16252–16262 ()

    Google Scholar 

  24. Lu, Y., Wang, Z., Liu, M., Wang, H., Wang, L.: Learning spatial-temporal implicit neural representations for event-guided video super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 1557–1567 (2023)

    Google Scholar 

  25. Ma, C., Yu, P., Lu, J., Zhou, J.: Recovering realistic details for magnification-arbitrary image super-resolution. IEEE Trans. Image Process. 31, 3669–3683 (2022)

    Article  Google Scholar 

  26. Mai, L., Liu, F.: Motion-adjustable neural implicit video representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 10738–10747 (2022)

    Google Scholar 

  27. Martel, J.N.P., Lindell, D.B., Lin, C.Z., Chan, E.R., Monteiro, M., Wetzstein, G.: ACORN: adaptive coordinate networks for neural scene representation. ACM Trans. Graph. 40(4), 1–13 (2021)

    Article  Google Scholar 

  28. Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 405–421. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_24

    Chapter  Google Scholar 

  29. Oechsle, M., Mescheder, L., Niemeyer, M., Strauss, T., Geiger, A.: Texture fields: learning texture representations in function space. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019, pp. 4531–4540 ()

    Google Scholar 

  30. Rahaman, N., et al.: On the spectral bias of neural networks. In: Proceedings of the 36th International Conference on Machine Learning (ICML), June 2019, vol. 97, pp. 5301–5310 (2019)

    Google Scholar 

  31. Ramasinghe, S., Lucey, S.: Beyond periodicity: towards a unifying framework for activations in coordinate-MLPs. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision, ECCV 2022. LNCS, vol. 13693, pp. 142–158. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19827-4_9

  32. Rebain, D., Jiang, W., Yazdani, S., Li, K., Yi, K.M., Tagliasacchi, A.: DeRF: decomposed radiance fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2021, pp. 14153–14161 (2021)

    Google Scholar 

  33. Reed, A.W., et al.: Dynamic CT reconstruction from limited views with implicit neural representations and parametric motion fields. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2021, pp. 2258–2268 (2021)

    Google Scholar 

  34. Reiser, C., Peng, S., Liao, Y., Geiger, A.: KiloNeRF: speeding up neural radiance fields with thousands of tiny mlps. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2021, pp. 14335–14345 (2021)

    Google Scholar 

  35. Saragadam, V., LeJeune, D., Tan, J., Balakrishnan, G., Veeraraghavan, A., Baraniuk, R.G.: WIRE: wavelet implicit neural representations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 18507–18516 (2023)

    Google Scholar 

  36. Saragadam, V., Tan, J., Balakrishnan, G., Baraniuk, R.G., Veeraraghavan, A.: MINER: multiscale implicit neural representation. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision, ECCV 2022. LNCS, vol. 13683, pp. 318–333. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20050-2_19

  37. Serrano, D., Szymkowiak, J., Musialski, P.: HOSC: a periodic activation function for preserving sharp features in implicit neural representations (2024)

    Google Scholar 

  38. Shen, Z., Cheng, Y., Chan, R.H., Liò, P., Schönlieb, C.B., Aviles-Rivero, A.I.: TRIDENT: the nonlinear trilogy for implicit neural representations (2023)

    Google Scholar 

  39. Sitzmann, V., Martel, J., Bergman, A., Lindell, D., Wetzstein, G.: Implicit neural representations with periodic activation functions. In: Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), vol. 33, pp. 7462–7473 (2020)

    Google Scholar 

  40. Su, K., Chen, M., Shlizerman, E.: INRAS: implicit neural representation for audio scenes. In: Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), vol. 35, pp. 8144–8158 (2022)

    Google Scholar 

  41. Sun, Y., Liu, J., Xie, M., Wohlberg, B., Kamilov, U.S.: CoIL: coordinate-based internal learning for tomographic imaging. IEEE Trans. Computat. Imaging 7, 1400–1412 (2021)

    Article  Google Scholar 

  42. Tancik, M., et al.: Fourier features let networks learn high frequency functions in low dimensional domains. In: Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), vol. 33, pp. 7537–7547 (2020)

    Google Scholar 

  43. Ulyanov, D., Vedaldi, A., Lempitsky, V.: Deep image prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018, pp. 9446–9454 (2018)

    Google Scholar 

  44. Strümpler, Y., Postels, J., Yang, R., Gool, L.V., Tombari, F.: Implicit neural representations for image compression. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision, ECCV 2022. LNCS, vol. 13686, pp. 74–91. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19809-0_5

  45. Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  46. Xiang, F., Xu, Z., Hašan, M., Hold-Geoffroy, Y., Sunkavalli, K., Su, H.: NeuTex: neural texture mapping for volumetric neural rendering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2021, pp. 7119–7128 (2021)

    Google Scholar 

  47. Xu, D., Wang, P., Jiang, Y., Fan, Z., Wang, Z.: Signal processing for implicit neural representations. In: Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), vol. 35, pp. 13404–13418 (2022)

    Google Scholar 

  48. Xu, R., Yao, M., Chen, C., Wang, L., Xiong, Z.: Continuous spectral reconstruction from RGB images via implicit neural representation. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds.) Computer Vision, ECCV 2022 Workshops. LNCS, vol 13805, pp. 78–94. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-25072-9_6

  49. Xu, W., Jiao, J.: Revisiting implicit neural representations in low-level vision. In: International Conference on Learning Representations Workshop (2023)

    Google Scholar 

  50. Yang, L., Kim, B., Zoss, G., Gözcü, B., Gross, M., Solenthaler, B.: Implicit neural representation for physics-driven actuated soft bodies. ACM Trans. Graph. 41(4) (2022)

    Google Scholar 

  51. Yang, S., Ding, M., Wu, Y., Li, Z., Zhang, J.: Implicit neural representation for cooperative low-light image enhancement. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2023, pp. 12918–12927 (2023)

    Google Scholar 

  52. Zhang, K., Zhu, D., Min, X., Zhai, G.: Implicit neural representation learning for hyperspectral image super-resolution. IEEE Trans. Geosci. Remote Sens. 61, 1–12 (2023)

    Google Scholar 

  53. Zhao, Q., Asif, M.S., Ma, Z.: DNeRV: modeling inherent dynamics via difference neural representation for videos. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2023, pp. 2031–2040 (2023)

    Google Scholar 

Download references

Acknowledgments

This research is supported by NSFC (No. 12371456, 12171-072, 62131005, 62306248), Sichuan Science and Technology Program (No. 2024NSFJQ0038, 2023ZYD0007), and National Key Research and Development Program of China (No. 2020YFA0714001).

Author information

Authors and Affiliations

  1. University of Electronic Science and Technology of China, Chengdu, China

    Jiayi Li & Xile Zhao

  2. Southwest Jiaotong University, Chengdu, China

    Jianli Wang

  3. Southern University of Science and Technology, Shenzhen, China

    Chao Wang

  4. Jiangxi University of Finance and Economics, Nanchang, China

    Min Wang

Authors
  1. Jiayi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xile Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianli Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Min Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xile Zhao .

Editor information

Editors and Affiliations

  1. University of Birmingham, Birmingham, UK

    Aleš Leonardis

  2. University of Trento, Trento, Italy

    Elisa Ricci

  3. Technical University of Darmstadt, Darmstadt, Germany

    Stefan Roth

  4. Princeton University, Princeton, NJ, USA

    Olga Russakovsky

  5. Czech Technical University in Prague, Prague, Czech Republic

    Torsten Sattler

  6. École des Ponts ParisTech, Marne-la-Vallée, France

    Gül Varol

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, J., Zhao, X., Wang, J., Wang, C., Wang, M. (2025). Superpixel-Informed Implicit Neural Representation for Multi-dimensional Data. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15060. Springer, Cham. https://doi.org/10.1007/978-3-031-72627-9_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-72627-9_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-72626-2

  • Online ISBN: 978-3-031-72627-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics