Skip to main content

City-on-Web: Real-Time Neural Rendering of Large-Scale Scenes on the Web

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Abstract

Existing neural radiance field-based methods can achieve real-time rendering of small scenes on the web platform. However, extending these methods to large-scale scenes still poses significant challenges due to limited resources in computation, memory, and bandwidth. In this paper, we propose City-on-Web, the first method for real-time rendering of large-scale scenes on the web. We propose a block-based volume rendering method to accommodate the independent resource characteristics of web-based rendering, and introduce a Level-of-Detail strategy combined with dynamic loading/unloading of resources to significantly reduce memory demands. Our system achieves real-time rendering of large-scale scenes at 32FPS with RTX 3060 GPU on the web and maintains quality comparable to the current state-of-the-art novel view synthesis methods. Project page: https://ustc3dv.github.io/City-on-Web/.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Barron, J.T., Mildenhall, B., Verbin, D., Srinivasan, P.P., Hedman, P.: Mip-nerf 360: unbounded anti-aliased neural radiance fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5470–5479 (2022)

    Google Scholar 

  2. Cao, J., et al.: Real-time neural light field on mobile devices. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8328–8337 (2023)

    Google Scholar 

  3. Charbonnier, P., Blanc-Féraud, L., Aubert, G., Barlaud, M.: Deterministic edge-preserving regularization in computed imaging. IEEE Trans. Image Process. 6(2), 298–311 (1997)

    Article  Google Scholar 

  4. Chen, A., Xu, Z., Geiger, A., Yu, J., Su, H.: TensoRF: tensorial radiance fields. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision - ECCV 2022, ECCV 2022, LNCS, vol. 13692, pp. 333–350. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19824-3_20

  5. Chen, Z., Funkhouser, T., Hedman, P., Tagliasacchi, A.: Mobilenerf: exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16569–16578 (2023)

    Google Scholar 

  6. Clark, J.H.: Hierarchical geometric models for visible surface algorithms. Commun. ACM 19(10), 547–554 (1976)

    Article  Google Scholar 

  7. Crassin, C., Neyret, F., Lefebvre, S., Eisemann, E.: Gigavoxels: ray-guided streaming for efficient and detailed voxel rendering. In: Proceedings of the 2009 Symposium on Interactive 3D Graphics and Games, pp. 15–22 (2009)

    Google Scholar 

  8. Duchaineau, M., Wolinsky, M., Sigeti, D.E., Miller, M.C., Aldrich, C., Mineev-Weinstein, M.B.: Roaming terrain: real-time optimally adapting meshes. In: Proceedings. Visualization’97 (Cat. No. 97CB36155), pp. 81–88. IEEE (1997)

    Google Scholar 

  9. Gao, X., Zhong, C., Xiang, J., Hong, Y., Guo, Y., Zhang, J.: Reconstructing personalized semantic facial nerf models from monocular video. ACM Trans. Graphics (TOG) 41(6), 1–12 (2022)

    Article  Google Scholar 

  10. Garbin, S.J., Kowalski, M., Johnson, M., Shotton, J., Valentin, J.: Fastnerf: high-fidelity neural rendering at 200fps. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14346–14355 (2021)

    Google Scholar 

  11. Gu, J., et al.: Ue4-nerf: neural radiance field for real-time rendering of large-scale scene. In: Advances in Neural Information Processing Systems, vol. 36 (2024)

    Google Scholar 

  12. Guo, J., et al.: Streetsurf: extending multi-view implicit surface reconstruction to street views. arXiv preprint arXiv:2306.04988 (2023)

  13. Guo, Y., Chen, K., Liang, S., Liu, Y.J., Bao, H., Zhang, J.: Ad-nerf: audio driven neural radiance fields for talking head synthesis. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5784–5794 (2021)

    Google Scholar 

  14. Guthe, S., Wand, M., Gonser, J., Straßer, W.: Interactive rendering of large volume data sets. In: IEEE Visualization, 2002. VIS 2002, pp. 53–60. IEEE (2002)

    Google Scholar 

  15. Hedman, P., Srinivasan, P.P., Mildenhall, B., Barron, J.T., Debevec, P.: Baking neural radiance fields for real-time view synthesis. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5875–5884 (2021)

    Google Scholar 

  16. Hong, Y., Peng, B., Xiao, H., Liu, L., Zhang, J.: Headnerf: a real-time nerf-based parametric head model. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20374–20384 (2022)

    Google Scholar 

  17. Hoppe, H.: Progressive meshes. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 99–108. SIGGRAPH 1996, Association for Computing Machinery, New York, NY, USA (1996). https://doi.org/10.1145/237170.237216

  18. Hu, W., et al.: TTri-MipRF: Tri-Mip representation for efficient anti-aliasing neural radiance fields. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 19774–19783 (2023)

    Google Scholar 

  19. Jiang, B., Hong, Y., Bao, H., Zhang, J.: Selfrecon: self reconstruction your digital avatar from monocular video. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5605–5615 (2022)

    Google Scholar 

  20. Kerbl, B., Kopanas, G., Leimkühler, T., Drettakis, G.: 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph. 42(4) (2023). https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/

  21. Li, Y., Jiang, L., Xu, L., Xiangli, Y., Wang, Z., Lin, D., Dai, B.: Matrixcity: a large-scale city dataset for city-scale neural rendering and beyond. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3205–3215 (2023)

    Google Scholar 

  22. Lindstrom, P., Pascucci, V.: Visualization of large terrains made easy. In: Proceedings Visualization, 2001. VIS’01, pp. 363–574. IEEE (2001)

    Google Scholar 

  23. Liu, L., Gu, J., Zaw Lin, K., Chua, T.S., Theobalt, C.: Neural sparse voxel fields. Adv. Neural. Inf. Process. Syst. 33, 15651–15663 (2020)

    Google Scholar 

  24. Losasso, F., Hoppe, H.: Geometry clipmaps: terrain rendering using nested regular grids. In: ACM Siggraph 2004 Papers, pp. 769–776 (2004)

    Google Scholar 

  25. Luebke, D.: Level of detail for 3D graphics. Morgan Kaufmann, Burlington (2003)

    Google Scholar 

  26. Martin-Brualla, R., Radwan, N., Sajjadi, M.S., Barron, J.T., Dosovitskiy, A., Duckworth, D.: Nerf in the wild: neural radiance fields for unconstrained photo collections. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7210–7219 (2021)

    Google Scholar 

  27. Müller, T., Evans, A., Schied, C., Keller, A.: Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graphics (ToG) 41(4), 1–15 (2022)

    Article  Google Scholar 

  28. Peng, B., Hu, J., Zhou, J., Gao, X., Zhang, J.: Intrinsicngp: intrinsic coordinate based hash encoding for human nerf. IEEE Trans. Vis. Comput. Graph. 30, 5679–5692 (2023)

    Article  Google Scholar 

  29. Reiser, C., Peng, S., Liao, Y., Geiger, A.: Kilonerf: speeding up neural radiance fields with thousands of tiny mlps. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14335–14345 (2021)

    Google Scholar 

  30. Reiser, C., et al.: Merf: memory-efficient radiance fields for real-time view synthesis in unbounded scenes. ACM Trans. Graph. (TOG) 42(4), 1–12 (2023)

    Article  Google Scholar 

  31. Sun, C., Sun, M., Chen, H.T.: Direct voxel grid optimization: super-fast convergence for radiance fields reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5459–5469 (2022)

    Google Scholar 

  32. Takikawa, T., et al.: Variable bitrate neural fields. In: ACM SIGGRAPH 2022 Conference Proceedings. SIGGRAPH ’22, Association for Computing Machinery (2022)

    Google Scholar 

  33. Takikawa, T., et al.: Neural geometric level of detail: real-time rendering with implicit 3d shapes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11358–11367 (2021)

    Google Scholar 

  34. Tancik, M., et al.: Block-nerf: scalable large scene neural view synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8248–8258 (2022)

    Google Scholar 

  35. Tang, J., et al.: Delicate textured mesh recovery from nerf via adaptive surface refinement. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2023)

    Google Scholar 

  36. Turki, H., et al.: Hybridnerf: efficient neural rendering via adaptive volumetric surfaces. In: Computer Vision and Pattern Recognition (CVPR) (2024)

    Google Scholar 

  37. Turki, H., Ramanan, D., Satyanarayanan, M.: Mega-nerf: scalable construction of large-scale nerfs for virtual fly-throughs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12922–12931 (2022)

    Google Scholar 

  38. Wang, P., et al.: F2-nerf: fast neural radiance field training with free camera trajectories. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4150–4159 (2023)

    Google Scholar 

  39. Wang, Z., et al.: Adaptive shells for efficient neural radiance field rendering. ACM Trans. Graph. 42(6) (2023). https://doi.org/10.1145/3618390

  40. Xiang, J., Gao, X., Guo, Y., Zhang, J.: Flashavatar: high-fidelity digital avatar rendering at 300fps. arXiv preprint arXiv:2312.02214 (2023)

  41. Xiangli, Y., et al.: BungeeNeRF: progressive neural radiance field for extreme multi-scale scene rendering. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision - ECCV 2022, ECCV 2022, LNCS, vol. 13692, pp. 106–122. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19824-3_7

  42. Xie, Z., et al.: S3im: stochastic structural similarity and its unreasonable effectiveness for neural fields. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 18024–18034 (2023)

    Google Scholar 

  43. Xu, L., et al.: Grid-guided neural radiance fields for large urban scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8296–8306 (2023)

    Google Scholar 

  44. Yariv, L., et al.: Bakedsdf: Meshing neural SDFs for real-time view synthesis. In: ACM SIGGRAPH 2023 Conference Proceedings, SIGGRAPH 2023, Los Angeles, CA, USA, 6–10 August 2023, pp. 46:1–46:9. ACM (2023)

    Google Scholar 

  45. Yu, A., Li, R., Tancik, M., Li, H., Ng, R., Kanazawa, A.: Plenoctrees for real-time rendering of neural radiance fields. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5752–5761 (2021)

    Google Scholar 

  46. Zeng, X., Song, K., Yang, L., Deng, B., Zhang, J.: Oblique-merf: Revisiting and improving merf for oblique photography. arXiv preprint arXiv:2404.09531 (2024)

  47. Zhang, K., Riegler, G., Snavely, N., Koltun, V.: Nerf++: analyzing and improving neural radiance fields. arXiv preprint arXiv:2010.07492 (2020)

  48. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–595 (2018)

    Google Scholar 

  49. Zhenxing, M., Xu, D.: Switch-nerf: learning scene decomposition with mixture of experts for large-scale neural radiance fields. In: The Eleventh International Conference on Learning Representations (2022)

    Google Scholar 

  50. Zhuang, Y., et al.: Anti-aliased neural implicit surfaces with encoding level of detail. arXiv preprint arXiv:2309.10336 (2023)

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (No.62122071, No.62272433), and the Fundamental Research Funds for the Central Universities (No. WK3470000021). The numerical calculations in this paper have been done on the supercomputing system in the Supercomputing Center of University of Science and Technology of China.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Juyong Zhang .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 10351 KB)

Supplementary material 2 (mp4 82706 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Song, K., Zeng, X., Ren, C., Zhang, J. (2025). City-on-Web: Real-Time Neural Rendering of Large-Scale Scenes on the Web. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15105. Springer, Cham. https://doi.org/10.1007/978-3-031-72970-6_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-72970-6_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-72969-0

  • Online ISBN: 978-3-031-72970-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics