skip to main content
10.1145/3674225.3674311acmotherconferencesArticle/Chapter ViewAbstractPublication PagespeaiConference Proceedingsconference-collections
research-article

A three-dimensional reconstruction method based on improved Mip-NeRF

Published: 31 July 2024 Publication History

Abstract

Image-based three-dimensional reconstruction technology is a technique used to restore the three-dimensional structure of a target from a two-dimensional image. It is widely applied in virtual reality, cultural preservation, medicine, and other fields. Mip-NeRF is characterized by its high fidelity and multi-scale input processing, and it introduces a multi-resolution representation to improve rendering quality and efficiency. However, the long training time limits its practical applicability. To address the issue of lengthy training time in Mip-NeRF, the study proposes an improved Mip-NeRF method by optimizing the neural network structure and introducing multi-importance sampling techniques. Experimental results demonstrate that this method can maintain high-quality reconstructed models and improve the training speed by 52%, significantly reducing the training time. It offers a new approach to reduce the training time for three-dimensional object reconstruction.

References

[1]
Zheng Taixiong, Huang Shuai, * * yongfu, Review of key technologies of 3 D reconstruction based on vision [J]. Journal of Automation, 2020,46 (4): 631-652.
[2]
Roberts L G. Machine perception of three-dimensional solids[D]. Massachusetts Institute of Technology, 1963.
[3]
Furukawa Y, Hernández C. Multi-view stereo: A tutorial[J]. Foundations and Trends® in Computer Graphics and Vision, 2015, 9(1-2): 1-148.
[4]
Gupta M, Agrawal A, Veeraraghavan A, Structured light 3D scanning in the presence of global illumination[C]//CVPR 2011. IEEE, 2011: 713-720.
[5]
Lazaros N, Sirakoulis G C, Gasteratos A. Review of stereo vision algorithms: from software to hardware[J]. International Journal of Optomechatronics, 2008, 2(4): 435-462.
[6]
Liu Xinyue. Research on the 3 D reconstruction and retrieval algorithm based on deep learning [D]. UESTC [2023-11-18].
[7]
Mildenhall B, Srinivasan P P, Tancik M, NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis[C]//2020.
[8]
Barron J T, Mildenhall B, Tancik M, Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 5855-5864.
[9]
Veach E, Guibas L J. Optimally combining sampling techniques for Monte Carlo rendering[C]//Proceedings of the 22nd annual conference on Computer graphics and interactive techniques. 1995: 419-428.
[10]
Hedman P, Philip J, Price T, Deep blending for free-viewpoint image-based rendering[J]. ACM Transactions on Graphics (ToG), 2018, 37(6): 1-15.
[11]
Mildenhall B, Srinivasan P P, Ortiz-Cayon R, Local light field fusion: Practical view synthesis with prescriptive sampling guidelines[J]. ACM Transactions on Graphics (TOG), 2019, 38(4): 1-14.
[12]
Choi I, Gallo O, Troccoli A, Extreme view synthesis[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 7781-7790.
[13]
Liu L, Gu J, Zaw Lin K, Neural sparse voxel fields[J]. Advances in Neural Information Processing Systems, 2020, 33: 15651-15663.
[14]
Zhang K, Riegler G, Snavely N, Nerf++: Analyzing and improving neural radiance fields[J]. ar**v preprint ar**v:2010.07492, 2020.
[15]
Yu A, Ye V, Tancik M, pixelnerf: Neural radiance fields from one or few images[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 4578-4587.
[16]
Tancik M, Mildenhall B, Wang T, Learned initializations for optimizing coordinate-based neural representations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 2846-2855.
[17]
Li Z, Niklaus S, Snavely N, Neural scene flow fields for space-time view synthesis of dynamic scenes[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 6498-6508.
[18]
Park K, Sinha U, Barron J T, Nerfies: Deformable neural radiance fields[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 5865-5874.
[19]
Yariv L, Gu J, Kasten Y,et al.Volume Rendering of Neural Implicit Surfaces[J]. 2021.
[20]
Liu L, Gu J, Zaw Lin K, Neural sparse voxel fields[J]. Advances in Neural Information Processing Systems, 2020, 33: 15651-15663.
[21]
Chen Z, Funkhouser T, Hedman P, Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 16569-16578.
[22]
Shao R, Zheng Z, Tu H, Tensor4d: Efficient neural 4d decomposition for high-fidelity dynamic reconstruction and rendering[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 16632-16642.
[23]
Reiser C, Szeliski R, Verbin D, Merf: Memory-efficient radiance fields for real-time view synthesis in unbounded scenes[J]. ACM Transactions on Graphics (TOG), 2023, 42(4): 1-12.
[24]
Cao A, Johnson J. Hexplane: A fast representation for dynamic scenes[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 130-141. Xu Q, Xu Z, Philip J, Point-nerf: Point-based neural radiance fields[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 5438-5448.
[25]
Xu Q, Xu Z, Philip J, Point-nerf: Point-based neural radiance fields[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 5438-5448.
[26]
Müller T, Evans A, Schied C, Instant neural graphics primitives with a multiresolution hash encoding[J]. ACM Transactions on Graphics (ToG), 2022, 41(4): 1-15.

Index Terms

  1. A three-dimensional reconstruction method based on improved Mip-NeRF

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    PEAI '24: Proceedings of the 2024 International Conference on Power Electronics and Artificial Intelligence
    January 2024
    969 pages
    ISBN:9798400716638
    DOI:10.1145/3674225
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 31 July 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    PEAI 2024

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 21
      Total Downloads
    • Downloads (Last 12 months)21
    • Downloads (Last 6 weeks)4
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media