Abstract
Because the light field camera can capture both the position and direction of light simultaneously, it enables us to estimate the depth map from a single light field image and subsequently obtain the 3D point cloud structure. However, the reconstruction results based on light field depth estimation often contain holes and noisy points, which hampers the clarity of the reconstructed 3D object structure. In this paper, we propose a depth optimization algorithm to achieve a more accurate depth map. We introduce a depth confidence metric based on the photo consistency of the refocused angular sampling image. By utilizing this confidence metric, we detect the outlier points in the depth map and generate an outlier mask map. Finally, we optimize the depth map using the proposed energy function. Experimental results demonstrate the superiority of our method compared to other algorithms, particularly in addressing issues related to holes, boundaries, and noise.
This work is supported by National Key Research and Development Project Grant, Grant/Award Number: 2018AAA0100802.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Farhood, H., Perry, S., Cheng, E., Kim, J.: Enhanced 3d point cloud from a light field image. Remote Sens. 12(7), 1125 (2020)
Galea, C., Guillemot, C.: Denoising of 3d point clouds constructed from light fields. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1882–1886. IEEE (2019)
Han, D., Jiao, Z., Zhou, L., Ding, C., Wu, Y.: Geometric constraints based 3d reconstruction method of tomographic sar for buildings. Sci. China Inf. Sci. 66(1), 1–13 (2023)
Han, K., Xiang, W., Wang, E., Huang, T.: A novel occlusion-aware vote cost for light field depth estimation. IEEE Trans. Pattern Anal. Mach. Intell., 1–1 (2021)
Honauer, K., Johannsen, O., Kondermann, D., Goldluecke, B.: A dataset and evaluation methodology for depth estimation on 4d light fields. In: Asian Conference on Computer Vision (2016)
Hua, S., Liu, Q., Yin, G., Guan, X., Jiang, N., Zhang, Y.: Research on 3d medical image surface reconstruction based on data mining and machine learning. Int. J. Intell. Syst. 37(8), 4654–4669 (2022)
Kim, C., Zimmer, H., Pritch, Y., Sorkine-Hornung, A., Gross, M.: Scene reconstruction from high spatio-angular resolution light fields. ACM Trans. Graph. 32(4), 1 (2013)
Peng, J., Xiong, Z., Zhang, Y., Liu, D., Wu, F.: Lf-fusion: dense and accurate 3d reconstruction from light field images. In: 2017 IEEE Visual Communications and Image Processing (VCIP), pp. 1–4. IEEE (2017)
Perra, C., Murgia, F., Giusto, D.: An analysis of 3d point cloud reconstruction from light field images. In: 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA), pp. 1–6. IEEE (2016)
Raj, A.S., Lowney, M., Shah, R., Wetzstein, G.: Stanford light field archives (2016). http://lightfields.stanford.edu/
Ren, N., Levoy, M., Bredif, M., Duval, G., Hanrahan, P.: Light field photography with a hand-held plenoptic camera. Stanford University Cstr (2005)
Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: International Conference on Computer Vision (2002)
Tsai, Y.J., Liu, Y.L., Ouhyoung, M., Chuang, Y.Y.: Attention-based view selection networks for light-field disparity estimation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12095–12103 (2020)
Wang, T.C., Efros, A.A., Ramamoorthi, R.: Depth estimation with occlusion modeling using light-field cameras. IEEE Trans. Pattern Anal. Mach. Intell. 38(11), 2170–2181 (2016)
Wang, Y., Wang, L., Liang, Z., Yang, J., An, W., Guo, Y.: Occlusion-aware cost constructor for light field depth estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19809–19818 (June 2022)
Wanner, S., Meister, S., Goldluecke, B.: Datasets and benchmarks for densely sampled 4d light fields. In: Vision, Modeling and Visualization, pp. 225–226 (2013)
Zhang, L., Liu, L., Chai, B., Xu, M., Song, Y.: Multi-resolution 3d reconstruction of cultural landscape heritage based on cloud computing and hd image data. J. Intell. Fuzzy Syst. 39(4), 5097–5107 (2020)
Zhang, S., Sheng, H., Li, C., Zhang, J., Xiong, Z.: Robust depth estimation for light field via spinning parallelogram operator. Comput. Vis. Image Underst. 145, 148–159 (2016)
Zhao, H., Liu, Y., Wei, L., Wang, Y.: Superpixel-based optimization for point cloud reconstruction from light field. In: 2022 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp. 1–6. IEEE (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, X., Chao, W., Duan, F. (2024). Depth Optimization for Accurate 3D Reconstruction from Light Field Images. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14426. Springer, Singapore. https://doi.org/10.1007/978-981-99-8432-9_7
Download citation
DOI: https://doi.org/10.1007/978-981-99-8432-9_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8431-2
Online ISBN: 978-981-99-8432-9
eBook Packages: Computer ScienceComputer Science (R0)