Jiaxian Wu
Skip Abstract Section
Skip Supplemental Material SectionAbstract
The target of the occluder is to use very few faces to maintain similar occlusion properties of the original 3D model. In this paper, we present DR-Occluder, a novel coarse-to-fine framework for occluder generation that leverages differentiable rendering to optimize a triangle set to an occluder. Unlike prior work, which has not utilized differentiable rendering for this task, our approach provides the ability to optimize a 3D shape to defined targets. Given a 3D model as input, our method first projects it to silhouette images, which are then processed by a convolution network to output a group of vertex offsets. These offsets are used to transform a group of distributed triangles into a preliminary occluder, which is further optimized by differentiable rendering. Finally, triangles whose area is smaller than a threshold are removed to obtain the final occluder. Our extensive experiments demonstrate that DR-Occluder significantly outperforms state-of-the-art methods in terms of occlusion quality. Furthermore, we compare the performance of our method with other approaches in a commercial engine, providing compelling evidence of its effectiveness.
Supplemental Material
- Timo Aila and Ville Miettinen. 2004. dpvs: An occlusion culling system for massive dynamic environments. IEEE Computer graphics and Applications 24, 2 (2004), 86--97.Google Scholar
- John M Airey, John H Rohlf, and Frederick P Brooks Jr. 1990. Towards image realism with interactive update rates in complex virtual building environments. ACM SIGGRAPH computer graphics 24, 2 (1990), 41--50.Google Scholar
- Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, and Michael J Black. 2016. Keep it SMPL: Automatic estimation of 3D human pose and shape from a single image. In European conference on computer vision. Springer, 561--578.Google ScholarCross Ref
- Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. 2015. ShapeNet: An Information-Rich 3D Model Repository. Technical Report arXiv:1512.03012 [cs.GR]. Stanford University --- Princeton University --- Toyota Technological Institute at Chicago.Google Scholar
- Wenzheng Chen, Huan Ling, Jun Gao, Edward Smith, Jaakko Lehtinen, Alec Jacobson, and Sanja Fidler. 2019. Learning to predict 3d objects with an interpolation-based differentiable renderer. Advances in Neural Information Processing Systems 32 (2019).Google Scholar
- Paolo Cignoni, Claudio Montani, and Roberto Scopigno. 1998. A comparison of mesh simplification algorithms. Computers & Graphics 22, 1 (1998), 37--54.Google ScholarCross Ref
- Jonathan Cohen, Amitabh Varshney, Dinesh Manocha, Greg Turk, Hans Weber, Pankaj Agarwal, Frederick Brooks, and William Wright. 1996. Simplification envelopes. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. 119--128.Google ScholarDigital Library
- N Darnell. 2011. Automated occluders for GPU culling. Game Developer Magazine (Sep. 2011) (2011).Google Scholar
- Epic Games. 2023. Visibility and Occlusion Culling. https://docs.unrealengine.com/5.1/en-US/visibility-and-occlusion-culling-in-unreal-engine/Google Scholar
- Michael Garland and Paul S Heckbert. 1997. Surface simplification using quadric error metrics. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques. 209--216.Google ScholarDigital Library
- Michael Garland and Paul S Heckbert. 1998. Simplifying surfaces with color and texture using quadric error metrics. In Proceedings Visualization'98 (Cat. No. 98CB36276). IEEE, 263--269.Google ScholarCross Ref
- Kyle Genova, Forrester Cole, Aaron Maschinot, Aaron Sarna, Daniel Vlasic, and William T Freeman. 2018. Unsupervised training for 3d morphable model regression. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8377--8386.Google ScholarCross Ref
- Rick Germs and Frederik W Jansen. 2001. Geometric simplification for efficient occlusion culling in urban scenes. (2001).Google Scholar
- Jon Hasselgren, Jacob Munkberg, Jaakko Lehtinen, Miika Aittala, and Samuli Laine. 2021. Appearance-Driven Automatic 3D Model Simplification. In Eurographics Symposium on Rendering.Google Scholar
- Paul Henderson and Vittorio Ferrari. 2020. Learning single-image 3d reconstruction by generative modelling of shape, pose and shading. International Journal of Computer Vision 128, 4 (2020), 835--854.Google ScholarDigital Library
- Hugues Hoppe. 1996. Progressive meshes. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. 99--108.Google ScholarDigital Library
- Hugues Hoppe. 1999. New quadric metric for simplifying meshes with appearance attributes. In Proceedings Visualization'99 (Cat. No. 99CB37067). IEEE, 59--510.Google ScholarCross Ref
- Hiroharu Kato, Deniz Beker, Mihai Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, and Adrien Gaidon. 2020. Differentiable rendering: A survey. arXiv preprint arXiv:2006.12057 (2020).Google Scholar
- Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada. 2018. Neural 3d mesh renderer. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3907--3916.Google ScholarCross Ref
- Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- Leif Kobbelt, Swen Campagna, and Hans-Peter Seidel. 1998. A general framework for mesh decimation. In Graphics interface, Vol. 98. Citeseer, 43--50.Google Scholar
- Vladlen Koltun, Yiorgos Chrysanthou, and Daniel Cohen-Or. 2000. Virtual occluders: An efficient intermediate pvs representation. In Eurographics Workshop on Rendering Techniques. Springer, 59--70.Google ScholarCross Ref
- Kiefer Kuah. 2016. Software Occlusion Culling. https://www.intel.com/content/www/us/en/developer/articles/technical/software-occlusion-culling.htmlGoogle Scholar
- Stanford Computer Graphics Laboratory. 2005. The stanford 3d scanning repository. http://www-graphics.stanford.edu/data/3Dscanrep/Google Scholar
- Fei-Ah Law and Tiow-Seng Tan. 1999. Preprocessing occlusion for real-time selective refinement. In Proceedings of the 1999 symposium on Interactive 3D graphics. 47--53.Google ScholarDigital Library
- Xueting Li, Sifei Liu, Kihwan Kim, Shalini De Mello, Varun Jampani, Ming-Hsuan Yang, and Jan Kautz. 2020. Self-supervised single-view 3d reconstruction via semantic consistency. In European Conference on Computer Vision. Springer, 677--693.Google ScholarDigital Library
- Peter Lindstrom. 2000. Out-of-core simplification of large polygonal models. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques. 259--262.Google ScholarDigital Library
- Peter Lindstrom and Greg Turk. 2000. Image-driven simplification. ACM Transactions on Graphics (ToG) 19, 3 (2000), 204--241.Google ScholarDigital Library
- Shichen Liu, Weikai Chen, Tianye Li, and Hao Li. 2019. Soft rasterizer: Differentiable rendering for unsupervised single-view mesh reconstruction. arXiv preprint arXiv:1901.05567 (2019).Google Scholar
- Songrun Liu, Zachary Ferguson, Alec Jacobson, and Yotam I Gingold. 2017. Seamless: seam erasure and seam-aware decoupling of shape from mesh resolution. ACM Trans. Graph. 36, 6 (2017), 216--1.Google ScholarDigital Library
- Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. 2022. A ConvNet for the 2020s. arXiv preprint arXiv:2201.03545 (2022).Google Scholar
- Matthew M Loper and Michael J Black. 2014. OpenDR: An approximate differentiable renderer. In European Conference on Computer Vision. Springer, 154--169.Google ScholarCross Ref
- Kok-Lim Low and Tiow-Seng Tan. 1997. Model simplification using vertex-clustering. In Proceedings of the 1997 symposium on Interactive 3D graphics. 75--ff.Google ScholarDigital Library
- David Luebke, Martin Reddy, Jonathan D Cohen, Amitabh Varshney, Benjamin Watson, and Robert Huebner. 2003. Level of detail for 3D graphics. Morgan Kaufmann.Google Scholar
- David P Luebke. 2001. A developer's survey of polygonal simplification algorithms. IEEE Computer Graphics and Applications 21, 3 (2001), 24--35.Google ScholarDigital Library
- Jacob Norris. 2015. Modular Building Set. https://www.unrealengine.com/marketplace/zh-CN/product/modular-building-setGoogle Scholar
- Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32 (2019).Google Scholar
- Georgios Pavlakos, Luyang Zhu, Xiaowei Zhou, and Kostas Daniilidis. 2018. Learning to estimate 3D human pose and shape from a single color image. In Proceedings of the IEEE conference on computer vision and pattern recognition. 459--468.Google ScholarCross Ref
- Felix Petersen, Amit H Bermano, Oliver Deussen, and Daniel Cohen-Or. 2019. Pix2vex: Image-to-geometry reconstruction using a smooth differentiable renderer. arXiv preprint arXiv:1903.11149 (2019).Google Scholar
- Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. 2020. Accelerating 3d deep learning with pytorch3d. arXiv preprint arXiv:2007.08501 (2020).Google Scholar
- David Salinas, Florent Lafarge, and Pierre Alliez. 2015. Structure-aware mesh decimation. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 211--227.Google Scholar
- Manolis Savva, Angel X. Chang, and Pat Hanrahan. 2015. Semantically-Enriched 3D Models for Common-sense Knowledge. CVPR 2015 Workshop on Functionality, Physics, Intentionality and Causality (2015).Google ScholarCross Ref
- William J Schroeder. 1997. A topology modifying progressive decimation algorithm. In Proceedings. Visualization'97 (Cat. No. 97CB36155). IEEE, 205--212.Google ScholarCross Ref
- William J Schroeder, Jonathan A Zarge, and William E Lorensen. 1992. Decimation of triangle meshes. In Proceedings of the 19th annual conference on Computer graphics and interactive techniques. 65--70.Google ScholarDigital Library
- Ari Silvennoinen, Hannu Saransaari, Samuli Laine, and Jaakko Lehtinen. 2014. Occluder simplification using planar sections. In Computer Graphics Forum, Vol. 33. Wiley Online Library, 235--245.Google Scholar
- Jerry O Talton. 2004. A short survey of mesh simplification algorithms. University of Illinois at Urbana-Champaign (2004).Google Scholar
- Gino van den Bergen. 2021. Conservative mesh decimation for collision detection and occlusion culling. In Game Developers Conference (GDC), Vol. 3.Google Scholar
- Kui Wu, Xu He, Zherong Pan, and Xifeng Gao. 2022. Occluder Generation for Buildings in Digital Games. (2022).Google Scholar
- Xiong Zhang, Qiang Li, Hong Mo, Wenbo Zhang, and Wen Zheng. 2019. End-to-end hand mesh recovery from a monocular rgb image. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 2354--2364.Google ScholarCross Ref
- Qingnan Zhou and Alec Jacobson. 2016. Thingi10k: A dataset of 10,000 3d-printing models. arXiv preprint arXiv:1605.04797 (2016).Google Scholar
Index Terms
- DR-Occluder: Generating Occluders Using Differentiable Rendering
Recommendations
Reconstructing Translucent Objects using Differentiable Rendering
SIGGRAPH '22: ACM SIGGRAPH 2022 Conference ProceedingsInverse rendering is a powerful approach to modeling objects from photographs, and we extend previous techniques to handle translucent materials that exhibit subsurface scattering. Representing translucency using a heterogeneous bidirectional ...
Reparameterizing discontinuous integrands for differentiable rendering
Differentiable rendering has recently opened the door to a number of challenging inverse problems involving photorealistic images, such as computational material design and scattering-aware reconstruction of geometry and materials from photographs. ...
Path-space differentiable rendering
Physics-based differentiable rendering, the estimation of derivatives of radiometric measures with respect to arbitrary scene parameters, has a diverse array of applications from solving analysis-by-synthesis problems to training machine learning ...
Comments