Segmentation of 3D Gaussians using Masked Gradients
Article No.: 77, Pages 1 - 2
Abstract
We present a simple yet powerful method for segmenting 3D Gaussian splatting models using 2D segmentation masks from a limited number of viewpoints. Our approach leverages inference-time gradient backpropagation, making it both easy to implement and fast enough for interactive segmentation. The underlying formulation ensures high accuracy, making this method a highly effective tool for 3D segmentation and related downstream tasks such as AR, VR, 3DGS editing, and digital twins.
References
[1]
[2]
Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan. 2024. YOLO-World: Real-Time Open-Vocabulary Object Detection. In Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR).
[3]
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 2023. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Transactions on Graphics 42, 4 (July 2023). https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
[4]
Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. 2023. LangSplat: 3D Language Gaussian Splatting. arXiv preprint arXiv:https://arXiv.org/abs/2312.16084 (2023).
[5]
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer. 2024. SAM 2: Segment Anything in Images and Videos. arXiv preprint arXiv:https://arXiv.org/abs/2408.00714 (2024). https://arxiv.org/abs/2408.00714
[6]
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2019. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. International Journal of Computer Vision 128, 2 (Oct. 2019), 336–359.
[7]
Shijie Zhou, Haoran Chang, Sicheng Jiang, Zhiwen Fan, Zehao Zhu, Dejia Xu, Pradyumna Chari, Suya You, Zhangyang Wang, and Achuta Kadambi. 2024. Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 21676–21685.
[8]
Matthias Zwicker, Hanspeter Pfister, Jeroen van Baar, and Markus Gross. 2001. Surface splatting. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH ’01). Association for Computing Machinery, New York, NY, USA, 371–378.
Index Terms
- Segmentation of 3D Gaussians using Masked Gradients
Recommendations
Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing
Computer Vision – ECCV 2024AbstractIn this paper, we present a novel differentiable point-based rendering framework to achieve photo-realistic relighting. To make the reconstructed scene relightable, we enhance vanilla 3D Gaussians by associating extra properties, including normal ...
Automatic segmentation of bladder and prostate using coupled 3D deformable models
MICCAI'07: Proceedings of the 10th international conference on Medical image computing and computer-assisted intervention - Volume Part IIn this paper, we propose a fully automatic method for the coupled 3D localization and segmentation of lower abdomen structures. We apply it to the joint segmentation of the prostate and bladder in a database of CT scans of the lower abdomen of male ...
3D segmentation of exterior wall surface of abdominal aortic aneurysm from CT images using variable neighborhood search
AbstractA 3D model of abdominal aortic aneurysm (AAA) can provide useful anatomical information for clinical management and simulation. Thin-slice contiguous computed tomographic (CT) angiography is the best source of medical images for ...
Comments
Information & Contributors
Information
Published In
Copyright © 2024 Copyright held by the owner/author(s).
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 02 December 2024
Check for updates
Author Tags
Qualifiers
- Poster
Conference
SA '24
Sponsor:
Acceptance Rates
Overall Acceptance Rate 178 of 869 submissions, 20%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 246Total Downloads
- Downloads (Last 12 months)246
- Downloads (Last 6 weeks)32
Reflects downloads up to 02 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format