demonstration

GetWild: A VR Editing System with AI-Generated 3D Object and Terrain

Authors:

Min-Chun HuAuthors Info & Claims

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Pages 6988 - 6990

https://doi.org/10.1145/3503161.3547733

Published: 10 October 2022 Publication History

Get Access

Abstract

3D environment artists typically use 2D screens and 3D modeling software to achieve their creation. However, creating 3D content using 2D tools is counterintuitive. Moreover, the process would be inefficient for junior artists in the absence of a reference. We develop a system called GetWild, which employs artificial intelligence (AI) models to generate the prototype of 3D objects/terrain and allows users to further edit the generated content in the virtual space. With the aid of AI, the user can capture an image to obtain a rough 3D object model, or start with drawing simple sketches representing the river, the mountain peak and the mountain ridge to create a 3D terrain prototype. Further, the virtual reality (VR) technique is used to provide an immersive design environment and intuitive interaction (such as painting, sculpturing, coloring, and transformation) for users to edit the generated prototypes. Compared with the existing 3D modeling software and systems, the proposed VR editing system with AI-generated 3D objects/terrain provides a more efficient way for the user to create virtual artwork.

References

[1]

Chien-Wen Chen, Jain-Wei Peng, Chia-Ming Kuo, Min-Chun Hu, and Yuan-Chi Tseng. 2018. Ontlus: 3d content collaborative creation via virtual reality. In International Conference on Multimedia Modeling. Springer, 386--389.

Crossref

Google Scholar

[2]

Éric Guérin, Julie Digne, Eric Galin, Adrien Peytavie, Christian Wolf, Bedrich Benes, and Benoît Martinez. 2017. Interactive example-based terrain authoring with conditional generative adversarial networks. Acm Transactions on Graphics (TOG) 36, 6 (2017), 1--13.

Digital Library

Google Scholar

[3]

Eric Guérin. 2022. Deep Terrains -- Code and data | Page perso -- Eric Guérin. http://web.archive.org/web/20210228103649/https://perso.liris.cnrs.fr/eguerin/new/blog/deep-terrains-code-and-data/

Crossref

Google Scholar

[4]

Johann Habakuk Israel, Eva Wiese, Magdalena Mateescu, Christian Zöllner, and Rainer Stark. 2009. Investigating three-dimensional sketching for early conceptual design-Results from expert discussions and user studies. Computers & Graphics 33, 4 (2009), 462--473.

Digital Library

Google Scholar

[5]

Google LLC. 2021. Blocks - Create 3D models in VR - Google VR. http://web.archive.org/web/20220423181749/https://arvr.google.com/blocks/

Google Scholar

[6]

Google LLC. 2021. Tilt Brush by Google. http://web.archive.org/web/20220502202135/https://www.tiltbrush.com/

Google Scholar

[7]

Tariq S Mujber, Tamas Szecsi, and Mohammed SJ Hashmi. 2004. Virtual reality applications in manufacturing process simulation. Journal of materials processing technology 155 (2004), 1834--1838.

Crossref

Google Scholar

[8]

Rainer Stark, Johann Habakuk Israel, and Thomas Wöhler. 2010. Towards hybrid modelling environments-Merging desktop-CAD and virtual reality-technologies. CIRP annals 59, 1 (2010), 179--182.

Google Scholar

[9]

Xingyuan Sun, Jiajun Wu, Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Tianfan Xue, Joshua B Tenenbaum, and William T Freeman. 2018. Pix3d: Dataset and methods for single-image 3d shape modeling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2974--2983.

Crossref

Google Scholar

[10]

Oculus VR. 2022. Mixed Reality with Passthrough. http://web.archive.org/web/20220416225425/https://developer.oculus.com/blog/mixed-reality-with-passthrough/

Google Scholar

[11]

Xin Wen, Junsheng Zhou, Yu-Shen Liu, Hua Su, Zhen Dong, and Zhizhong Han. 2022. 3D Shape Reconstruction from 2D Images with Disentangled Attribute Flow. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

Crossref

Google Scholar

[12]

Jiajun Wu, Chengkai Zhang, Tianfan Xue, Bill Freeman, and Josh Tenenbaum. 2016. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. Advances in neural information processing systems 29 (2016).

Google Scholar

Cited By

View all

Hu YWang KShao YPlass JWang ZPerlin K(2024)Generative Terrain Authoring with Mid-air Hand Sketching in Virtual RealityProceedings of the 30th ACM Symposium on Virtual Reality Software and Technology10.1145/3641825.3687736(1-10)Online publication date: 9-Oct-2024
https://dl.acm.org/doi/10.1145/3641825.3687736
Liu PKitson APicard-Deland CCarr MLiu SLc RZhu-Tian C(2024)Virtual Dream Reliving: Exploring Generative AI in Immersive Environment for Dream Re-experiencingExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3644054(1-11)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3644054

Index Terms

GetWild: A VR Editing System with AI-Generated 3D Object and Terrain
1. Computing methodologies
  1. Artificial intelligence
  2. Computer graphics
    1. Graphics systems and interfaces
      1. Virtual reality
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interactive systems and tools

Recommendations

Interactive example-based terrain authoring with conditional generative adversarial networks

Authoring virtual terrains presents a challenge and there is a strong need for authoring tools able to create realistic terrains with simple user-inputs and with high user control. We propose an example-based authoring pipeline that uses a set of ...
In situ editing (EiS) for fulldomes
VRST '17: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology

The creation workflow for fulldome consists today of many back and forth between the immersive space for visualization and desktop computers for editing. The EiS platform is an interactive creation tool that enables editing while being inside the ...
Frequency-based controls for terrain editing
CVMP '14: Proceedings of the 11th European Conference on Visual Media Production

Authoring virtual terrains can be a challenging task. Procedural and stochastic methods for automated terrain generation produce plausible results but lack intuitive control of the terrain features, while data driven methods offer more creative control ...

Comments

Information & Contributors

Information

Published In

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

October 2022

7537 pages

ISBN:9781450392037

DOI:10.1145/3503161

General Chairs:
João Magalhães
NOVA University of Lisbon, Portugal
,
Alberto del Bimbo
University of Florence, Italy
,
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Nicu Sebe
University of Trento, Italy
,
Program Chairs:
Xavier Alameda-Pineda
Inria, Grenoble, France
,
Qin Jin
Renmin University of China, China
,
Vincent Oria
New Jersey Institute of Technology, USA
,
Laura Toni
University College London, UK

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2022

Check for updates

Author Tags

Qualifiers

Demonstration

Conference

MM '22

Sponsor:

SIGMM

MM '22: The 30th ACM International Conference on Multimedia

October 10 - 14, 2022

Lisboa, Portugal

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
284
Total Downloads

Downloads (Last 12 months)90
Downloads (Last 6 weeks)7

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Hu YWang KShao YPlass JWang ZPerlin K(2024)Generative Terrain Authoring with Mid-air Hand Sketching in Virtual RealityProceedings of the 30th ACM Symposium on Virtual Reality Software and Technology10.1145/3641825.3687736(1-10)Online publication date: 9-Oct-2024
https://dl.acm.org/doi/10.1145/3641825.3687736
Liu PKitson APicard-Deland CCarr MLiu SLc RZhu-Tian C(2024)Virtual Dream Reliving: Exploring Generative AI in Immersive Environment for Dream Re-experiencingExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3644054(1-11)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3644054

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Interactive example-based terrain authoring with conditional generative adversarial networks

In situ editing (EiS) for fulldomes

Frequency-based controls for terrain editing

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations