skip to main content
research-article

Broomrocket: Open Source Text-to-3D Algorithm for 3D Object Placement

Published: 30 August 2024 Publication History

Abstract

Story writers and other creative professionals often rely on concept artists to visualize and then further iterate on their work during game development and other visualization processes. This exchange and its various stages are time consuming, and there is no easy remedy for creating a walkable 3D concept art without involving a 3D artist yet. As a first step, we present Broomrocket, an open source text-to-3D algorithm for 3D concept art. Broomrocket’s contribution is an object relation and placement algorithm that transforms user input describing a 3D scene given in plain English language into actual models placed in a 3D scene. It runs locally using an existing downloaded natural language processing model and does not require third-party services unless a connection to an online 3D model distribution platform is desired. In that case, Broomrocket will search for the keywords from the user’s narrative input and desired license, and place them in the 3D scene, adding each model’s individual license to a license file for further usage.

References

[1]
George Anastassakis and Themis Panayiotopoulos. 2012. An architectural pattern for X3D-based virtual environments—An object-oriented approach. In Proceedings of the International Conference on Computer Graphics Theory and Applications and the International Conference on Information Visualization Theory and Applications (VISIGRAPP’12). 466–471. DOI:
[2]
Olavo Da Rosa Belloc, Rodrigo B. D. Ferraz, Marcio Calixto Cabral, Roseli De Deus Lopes, and Marcelo Knörich Zuffo. 2012. Virtual reality procedure training simulators in X3D. In Proceedings of the 17th International Conference on 3D Web Technology (Web3D’12). 153–160. DOI:
[3]
Steven Bird. 2006. NLTK: The natural language toolkit. In Proceedings of the COLING/ACL on Interactive Presentation Sessions (COLING-ACL’06). 69–72. DOI:
[4]
Blender Foundation. 2023. Blender Website. Retrieved June 30, 2023 from https://blender.org
[5]
Epic Games. 2023. Unreal Engine 5: Procedural Content Generation Overview. Retrieved November 11, 2023 from https://docs.unrealengine.com/5.2/en-US/procedural-content-generation-overview/
[6]
ExplosionAI. 2023. SpaCy Website. Retrieved June 30, 2023 from https://spacy.io/
[7]
gd3kr. 2023. Blender GPT. Retrieved November 11, 2023 from https://github.com/gd3kr/BlenderGPT
[8]
Godot Engine. 2023. Godot Website. Retrieved November 1, 2023 from https://godotengine.org/
[9]
GraphN. 2023. Dash. Retrieved November 11, 2023 from https://www.polygonflow.io/
[10]
Young-Seok Hwang, Jung-Sup Um, Biswajeet Pradhan, Tanupriya Choudhury, and Stephan Schlueter. 2023. How does ChatGPT evaluate the value of spatial information in the 4th industrial revolution? Spatial Information Research. Published Online, December 12, 2023. DOI:
[11]
Cristina Improta. 2023. Poisoning programs by un-repairing code: Security concerns of AI-generated code. In Proceedings of the 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshops (ISSREW’23). IEEE, 128–131. DOI:
[12]
Heewoo Jun and Alex Nichol. 2023. Shap-E: Generating conditional 3D implicit functions. arXiv:cs.CV/2305.02463 (2023).
[13]
Fahad Khan, Kashif Irfan, Razzaq Saad, Maqbool Fahad, Farid Ahmad, and Rao Anwer. 2008. Using VRML to build a virtual reality campus environment. In Proceedings of the World Congress on Engineering (WCE’08). 1–5.
[14]
Khronos Group 2023. glTF Overview—The Khronos Group Inc. Retrieved November 1, 2023 from https://www.khronos.org/gltf/
[15]
Zhang Lin, Chen Zhixing, and Zhao Chunxiao. 2010. Research and modeling the ancient architecture system in VRML. In Proceedings of the 2010 International Conference on Computer and Communication Technologies in Agriculture Engineering, Vol. 3. 587–590. DOI:
[16]
Hartmut Luttermann and Manfred Grauer. 1998. VRML-basierte präsentation raum-zeitlicher geschäfts- und wissenschaftsdaten mit WWW-browsern. In Informatik’98. Springer, Berlin, Germany, 57–66.
[17]
Alexey Mazalov, Bruno Martins, and David Matos. 2015. Spatial role labeling with convolutional neural networks. In Proceedings of the 9th Workshop on Geographic Information Retrieval (GIR’15). ACM, Article 12, 7 pages. DOI:
[18]
Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. 2022. Point-E: A system for generating 3D point clouds from complex prompts. arXiv:cs.CV/2212.08751 (2022).
[19]
Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. 2022. DreamFusion: Text-to-3D using 2D diffusion. arXiv:cs.CV/2209.14988 (2022).
[20]
Procedural Worlds. 2023. GeNa Pro. Retrieved November 11, 2023 from https://www.procedural-worlds.com/products/professional/gena-pro/
[21]
Sketchfab. 2023. Sketchfab Website. Retrieved June 30, 2023 from https://sketchfab.com/
[22]
Something Pink. 2024. Broomrocket: Open Source Text to 3D Scene Creation. Retrieved February 9, 2024 from https://something.pink/software/broomrocket/
[23]
VRML Specification. 2023. VRML97. Retrieved June 30, 2023 from https://www.web3d.org/documents/specifications/14772/V2.0/
[24]
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. 2023. ProlificDreamer: High-fidelity and diverse text-to-3D generation with variational score distillation. arXiv:cs.LG/2305.16213 (2023).
[25]
World Machine Software. 2023. World Machine. Retrieved November 11, 2023 from https://www.world-machine.com/
[26]
Hongwei Yi, Chun-Hao P. Huang, Shashank Tripathi, Lea Hering, Justus Thies, and Michael J. Black. 2022. MIME: Human-aware 3D scene generation. arXiv:cs.CV/2212.04360 (2022).
[27]
Livia Ştefan, Sorin Hermon, and Marina Faka. 2018. Prototyping 3D virtual learning environments with X3D-based content and visualization tools. BRAIN: Broad Research in Artificial Intelligence and Neuroscience 9 (2018), 6–20.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Games: Research and Practice
Games: Research and Practice  Volume 2, Issue 3
September 2024
163 pages
EISSN:2832-5516
DOI:10.1145/3613704
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 August 2024
Online AM: 14 February 2024
Accepted: 08 February 2024
Revised: 12 January 2024
Received: 24 July 2023
Published in GAMES Volume 2, Issue 3

Check for updates

Author Tags

  1. Text-to-3D
  2. level design
  3. scene generation
  4. 3D concept art
  5. prototyping
  6. language processing
  7. computing

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 317
    Total Downloads
  • Downloads (Last 12 months)317
  • Downloads (Last 6 weeks)24
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media