Abstract
Much of our daily learning is done through visual information. Visual information is an indispensable part of our life and tends to convey a lot more details than either speech or text. A visual portrayal of a story is generally more appealing and convincing. It is also useful in a variety of applications, such as an accident/crime scene analysis, education and treatment of various psychological or mental disorders like Post-Traumatic Stress Disorder (PTSD). Some individuals develop PTSD due to their exposure to some dangerous or shocking life experience, such as military conflict, physical or sexual assault, traffic or fire accident, natural disasters, etc. People suffering from PTSD can be treated using Virtual Reality Exposure Therapy (VRET), where they are immersed in a virtual environment to face feared situations that may not be safe to encounter in real life. In addition, generated 3D scenes can also be used as a visual aid for teaching children. Since crating 3D context and scenarios for such situations is tedious, time-consuming and requires special expertise in 3D application development environments and software, there is a need for automatic 3D scene generation systems from simple text descriptions. In this paper, we present a new framework for creating 3D scenes from a user-provided simple text. This proposed framework allows us to incorporate motion as well as special effects into the created scenes. In particular, the framework extracts the objects and entities that are present in a given textual narrative as well as spatial relationships. Depending on the description, it then creates either a 3D scene or a 3D scene with corresponding animation. This framework allows creation of a visualization using a set of pre-existing objects using \(Autodesk Maya ^{\textregistered }\) as an implementation environment.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Anagnostopoulos, C.N., Anagnostopoulos, I., Loumos, V., Kayafas, E.: The primary care PTSD screen (PC-PTSD): development and operating characteristics. Primary Care Psychiatry 9(1), 9–14 (2003)
Chang, A.X., Eric, M., Savva, M., Manning, C.D.: SceneSeer: 3D scene design with natural language. ArXiv abs/1703.00050 (2017)
Chang, A.X., Savva, M., Manning, C.D.: Interactive learning of spatial knowledge for text to 3D scene generation. In: Proceedings of the Workshop on Interactive Language Learning, Visualization, and Interfaces, pp. 14–21 (2014)
Clay, S.R., Wilhelms, J.P.: Put: language-based interactive manipulation of objects. IEEE Comput. Graph. Appl. 16(2), 31–39 (1996)
Coyne, B., Sproat, R.: WordsEye: an automatic text-to-scene conversion system. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 487–496 (2001)
Dupuy, S., Egges, A., Legendre, V., Nugues, P.: Generating a 3D simulation of a car accident from a written description in natural language: the CarSim system. In: Proceedings of the Workshop on Temporal and Spatial Information Processing, vol. 13, pp. 1–8 (2001)
Glass, K.R., Bangay, S.: Automating the creation of 3D animation from annotated fiction text. In: Proceedings of the IADIS International Conference on Computer Graphics and Visualization, pp. 3–10 (2008)
Li, C., Yin, C., Lu, J., Ma, L.: Automatic 3D scene generation based on Maya. In: Proceedings of 2009 IEEE 10th International Conference on Computer-Aided Industrial Design Conceptual Design, pp. 981–985 (2009)
Loper, E., Bird, S.: NLTK: the natural language toolkit. ArXiv arXiv preprint cs/0205028 (2002)
Lu, J., Li, C., Yin, C., Ma, L.: A new framework for automatic 3D scene construction from text description. In: Proceedings of the 2010 IEEE International Conference on Progress in Informatics and Computing, vol. 2, pp. 964–968 (2010)
Lu, R., Zhang, S.: Automatic generation of computer animation: using AI for movie animation (2002)
Oshita, M.: Generating animation from natural language texts and semantic analysis for motion search and scheduling. Vis. Comput. 26(5), 339–352 (2010)
Rizzo, A., et al.: A virtual reality exposure therapy application for Iraq war military personnel with post traumatic stress disorder: from training to toy to treatment. In: NATO Advanced Research Workshop on Novel Approaches to the Diagnosis and Treatment of Posttraumatic Stress Disorder (2006)
Roy, M.J., Rizzo, A., Difede, J., Rothbaum, B.O.: Virtual reality exposure therapy for PTSD (2016)
Ulinski, M., Coyne, B., Hirschberg, J.: Evaluating the WordsEye text-to-scene system: imaginative and realistic sentences. In: Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May 2018
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ahmad, I.S., Kadiyala, H., Boufama, B. (2021). From a Textual Narrative to a Visual Story. In: Djeddi, C., Kessentini, Y., Siddiqi, I., Jmaiel, M. (eds) Pattern Recognition and Artificial Intelligence. MedPRAI 2020. Communications in Computer and Information Science, vol 1322. Springer, Cham. https://doi.org/10.1007/978-3-030-71804-6_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-71804-6_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-71803-9
Online ISBN: 978-3-030-71804-6
eBook Packages: Computer ScienceComputer Science (R0)