Skip to main content
Springer Nature Link
Log in

SceneMaker: Automatic Visualisation of Screenplays

  • Conference paper
KI 2009: Advances in Artificial Intelligence (KI 2009)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 5803))

Included in the following conference series:

Abstract

Our proposed software system, SceneMaker, aims to facilitate the production of plays, films or animations by automatically interpreting natural language film scripts and generating multimodal, animated scenes from them. During the generation of the story content, SceneMaker will give particular attention to emotional aspects and their reflection in fluency and manner of actions, body posture, facial expressions, speech, scene composition, timing, lighting, music and camera work. Related literature and software on Natural Language Processing, in particular textual affect sensing, affective embodied agents, visualisation of 3D scenes and digital cinematography are reviewed. In relation to other work, SceneMaker will present a genre-specific text-to-animation methodology which combines all relevant expressive modalities. In conclusion, SceneMaker will enhance the communication of creative ideas providing quick pre-visualisations of scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Ma, M.: Automatic Conversion of Natural Language to 3D Animation. PhD Thesis, School of Computing and Intelligent Systems, University of Ulster (2006)

    Google Scholar 

  2. Ekman, P., Rosenberg, E.L.: What the face reveals: Basic and applied studies of spontaneous expression using the facial action coding system. Oxford University Press, Oxford (1997)

    Google Scholar 

  3. Mehrabian, A.: Framework for a Comprehensive Description and Measurement of Emotional States. Genetic, Social, and General Psychology Monographs. Heldref Publishing 121(3), 339–361 (1995)

    Google Scholar 

  4. Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press, Cambridge (1988)

    Book  Google Scholar 

  5. Francisco, V., Hervás, R., Gervás, P.: Two Different Approaches to Automated Mark Up of Emotions in Text. In: Research and development in intelligent systems XXIII: Proceedings of AI 2006, pp. 101–114. Springer, Heidelberg (2006)

    Google Scholar 

  6. Strapparava, C., Mihalcea, R.: Learning to identify emotions in text. In: Proceedings of the 2008 ACM Symposium on Applied Computing. SAC 2008, pp. 1556–1560. ACM, New York (2008)

    Chapter  Google Scholar 

  7. Su, W.-P., Pham, B., Wardhani, A.: Personality and Emotion-Based High-Level Control of Affective Story Characters. IEEE Transactions on Visualization and Computer Graphics 13(2), 281–293 (2007)

    Article  Google Scholar 

  8. Liu, H., Lieberman, H., Selker, T.: A model of textual affect sensing using real-world knowledge. In: Proceedings of the 8th International Conference on Intelligent User Interfaces. IUI 2003, pp. 125–132. ACM, New York (2003)

    Google Scholar 

  9. Liu, H., Singh, P.: ConceptNet: A practical commonsense reasoning toolkit. BT Technology Journal. Springer Netherlands 22(4), 211–226 (2004)

    Article  Google Scholar 

  10. Shaikh, M.A.M., Prendinger, H., Ishizuka, M.: A Linguistic Interpretation of the OCC Emotion Model for Affect Sensing from Text. In: Affective Information Processing, pp. 45–73. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  11. Choujaa, D., Dulay, N.: Using screenplays as a source of context data. In: Proceeding of the 2nd ACM international Workshop on Story Representation, Mechanism and Context. SRMC 2008, pp. 13–20. ACM, New York (2008)

    Chapter  Google Scholar 

  12. Cassell, J., Vilhjálmsson, H.H., Bickmore, T.: BEAT: the Behavior Expression Animation Toolkit. In: Proceedings of the 28th Annual Conference on Computer Graphics and interactive Techniques. SIGGRAPH 2001, pp. 477–486. ACM, New York (2001)

    Google Scholar 

  13. Breitfuss, W., Prendinger, H., Ishizuka, M.: Automated generation of non-verbal behavior for virtual embodied characters. In: Proceedings of the 9th International Conference on Multimodal Interfaces. ICMI 2007, pp. 319–322. ACM, New York (2007)

    Google Scholar 

  14. Prendinger, H., Ishizuka, M.: SCREAM: scripting emotion-based agent minds. In: Proceedings of the First International Joint Conference on Autonomous Agents and Multiagent Systems: Part 1. AAMAS 2002, pp. 350–351. ACM, New York (2002)

    Chapter  Google Scholar 

  15. Gebhard, P.: ALMA - Layered Model of Affect. In: Proceedings of the 4th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2005), Utrecht University, Netherlands, pp. 29–36. ACM, New York (2005)

    Chapter  Google Scholar 

  16. De Raad, B.: The Big Five Personality Factors. In: The Psycholexical Approach to Personality, Hogrefe & Huber (2000)

    Google Scholar 

  17. Kopp, S., Allwood, J., Grammer, K., Ahlsen, E., Stocksmeier, T.: Modeling Embodied Feedback with Virtual Humans. In: Modeling Communication with Robots and Virtual Humans, pp. 18–37. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  18. Pelachaud, C.: Multimodal expressive embodied conversational agents. In: Proceedings of the 13th Annual ACM International Conference on Multimedia. MULTIMEDIA 2005, pp. 683–689. ACM, New York (2005)

    Google Scholar 

  19. Gunes, H., Piccardi, M.: A Bimodal Face and Body Gesture Database for Automatic Analysis of Human Nonverbal Affective Behavior. In: 18th International Conference on Pattern Recognition, ICPR, vol. 1, pp. 1148–1153. IEEE Computer Society, Washington (2006)

    Google Scholar 

  20. Coyne, B., Sproat, R.: WordsEye: an automatic text-to-scene conversion system. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 487–496. ACM Press, Los Angeles (2001)

    Google Scholar 

  21. Kennedy, K., Mercer, R.E.: Planning animation cinematography and shot structure to communicate theme and mood. In: Proceedings of the 2nd International Symposium on Smart Graphics. SMARTGRAPH 2002, vol. 24, pp. 1–8. ACM, New York (2002)

    Google Scholar 

  22. De Melo, C., Paiva, A.: Multimodal Expression in Virtual Humans. Computer Animation and Virtual Worlds 17(3-4), 239–348 (2006)

    Article  Google Scholar 

  23. Liu, Z., Leung, K.: Script visualization (ScriptViz): a smart system that makes writing fun. Soft Computing 10(1), 34–40 (2006)

    Article  Google Scholar 

  24. Rasheed, Z., Sheikh, Y., Shah, M.: On the use of computable features for film classification. IEEE Transactions on Circuits and Systems for Video Technology. IEEE Circuits and Systems Society 15(1), 52–64 (2005)

    Article  Google Scholar 

  25. Shim, H., Kang, B.G.: CAMEO - camera, audio and motion with emotion orchestration for immersive cinematography. In: Proceedings of the 2008 international Conference on Advances in Computer Entertainment Technology. ACE 2008, vol. 352, pp. 115–118 (2008)

    Google Scholar 

  26. Kuo, F., Chiang, M., Shan, M., Lee, S.: Emotion-based music recommendation by association discovery from film music. In: Proceedings of the 13th Annual ACM international Conference on Multimedia, MULTIMEDIA 2005, pp. 507–510. ACM, New York (2005)

    Google Scholar 

  27. Turner, S.R.: The Creative Process: A Computer Model of Storytelling and Creativity. Lawrence Erlbaum Associates, Hillsdale (1994)

    Google Scholar 

  28. Okada, N., Inui, K., Tokuhisa, M.: Towards affective integration of vision, behavior, and speech processing. In: Proceedings of the Integration of Speech and Image Understanding, SPELMG, pp. 49–77. IEEE Computer Society, Washington (1999)

    Chapter  Google Scholar 

  29. Perlin, K., Goldberg, A.: Improv: a system for scripting interactive actors in virtual worlds. In: Proceedings of the 23rd Annual Conference on Computer Graphics and interactive Techniques, SIGGRAPH 1996, pp. 205–216. ACM, New York (1996)

    Chapter  Google Scholar 

  30. Connexor, http://www.connexor.eu/technology/machinese

  31. Tesniere, L.: Elements de syntaxe structurale. Klincksieck, Paris (1959)

    Google Scholar 

  32. Fellbaum, C.: WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998)

    MATH  Google Scholar 

  33. Lexical Conceptual Structure Database, http://www.umiacs.umd.edu/~bonnie/LCS_Database_Documentation.html

  34. Strapparava, C., Valitutti, A.: WordNet-Affect: an affective extension of WordNet. In: Proceedings of the 4th International Conference on Language Resources and Evaluation, LREC 2004, vol. 4, pp. 1083–1086 (2004)

    Google Scholar 

  35. Humanoid Animation Working Group, http://www.h-anim.org

  36. FreeTTS 1.2 - A speech synthesizer written entirely in the JavaTM programming language: http://freetts.sourceforge.net/docs/index.php

Download references

Author information

Authors and Affiliations

  1. School of Computing & Intelligent Systems Faculty of Computing & Engineering, University of Ulster, Magee, Derry/Londonderry, BT48 7JL, Northern Ireland

    Eva Hanser, Paul Mc Kevitt, Tom Lunney & Joan Condell

Authors
  1. Eva Hanser
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul Mc Kevitt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tom Lunney
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joan Condell
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. GET Lab, University of Paderborn, Pohlweg 47-49, 33098, Paderborn, Germany

    Bärbel Mertsching , Marcus Hund  & Zaheer Aziz ,  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Hanser, E., Mc Kevitt, P., Lunney, T., Condell, J. (2009). SceneMaker: Automatic Visualisation of Screenplays. In: Mertsching, B., Hund, M., Aziz, Z. (eds) KI 2009: Advances in Artificial Intelligence. KI 2009. Lecture Notes in Computer Science(), vol 5803. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04617-9_34

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-04617-9_34

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-04616-2

  • Online ISBN: 978-3-642-04617-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics