Skip to main content
SpringerLink
Log in

SELFHOOD: An Evolutionary and Interactive Experience Synthesizing Images and Sounds

  • Conference paper
  • First Online:
Book cover Music Technology with Swing (CMMR 2017)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11265))

Included in the following conference series:

Abstract

The SELFHOOD installation was conceived aiming to instigate a reflection on the self through a practical and interactive experience. A representation of each participant is created in a form of a cloud of points and a sound drone, suggesting their selves. The dynamics of the visitors’ movements is sonified in such way that colours and sound textures are fused in a surrounding hexaphonic system. CromaCrono≈, the system for immersive improvisation that produces digitally synthesized sounds in real time, is described. Philosophical concepts concerning notions of the Self are presented. We propose that the notion of Presence can be induced by virtual and/or physical sources of stimulation governed by a number of principles that underlie human experience, creativity, and discovery. The methodological point of view is that the notion of Presence indicates that there are essential inputs for the construction of self-referral agents.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Mura, A., Manzolli, J., Verschure, P. F., Rezazadeh, B., Groux, S. L., Wierenga, S.: re(PER)curso: a mixed reality chronicle. In: Proceedings of the 2008 ACM SIGGRAPH Conference (2008)

    Google Scholar 

  2. Inderbitzin, M., Wierenga, S., Väljama, A., Bernardet, U., Verschure, P.F.M.J.: Social cooperation and competition in the mixed reality space eXperience Induction Machine (XIM). Virtual Reality 13, 153–158 (2009)

    Article  Google Scholar 

  3. Papachristodoulou, P., Betella, A., Manzolli, J., Verschure, P.F.M.J.: Augmenting the navigation of complex data sets using sonification: a case study with BrainX3. In: Virtual Reality (VR), IEEE (2015)

    Google Scholar 

  4. Miranda, E.R., Wanderley, M.M.: New Digital Musical Instruments: Control and Interaction Beyond the Keyboard. A-R Editions, Middleton (2006)

    Google Scholar 

  5. Bernhardt, D., Robinson, P.: Detecting affect from non-stylised body motions. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 59–70. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74889-2_6

    Chapter  Google Scholar 

  6. Bernhardt, D., Robinson, P.: Interactive control of music using emotional body expressions. In: Human Factors in Computing Systems, pp. 3117—3122 (2008)

    Google Scholar 

  7. Castelli, F., Happé, F., Frith, U., Frith, C.: Movement and mind: a functional imaging study of perception and interpretation of complex intentional movement patterns. NeuroImage 12(3), 314–325 (2000)

    Article  Google Scholar 

  8. Reynolds, C.W.: Flocks, herds, and schools: a distributed behavioral model. In: SIGGRAPH Conference Proceedings, vol. 21, pp. 25–34 (1987)

    Article  Google Scholar 

  9. Colton, S., Wiggins G.A.: Computational creativity: the final frontier? In: 20th European Conference on Artificial Intelligence (2012)

    Google Scholar 

  10. Sternberg, R.S. (ed.): Handbook of Creativity. Cambridge University Press, Cambridge (1999)

    Google Scholar 

  11. Pope, R.: Creativity: Theory, History, Practice. Routledge, New York (2005)

    Google Scholar 

  12. Boden, M.: What is creativity? In: Boden, M. (ed.) Dimensions of Creativity, pp. 75—117. MIT Press, Cambridge (2005)

    Google Scholar 

  13. Bernardet, U.: The eXperience induction machine: a new paradigm for mixed-reality interaction design and psychological experimentation. In: Dubois, M., Gray, P., Nigay, L. (eds.) The Engineering of Mixed Reality Systems. HCIS, pp. 357–379. Springer, London (2010). https://doi.org/10.1007/978-1-84882-733-2_18

    Chapter  Google Scholar 

  14. Wasserman, K., Manzolli, J., Eng, K., Verschure, P.F.M.J.: Live soundscape composition based on synthetic emotions: using music to communicate between an interactive exhibition and its visitors. IEEE Multimedia 10, 82–90 (2003)

    Article  Google Scholar 

  15. Prescott, T.J., Verschure, P.F.M.J., Lepora, N.: A Handbook of Research in Biomimetic and Biohybrid Systems. Oxford University Press, Oxford, In Press (To appear)

    Google Scholar 

  16. Strawson, G.: The self. J. Conscious. Stud. 4, 405–428 (1997)

    Google Scholar 

  17. Descartes, R.: The Philosophical Writings of Descartes, vols. 1 and 2, translated by J. Cottingham et al. Cambridge University Press, Cambridge (1985)

    Google Scholar 

  18. Hume, D.: Dialogues Concerning Natural Religion. In: Smith, N.K. (ed.). Nelson, Edinburgh (1947)

    Google Scholar 

  19. Fichte, J.G.: The Science of Knowledge, translated by Heath P. and Lachs, J. CUP, Cambridge (1982)

    Google Scholar 

  20. James, W.: The Principles of Psychology. Dover, New York (1950)

    Google Scholar 

  21. James, W.: Psychology: Briefer Course. Harvard University Press, Cambridge (1984)

    Google Scholar 

  22. Husserl, E.: Cartesian Meditations, translated by D. Cairns. Nijhoff, The Hague (1973)

    Book  Google Scholar 

  23. Ginsberg, A.: Statement to the Burning Bush. Burning Bush II, San Francisco (1963)

    Google Scholar 

  24. Nozick, R.: Philosophical Explanations. Clarendon Press, Oxford (1981)

    Google Scholar 

  25. Kenny, A.: The Self. Marquette University Press, Marquette (1988)

    Google Scholar 

  26. Dennett, D.: Consciousness Explained. Little, Brown, Boston (1991)

    Google Scholar 

  27. Wöllner, C. (ed.): Body, Sound and Space in Music and Beyond: Multimodal Explorations. Routledge, New York (2017)

    Google Scholar 

  28. Krausz, M., Dutton, D., Bardsley, K.: The Idea of Creativity. Brill Academic Publishers, Leiden (2009)

    Google Scholar 

  29. Sanchez-Vives, M.V., Slater, M.: From presence to consciousness through virtual reality. Nat. Rev. Neurosci. 6(4), 332–339 (2005)

    Article  Google Scholar 

  30. Anderson, M.L.: Embodied cognition: a field guide. Artif. Intell. 149(1), 91–130 (2003)

    Article  MathSciNet  Google Scholar 

  31. Moroni, A., Manzolli, J., Valarini, G.: SELFHOOD Installation. https://www.youtube.com/watch?v=lavHYLFXL8w&feature=youtu.be

  32. Moroni, A., Manzolli, J.: SELFHOOD—cloudiness of the selves. https://www.youtube.com/watch?v=9F-s3Cafhc4&feature=youtu.be

  33. Castellano, G., Villalba, S.D., Camurri, A.: Recognising human emotions from body movement and gesture dynamics. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 71–82. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74889-2_7

    Chapter  Google Scholar 

  34. Le Groux, S., Manzolli, J., Verschure, P.F.M.J.: Disembodied and collaborative musical interaction in the Multimodal Brain Orchestra. In: Proceedings of NIME, pp. 309–314 (2010)

    Google Scholar 

  35. Heppner, F.H.: Avian flight formations. Bird-Banding 45(2), 160–169 (1974)

    Article  Google Scholar 

  36. Manzolli, J.: continuaMENTE: Integrating percussion, audiovisual and improvisation. In: Proceedings of the International Computer Music Conference, Belfast, Ireland, (2008)

    Google Scholar 

  37. Moroni, A., Manzolli, J.: From evolutionary composition to robotic sonification. In: Di Chio, C., et al. (eds.) EvoApplications 2010. LNCS, vol. 6025, pp. 401–410. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12242-2_41

    Chapter  Google Scholar 

  38. Moroni, A., Von Zuben, F., Manzolli, J.: ArTbitration: human-machine interaction in artistic domains. Leonardo (Oxford) 35, 185–188 (2002)

    Article  Google Scholar 

  39. Wassermann, K.C., Manzolli, J., Eng, K., Verschure, P.: Live soundscape composition based on synthetic emotions: using music to communicate between an interactive exhibition and its visitors. IEEE Multimedia 10, 82–90 (2003)

    Article  Google Scholar 

  40. Verschure, P., Manzolli, J.: Computational modeling of mind and music. In: Language, Music, and the Brain. ESF Reports, MIT Press (2013)

    Google Scholar 

  41. Manzolli, J.: Multimodal generative installations and the creation of new art form based on interactive narratives. In: GA2015—XVIII Generative Art Conference, (2015). http://www.generativeart.com/ga2015_WEB/Multimodal-Installation_Manzolli.pdf

  42. Manzolli, J.: CromaCrono ≈ . https://vimeo.com/145326063

  43. Reynolds, C.W.: Flocks, herds and schools: a distributed behavioral model. ACM SIGGRAPH Comput. Graph. 21(4), 25–34 (1987)

    Article  Google Scholar 

  44. Karplus, K., Strong, A.: Digital synthesis of plucked string and drum timbres. Comput. Music J. 7(2), 43–55 (1983)

    Article  Google Scholar 

  45. Macedo, L., Cardoso, A., Reisenzein, R., Lorini, E., Castelfranchi C.: Artificial surprise. In: Handbook of Research on Synthetic Emotions Sociable Robotics New Applications in Affective Computing Artificial Intelligence. IGI Global, Hershey, USA (2009)

    Google Scholar 

  46. Paraense, A.L.O., et al.: The cognitive systems toolkit and the CST reference cognitive architecture. Biol. Inspired Cogn. Archit. J. 17, 32–48 (2016)

    Google Scholar 

  47. Gudwin, R., Paraense, A., de Paula, S.M., Fróes, E., Gibaut, W., Castro, E., Raizer, K.: The multipurpose enhanced cognitive architecture (MECA). Biol. Inspired Cogn. Archit. J. 22, 20–34 (2017)

    Google Scholar 

  48. Dourish, P.: Where the Action Is: Foundations of Embodied Interaction. MIT Press, Cambridge (2001)

    Google Scholar 

Download references

Acknowledgments

We thank Thomas Taichi Okubo for his commitment and dedication. We thank Georges Schwarzstein for his careful review and appropriate suggestions. Okubo and Valarini were supported by the PIBIC/CNPq program and Manzolli is supported by projects 470358/2014-9 and 305065/2014-9 also from CNPq. The sensory devices used on SELHOOD installation were provided by project FAPESP 2013/26453-1.

Author information

Authors and Affiliations

  1. IA, NICS/University of Campinas – UNICAMP, Campinas, SP, 13083-970, Brazil

    Jonatas Manzolli

  2. CTI Renato Archer, Dom Pedro Highway (SP-65), Km 143.6 – Amarais-Campinas, Campinas, SP, 13069-901, Brazil

    Artemis Moroni & Guilherme A. Valarini

Authors
  1. Jonatas Manzolli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Artemis Moroni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guilherme A. Valarini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Artemis Moroni .

Editor information

Editors and Affiliations

  1. Laboratoire PRISM, AMU-CNRS, Marseille, France

    Mitsuko Aramaki

  2. INESC TEC, Porto, Portugal

    Matthew E. P. Davies

  3. Laboratoire PRISM, AMU-CNRS, Marseille, France

    Richard Kronland-Martinet

  4. Laboratoire PRISM, AMU-CNRS, Marseille, France

    Sølvi Ystad

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Manzolli, J., Moroni, A., Valarini, G.A. (2018). SELFHOOD: An Evolutionary and Interactive Experience Synthesizing Images and Sounds. In: Aramaki, M., Davies , M., Kronland-Martinet, R., Ystad, S. (eds) Music Technology with Swing. CMMR 2017. Lecture Notes in Computer Science(), vol 11265. Springer, Cham. https://doi.org/10.1007/978-3-030-01692-0_41

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-01692-0_41

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-01691-3

  • Online ISBN: 978-3-030-01692-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation