skip to main content
10.1145/3490100.3516479acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiuiConference Proceedingsconference-collections
extended-abstract

Interactive Intelligent Tools for Creative Processes using Multimodal Information

Published: 22 March 2022 Publication History

Abstract

The individual differences in human behavior, combined with the ubiquity and constant evolution of computing devices, led to different forms of human-computer interaction. Keyboard, touch, voice, or gesture, among others, are used to provide input and receive feedback. However, applications that provide users with multimodal interaction for creative processes are usually agnostic about media content and user profile, unable to highlight significant elements that may be useful to foster creativity. This work's primary goal is to design models and provide techniques to create intelligent interactive tools that use rich multimodal information to support people in creative processes.

References

[1]
Bolt, R.A. 1980. “Put-that-there.” Proceedings of the 7th annual conference on Computer graphics and interactive techniques - SIGGRAPH ’80 (New York, New York, USA, Jul. 1980), 262–270.
[2]
Cabral, D., Valente, J.G., Aragão, U., Fernandes, C. and Correia, N. 2012. Evaluation of a multimodal video annotator for contemporary dance. Proc. of the Workshop on Advanced Visual Interfaces AVI (2012).
[3]
Cohen, P.R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L. and Clow, J. 1997. QuickSet. Proceedings of the fifth ACM international conference on Multimedia - MULTIMEDIA ’97 (New York, New York, USA, Nov. 1997), 31–40.
[4]
DeLahunta, S. and Jenett, F. 2015. Making digital choreographic objects interrelate: a focus on coding practices. 6, (2015).
[5]
Home | Stanford HAI: 2022. https://hai.stanford.edu/. Accessed: 2022-01-08.
[6]
Humane AI | Human-Centered Artificial Intelligence: 2022. https://www.humane-ai.eu/. Accessed: 2022-01-08.
[7]
Li, Y., Abstract, W.S.L., Kumar, R. and Hilliges, O. Artificial Intelligence for HCI: A Modern Approach. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems.
[8]
Mitchell, T.M. 1997. Artificial neural networks. Machine Learning. McGraw Hill.
[9]
Mitrevski, M. 2018. Getting Started with Wit.ai. Developing Conversational Interfaces for iOS. Apress. 143–164.
[10]
ml5js·Friendly Machine Learning For The Web: 2021. https://ml5js.org/. Accessed: 2021-01-25.
[11]
Nilsson, N.J. 1998. Artificial Intelligence: a new synthesis. (1998), 513.
[12]
Oviatt, S. and Cohen, P. 2000. Perceptual user interfaces: multimodal interfaces that process what comes naturally. Communications of the ACM. 43, 3 (Mar. 2000), 45–53.
[13]
Risko, E.F., Foulsham, T., Dawson, S. and Kingstone, A. 2013. The collaborative lecture annotation system (CLAS): A new TOOL for distributed learning. IEEE Transactions on Learning Technologies. 6, 1 (2013), 4–13.
[14]
Rodrigues, R., Madeira, R.N. and Correia, N. 2021. Exploring the User Interaction with a Multimodal Web- Based Video Annotator. 13th EAI International Conference on Intelligent Technologies for Interactive En-tertainment (Intetain 2021) (Qingdao, People's Republic of China, 2021), 1–10.
[15]
Rodrigues, R., Madeira, R.N., Correia, N., Fernandes, C. and Ribeiro, S. 2019. Multimodal Web Based Video Annotator with Real-Time Human Pose Estimation. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 23–30.
[16]
Rui Rodrigues, Rui Neves Madeira, and N.C. 2021. Studying Natural User Interfaces for Smart Video Annotation towards Ubiquitous Environments. 20th International Conference on Mobile and Ubiquitous Multimedia (MUM 2021) (Leuven, Belgium, 2021), 1–18.
[17]
Schmidhuber, J. 2015. Deep learning in neural networks: An overview. Neural Networks. 61, (Jan. 2015), 85–117.
[18]
Schmidt, A. 2020. Interactive Human Centered Artificial Intelligence: A Definition and Research Challenges. PervasiveHealth: Pervasive Computing Technologies for Healthcare. (2020).
[19]
Sebe, N. 2009. Multimodal interfaces: Challenges and perspectives. Journal of Ambient Intelligence and Smart Environments. (2009).
[20]
Singh, V., Latulipe, C., Carroll, E. and Lottridge, D. 2011. The choreographer's notebook-a video annotation system for dancers and choreographers. C and C 2011 - Proceedings of the 8th ACM Conference on Creativity and Cognition (2011).
[21]
Turk, M. 2014. Multimodal interaction: A review. Pattern Recognition Letters. 36, (Jan. 2014), 189–195.
[22]
Wittenburg, P., Brugman, H., Russel, A., Klassmann, A. and Sloetjes, H. 2006. ELAN: A professional framework for multimodality research. Proceedings of the 5th International Conference on Language Resources and Evaluation, LREC 2006 (2006).

Cited By

View all

Index Terms

  1. Interactive Intelligent Tools for Creative Processes using Multimodal Information
      Index terms have been assigned to the content through auto-classification.

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      IUI '22 Companion: Companion Proceedings of the 27th International Conference on Intelligent User Interfaces
      March 2022
      142 pages
      ISBN:9781450391450
      DOI:10.1145/3490100
      Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 22 March 2022

      Check for updates

      Author Tags

      1. Creative Processes
      2. Human-Computer Interaction
      3. Intelligent Tools
      4. Multimodal Data

      Qualifiers

      • Extended-abstract
      • Research
      • Refereed limited

      Conference

      IUI '22
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 746 of 2,811 submissions, 27%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)19
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 27 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media