skip to main content
10.1145/3450337.3483510acmconferencesArticle/Chapter ViewAbstractPublication Pageschi-playConference Proceedingsconference-collections
extended-abstract

Harmonionz, Rescue The Planet: A Voice Visualizing Game that Match Pitch with Color.

Published: 15 October 2021 Publication History

Abstract

Voice input has been used interactively in games for a long time. Existing voice games showed the potential of voice input games by utilizing various features of voice such as voice volume detection, pitch and rhythm detection, speech recognition, and conversation. However, most of these conventional voice games showed that the main game characteristics depended only on the voice input. In this context, we propose Harmonionz, a two-player collaborative game that goes beyond the limits of the existing voice input game genre. Our game has a novelty in which the voice plays a functional and story-wise core role, but the fun of the game is not limited to the voice. In our game, players can win the game when they explore a colorful map, avoid enemies, and eventually create a pitch required by the target with their voice. Each player has a different role. One defeats the approaching enemies and the other creates the pitch. The target pitch is mapped to the color to indicate players the color of the notes that they need to make. Players must find the required pitch by looking at the color on the target. The fun part of conventional voice input games was the voice itself only. However, voice input in our game can easily interact with overall game design including shooting and team-work. The worldview of our game, colorizing the pitch in order to interact with the colorful map, interacting with the background music are our attempts to make our game contextually and logically persuasive.

Supplementary Material

VTT File (p356-harmonionz_playvideo.vtt)
MP4 File (p356-harmonionz_playvideo.mp4)
Supplemental video and captions

References

[1]
Andrea Corradini, Manish Mehta, Niels Ole Bernsen, J Martin, and Sarkis Abrilian. 2005. Multimodal input fusion in human-computer interaction. NATO Science Series Sub Series III Computer and Systems Sciences 198 (2005), 223.
[2]
Alain De Cheveigné and Hideki Kawahara. 2002. YIN, a fundamental frequency estimator for speech and music. The Journal of the Acoustical Society of America 111, 4 (2002), 1917–1930.
[3]
Tracy Fullerton. 2014. Game design workshop: a playcentric approach to creating innovative games. CRC press.
[4]
William Griscom. 2015. Visualizing Sound: Cross-Modal Mapping Between Music and Color. Ph.D. Dissertation. UC Berkeley.
[5]
Gerhard Hagerer, Florian Eyben, Dagmar Schuller, Klaus R Scherer, and Björn Schuller. 2017. VoicePlay—An affective sports game operated by speech emotion recognition based on the component process model. In 2017 Seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW). IEEE, 74–76.
[6]
Sujin Ko. 2003. Study of color hearing according to sound and pitch. Master’s thesis. Dongguk University.
[7]
P Toprak. 2013. The psychology of control and video games. Ctrl-alt-play: Essays on control in video gaming (2013), 21–33.
[8]
Katharina Von Kriegstein and Anne-Lise Giraud. 2006. Implicit multisensory associations influence voice recognition. PLoS biology 4, 10 (2006), e326.

Cited By

View all
  • (2024)Exploration of Novelty as Part of the Player Experience of Need Satisfaction in GamesInteracting with Computers10.1093/iwc/iwae006Online publication date: 15-Mar-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI PLAY '21: Extended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play
October 2021
414 pages
ISBN:9781450383561
DOI:10.1145/3450337
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2021

Check for updates

Author Tags

  1. Cross modal mapping
  2. Games
  3. Multi-sensory
  4. Voice Visualization
  5. Voice interaction

Qualifiers

  • Extended-abstract
  • Research
  • Refereed limited

Funding Sources

  • Ministry of Culture, Sports and Tourism and Korea Creative Content Agency

Conference

CHI PLAY '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 421 of 1,386 submissions, 30%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)15
  • Downloads (Last 6 weeks)2
Reflects downloads up to 08 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Exploration of Novelty as Part of the Player Experience of Need Satisfaction in GamesInteracting with Computers10.1093/iwc/iwae006Online publication date: 15-Mar-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media