research-article

Rhythmic walking interactions with auditory feedback: an exploratory study

Authors:

Stefania Serafin,

Cumhur ErkutAuthors Info & Claims

AM '12: Proceedings of the 7th Audio Mostly Conference: A Conference on Interaction with Sound

Pages 68 - 75

https://doi.org/10.1145/2371456.2371467

Published: 26 September 2012 Publication History

Abstract

Walking is a natural rhythmic activity that has become of interest as a means of interacting with software systems such as computer games. Therefore, designing multimodal walking interactions calls for further examination. This exploratory study presents a system capable of different kinds of interactions based on varying the temporal characteristics of the output, using the sound of human walking as the input. The system either provides a direct synthesis of a walking sound based on the detected amplitude envelope of the user's footstep sounds, or provides a continuous synthetic walking sound as a stimulus for the walking human, either with a fixed tempo or a tempo adapting to the human gait. In a pilot experiment, the different interaction modes are studied with respect to their effect on the walking tempo and the experience of the subjects. The results tentatively outline different user profiles in interacting with such a system.

References

[1]

P. Cook. Physically informed sonic modeling (PhISM): Synthesis of percussive sounds. Computer Music J., 21(3):38--49, Summer 1997.

[2]

P. Cook. Modeling Bill's gait: Analysis and parametric synthesis of walking sounds. In Proc. AES 22nd Int. Conf. Virtual, Synthetic, and Entertainment Audio, pages 73--78, June 2002.

[3]

U. Şimşekli, A. Jylhä, C. Erkut, and A. Cemgil. Real-time recognition of percussive sounds by a model-based method. EURASIP J. Advances in Signal Processing, 2011:1--14, 2011. Special Issue on Musical Applications of Real-Time Signal Processing.

Digital Library

[4]

C. Erkut, A. Jylhä, and D. Rocchesso. Heigh Ho: Rhythmicity in sonic interaction. In S. Serafin and K. Franinovic, editors, Sonic Interaction Design: Fresh Perspectives, chapter 17. The MIT Press, Boston, MA, USA, October 2012. In press.

[5]

A. Farnell. Marching onwards: procedural synthetic footsteps for video games and animation. In Proc. Pure Data Conv., pages 1--5, August 2007.

[6]

F. Fontana and R. Bresin. Physics-based sound synthesis and control: crushing, walking and running by crumpling sounds. In Proc. Colloquium on Musical Informatics, pages 109--114, May 2003.

[7]

E. Ghomi, G. Faure, S. Huot, O. Chapuis, and M. Beaudouin-Lafon. Using rhythmic patterns as an input method. In Proc. Human Factors in Computing Systems (CHI '12), pages 1253--1262, May 2012.

Digital Library

[8]

A. Hazan. Performing expressive rhythms with BillaBoop voice-driven drum generator. In Proc. 7th Intl. Conf. Digital Audio Effects (DAFx), pages 1--4, October 2004.

[9]

T. Howe, B. Lövgreen, F. Cody, V. Ashton, and J. Oldham. Auditory cues can modify the gait of persons with early-stage parkinson's disease: a method for enhancing parkinsonian walking performance? Clinical Rehabilitation, 17(4):363--367, July 2003.

[10]

A. Jylhä. Sonic Gestures and Rhythmic Interaction between the Human and the Computer. PhD thesis, Aalto University School of Electrical Engineering, Espoo, Finland, 2012.

[11]

A. Jylhä, I. Ekman, C. Erkut, and K. Tahiroğlu. Design and evaluation of rhythmic interaction with an interactive tutoring system. Computer Music J., 35(2):36--48, May 2011.

Digital Library

[12]

A. Jylhä, C. Erkut, I. Ekman, and K. Tahiroğlu. iPalmas - An interactive Flamenco rhythm machine. In Proc. Audio Mostly, pages 69--76, September 2009.

[13]

O. A. Kannape and O. Blanke. Agency, gait and self-consciousness. Int. J. Psychophysiology, 83(2):191--199, February 2012.

[14]

O. Lartillot, P. Toiviainen, and T. Eerola. A Matlab toolbox for music information retrieval. In C. Preisach, H. Burkhardt, L. Schmidt-Thieme, and R. Decker, editors, Data Analysis, Machine Learning and Applications, pages 261--268. Springer, Berlin, 2008.

[15]

J. Löwgren. Toward an articulation of interaction aesthetics. New Review of Hypermedia and Multimedia, 15(2):129--146, December 2009.

Digital Library

[16]

F. Menzer, A. Brooks, P. Halje, C. Faller, M. Vetterli, and O. Blanke. Feeling in control of your footsteps: Conscious gait monitoring and the auditory consequences of footsteps. Cognitive Neuroscience, 1(3):184--192, August 2010.

[17]

B. Moens, L. Van Noorden, and M. Leman. D-jogger: Syncing music with walking. In Proc. Sound and Music Computing Conf. (SMC '10), pages 451--456, July 2010.

[18]

R. Nordahl, L. Turchet, and S. Serafin. Sound synthesis and evaluation of interactive footsteps and environmental sounds rendering for virtual reality applications. IEEE Trans. Visualization and Computer Graphics, 17(9):1234--1244, September 2011.

Digital Library

[19]

N. Oliver and L. Kreger-Stickles. Papa: Physiology and purpose-aware automatic playlist generation. In Proc. 7th Int. Conf. Music Inf. Retrieval, pages 250--253, October 2006.

[20]

S. Papetti, F. Fontana, M. Civolani, A. Berrezag, and V. Hayward. Audio-tactile display of ground properties using interactive shoes. In Proc. Int. Conf. Haptic and Audio Interaction Design, pages 117--128. Springer, September 2010.

Digital Library

[21]

L. Peltola, C. Erkut, P. Cook, and V. Välimäki. Synthesis of hand clapping sounds. IEEE Trans. Audio, Speech and Language Processing, 15(3):1021--1029, March 2007.

Digital Library

[22]

S. Prassas, M. Thaut, G. McIntosh, and R. Rice. Effect of auditory rhythmic cuing on gait kinematic parameters of stroke patients. Gait & Posture, 6(3):218--223, December 1997.

[23]

M. Puckette, T. Apel, and D. Zicarelli. Real-time audio analysis tools for pd and msp. In Proc. Int. Computer Music Conf. (ICMC), pages 109--112, January 1998.

[24]

A. Robertson and M. Plumbley. B-Keeper: A beat-tracker for live performance. In Proc. 7th Int. Conf. New Interfaces for Musical Expression (NIME), pages 234--237, June 2007.

Digital Library

[25]

D. Rocchesso, P. Polotti, and S. Delle Monache. Designing continuous sonic interaction. Int. J. Design, 3(3):13--25, December 2009.

[26]

M. Roerdink, P. J. M. Bank, C. L. E. Peper, and P. J. Beek. Walking to the beat of different drums: Practical implications for the use of acoustic rhythms in gait rehabilitation. Gait & Posture, 33(4):690--694, April 2011.

[27]

F. Styns, L. van Noorden, D. Moelants, and M. Leman. Walking on music. Human Movement Science, 26(5):769--785, October 2007.

[28]

M. Thaut. Rhythm, Music, and the Brain. Scientific Foundations and Clinical Applications. Routledge, New York, NY, USA, 2005.

[29]

M. Thaut and M. Abiru. Rhythmic auditory stimulation in rehabilitation of movement disorders: a review of current research. Music Perception, 27(4):263--269, April 2010.

[30]

M. H. Thaut and M. Abiru. Rhythmic auditory stimulation in rehabilitation of movement disorders: A review of current research. Music Perception, 27(4):263--269, April 2010.

[31]

Y. Visell, F. Fontana, B. Giordano, R. Nordahl, S. Serafin, and R. Bresin. Sound design and perception in walking interactions. Int. J. Human-Computer Studies, 67(11):947--959, November 2009.

Digital Library

[32]

G. Weinberg and S. Driscoll. Toward robotic musicianship. Computer Music J., 30(4):28--45, January 2006.

Digital Library

Cited By

Dublin TEzer TZuckerman O(2024)A Journey Inward: The Somaesthetic Experience of a Heated Walking CarpetExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650915(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650915
Tan PRen X(2024)The Effects of Rhythmic Footstep and Sound Interactions on Creativity: A Design and Evaluation StudyInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2378253(1-18)Online publication date: 31-Jul-2024
https://doi.org/10.1080/10447318.2024.2378253
Nguyen BHan JHouben MBayoumi YVande Moere A(2023)Engaging Passers-by with Rhythm: Applying Feedforward Learning to a Xylophonic Media Architecture FacadeProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580761(1-21)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580761
Show More Cited By

Index Terms

Rhythmic walking interactions with auditory feedback: an exploratory study

Recommendations

An investigation of the effect of immersive visual and auditory feedback on rhythmic walking interaction
AM '16: Proceedings of the Audio Mostly 2016

We present a study whose goal is to investigate the role of immersive visual and auditory feedback to affect preferred walking pace and perceived ease and naturalness of walking actions. Additionally, we measured how the congruence between visual and ...
An investigation on the impact of auditory and haptic feedback on rhythmic walking interactions

This paper presents a system capable of rhythmic walking interaction by auditory and haptic display. Likewise, it summarizes the results of the research on the influence of the audio and haptic stimulation on rhythmic walking interaction. The system ...
The Effects of Ecological Auditory Feedback on Rhythmic Walking Interaction
The authors present a system capable of rhythmic walking interactions using auditory display. The feedback is based on footstep sounds and follows either detected footsteps or suggests a tempo, which is either constant or adapts to the walker. The ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AM '12: Proceedings of the 7th Audio Mostly Conference: A Conference on Interaction with Sound

September 2012

174 pages

ISBN:9781450315692

DOI:10.1145/2371456

Conference Chairs:
Andreas Floros
Ionian University, Greece
,
Andreas Mniestris
Ionian University, Greece
,
Program Chairs:
Iani Zannos
Ionian University, Greece
,
Theodoros Lotis
Ionian University, Greece

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 September 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

AM '12

AM '12: A conference on interaction with sound

September 26 - 28, 2012

Corfu, Greece

Acceptance Rates

Overall Acceptance Rate 177 of 275 submissions, 64%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
341
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)6

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dublin TEzer TZuckerman O(2024)A Journey Inward: The Somaesthetic Experience of a Heated Walking CarpetExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650915(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650915
Tan PRen X(2024)The Effects of Rhythmic Footstep and Sound Interactions on Creativity: A Design and Evaluation StudyInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2378253(1-18)Online publication date: 31-Jul-2024
https://doi.org/10.1080/10447318.2024.2378253
Nguyen BHan JHouben MBayoumi YVande Moere A(2023)Engaging Passers-by with Rhythm: Applying Feedforward Learning to a Xylophonic Media Architecture FacadeProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580761(1-21)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580761
Tan PRen X(2023)Rhythm Research in Interactive System Design: A Literature ReviewInternational Journal of Human–Computer Interaction10.1080/10447318.2023.229462841:1(31-50)Online publication date: 27-Dec-2023
https://doi.org/10.1080/10447318.2023.2294628
Cochrane KLoke LLeete MCampbell AAhmadpour NKaltenbrunner MMurer MWolf KOakley I(2021)Understanding the First Person Experience of Walking Mindfulness Meditation Facilitated by EEG Modulated Interactive SoundscapeProceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3430524.3440637(1-17)Online publication date: 14-Feb-2021
https://dl.acm.org/doi/10.1145/3430524.3440637
Maculewicz JJylha ASerafin SErkut C(2015)The Effects of Ecological Auditory Feedback on Rhythmic Walking InteractionIEEE MultiMedia10.1109/MMUL.2015.1722:1(24-31)Online publication date: Jan-2015
https://doi.org/10.1109/MMUL.2015.17
Bonnin GJannach D(2014)Automated Generation of Music Playlists: Survey and ExperimentsACM Computing Surveys10.1145/265248147:2(1-35)Online publication date: 12-Nov-2014
https://dl.acm.org/doi/10.1145/2652481

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten