skip to main content
10.1145/3025453.3025610acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Exploring the Design Space of AAC Awareness Displays

Published: 02 May 2017 Publication History

Abstract

Augmentative and alternative communication (AAC) devices are a critical technology for people with disabilities that affect their speech. One challenge with AAC systems is their inability to portray aspects of nonverbal communication that typically accent, complement, regulate, or substitute for verbal speech. In this paper, we explore the design space of awareness displays that can supplement AAC devices, considering their output features and their effects on the perceptions of interlocutors. Through designing prototypes and getting feedback on our designs from people with ALS, their primary caregivers, and other communication partners, we consider (1) the consistent tensions that arose between abstractness and clarity in meaning for these designs and (2) the ways in which these designs can further mark users as "other." Overall, we contribute a generative understanding of designing AAC awareness displays to augment and contextualize communication.

Supplementary Material

ZIP File (pn1762-file4.zip)
suppl.mov (pn1762-file3.mp4)
Supplemental video

References

[1]
Sudhanshu S.D.P. Ayyagari, Kunal Gupta, Matt Tait, and Mark Billinghurst. 2015. CoSense: Creating Shared Emotional Experiences. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15). ACM, NY, NY, USA, 2007--2012.
[2]
Kevin Baker, Saul Greenberg, and Carl Gutwin. 2001. Heuristic Evaluation of Groupware Based on the Mechanics of Collaboration. In Engineering for HumanComputer Interaction, Murray Reed Little and Laurence Nigay (eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 123--139.
[3]
Laura J. Ball, Amy S. Nordness, Susan Fager, Katie Kersch, Brianae Mohr, Gary L. Pattee, and David Beukelman. 2010. Eye-gaze access of AAC technology for persons with amyotrophic lateral sclerosis. Journal of Medical Speech-Language Pathology 18, 3 (2010), 11--23.
[4]
Susan Baxter, Pam Enderby, Philippa Evans, and Simon Judge. 2012. Barriers and facilitators to the use of high-technology augmentative and alternative communication devices: a systematic review and qualitative synthesis. International Journal of Language & Communication Disorders 47, 2 (March 2012), 115--129.
[5]
David Beukelman, Pat Mirenda, Kathryn Garrett, and Janice Light. 2012. Augmentative and Alternative Communication: Supporting Children and Adults with Complex Communication Needs, Fourth Edition. Paul H. Brookes Publishing Co, Baltimore.
[6]
LouAnne E. Boyd, Alejandro Rangel, Helen Tomimbang, Andrea Conejo-Toledo, Kanika Patel, Monica Tentori, and Gillian R. Hayes. 2016. SayWAT: Augmenting Face-to-Face Conversations for Adults with Autism. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 4872--4883.
[7]
Marco Caligari, Marco Godi, Simone Guglielmetti, Franco Franchignoni, and Antonio Nardone. 2013. Eye tracking communication devices in amyotrophic lateral sclerosis: Impact on disability and quality of life. Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration 14, 7--8 (July 2013), 546--552.
[8]
Toby Churchill. Lightwriter® SL40. Retrieved September 10, 2016 from http://www.toby-churchill.com/products/lightwriter-sl40/.
[9]
Felecia Davis, Asta Roseway, Erin Carroll, and Mary Czerwinski. 2013. Actuating mood: design of the textile mirror. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (TEI '13). ACM, NY, NY, USA, 99106.
[10]
Shelly Farnham, Harry R. Chesley, Debbie E. McGhee, Reena Kawal, and Jennifer Landau. 2000. Structured online interactions: improving the decision-making of small discussion groups. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (CSCW '00). ACM, NY, NY, USA, 299308.
[11]
Jessica Feuston. 2015. Expressiv: An Expressive System for Individuals with Facial Paralysis. MS Thesis. GA Institute of Technology, Atlanta, GA.
[12]
Alexander Fiannaca, Ann Paradiso, Mira Shah, and Meredith Ringel Morris. 2017. AACrobat: Using mobile devices to lower communication barriers and provide autonomy with gaze-based AAC. To appear in Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work.
[13]
Christopher Frauenberger. 2015. Disability and Technology: A Critical Realist Perspective. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility (ASSETS '15). ACM, NY, NY, USA, 89--96.
[14]
William Gaver. 2012. What should we expect from research through design? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, NY, NY, USA, 937--946.
[15]
Katya Hill. 2010. Advances in augmentative and alternative communication as quality-of-life technology. Physical Medicine and Rehabilitation Clinics of North America 21, 1 (February 2010), 43--58.
[16]
Ellen Isaacs, Alan Walendowski, and Dipti Ranganthan. 2002. Hubbub: a sound-enhanced mobile instant messenger that supports awareness and opportunistic interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '02). ACM, NY, NY, USA, 179--186.
[17]
Aura Kagan, Joanne Winckel, Sandra Black, Judith Felson Duchan, Nina Simmons-Mackie & Paula Square. 2004. A Set of Observational Measures for Rating Support and Participation in Conversation Between Adults with Aphasia and Their Conversation Partners. Topics in Stroke Rehabilitation 11, 1 (January 2004), 67--83.
[18]
Shaun K. Kane, Chandrika Jayant, Jacob O. Wobbrock, and Richard E. Ladner. 2009. Freedom to roam: a study of mobile device adoption and accessibility for people with visual and motor disabilities. In Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility (Assets '09). ACM, NY, NY, USA, 115--122.
[19]
Shaun K. Kane, Meredith Ringel Morris, Ann Paradiso, and Jon Campbell. 2017. "At times avuncular and cantankerous, with the reflexes of a mongoose": Understanding self-expression through augmentative and alternative communication devices. To appear in Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work.
[20]
Sreekar Krishna and Sethuraman Panchanathan. 2010. Assistive technologies as effective mediators in interpersonal social interactions for persons with visual disability. In International Conference on Computers for Handicapped Persons, 316--323.
[21]
Per Ola Kristensson and Keith Vertanen. 2012. The potential of dwell-free eye-typing for fast assistive gaze communication. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA '12), Stephen N. Spencer (Ed.). ACM, NY, NY, USA, 241--244.
[22]
Andrew Kurauchi, Wenxin Feng, Ajjen Joshi, Carlos Morimoto, and Margrit Betke. 2016. EyeSwipe: Dwellfree text entry using gaze paths. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 1952--1956.
[23]
Drew Leder. 1990. The Absent Body. University of Chicago Press.
[24]
Daniel G. Lee, Deborah I. Fels, and John Patrick Udo. 2007. Emotive captioning. Computers in Entertainment (CIE) Interactive TV. 5, 2 (August 2007), 1--15.
[25]
Min Kyung Lee and Leila Takayama. 2011. "Now, I have a body": Uses and social norms for mobile remote presence in the workplace. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, NY, NY, USA, 33--42.
[26]
Mandy Leung, Martin Tomitsch, and Andrew Vande Moere. 2011. Designing a personal visualization projection of online social identity. In CHI '11 Extended Abstracts on Human Factors in Computing Systems (CHI EA '11). ACM, NY, NY, USA, 18431848.
[27]
Diana MacLean, Asta Roseway, and Mary Czerwinski. 2013. MoodWings: A wearable biofeedback device for real-time stress intervention. In Proceedings of the 6th International Conference on Pervasive Technologies Related to Assistive Environments (PETRA '13). ACM, NY, NY, USA.
[28]
Päivi Majaranta, Ulla-Kaija Ahola, and Oleg Špakov. 2009. Fast gaze typing with an adjustable dwell time. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, NY, NY, USA, 357--360.
[29]
Pamela A. McCombe and Robert D. Henderson. 2010. Effects of gender in amyotrophic lateral sclerosis. Gender Medicine 7, 6 (December 2010), 557--570.
[30]
Troy McDaniel, Shantanu Bala, Jacob Rosenthal, Ramin Tadayon, Arash Tadayon, and Sethuraman Panchanathan. 2014. Affective haptics for enhancing access to social interactions for individuals who are blind. In International Conference on Universal Access in Human-Computer Interaction, 419--429.
[31]
Hiroshi Mitsumoto. 2009. Amyotrophic Lateral Sclerosis: A Guide for Patients and Families. Demos Medical Publishing.
[32]
Gökhan Mura. 2008. Wearable Technologies for Emotion Communication. Middle East Technical University (METU) Journal of the Faculty of Architecture. 25, 1 (April 2008), 153--161.
[33]
Joan Murphy. 2004. "I Prefer Contact This Close": Perceptions of AAC by People with Motor Neurone Disease and their Communication Partners. Augmentative and Alternative Communication 20, 4 (January 2004), 259--271.
[34]
Michael Oliver. 2009. Understanding Disability: From Theory to Practice. London: Palgrave Macmillan.
[35]
Andrew Ortony and Terence J. Turner. 1990. What's basic about basic emotions? Psychological Review 97, 3 (July 1990), 315--331.
[36]
Alex F. Osborn. Applied imagination: Principles and procedures of creative problem-solving. Scribners.
[37]
Joohee Park, Young-Woo Park, and Tek-Jin Nam. 2014. Wrigglo: shape-changing peripheral for interpersonal mobile communication. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 39733976.
[38]
Diogo Pedrosa, Maria da Graça Pimentel, and Khai N. Truong. 2015. Filteryedping: A dwell-free eye typing technique. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15). ACM, NY, NY, USA, 303--306.
[39]
Robert Plutchik. 1980. Emotion: A Psychoevolutionary Synthesis. Harper & Row NY.
[40]
Jonathan Posner, James A. Russell, and Bradley S. Peterson. 2005. The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and psychopathology 17, 3 (September 2005), 715--734.
[41]
Halley Profita, Reem Albaghli, Leah Findlater, Paul Jaeger, and Shaun K. Kane. 2016. The AT Effect: How Disability Affects the Perceived Social Acceptability of Head-Mounted Display Use. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 4884--4895.
[42]
Graham Pullin. 2009. Design meets disability. MIT press.
[43]
Joshua Rader, Troy McDaniel, Artemio Ramirez Jr, Shantanu Bala, and Sethuraman Panchanathan. 2014. A Wizard of Oz Study Exploring How Agreement/Disagreement Nonverbal Cues Enhance Social Interactions for Individuals Who Are Blind. In HCI International 2014 -- Posters' Extended Abstracts, 243--248.
[44]
A. K. M. Mahbubur Rahm, A. S. M. Iftekhar Anam, and Mohammed Yeasin. EmoAssist: emotion enabled assistive tool to enhance dyadic conversation for the blind. Multimedia Tools and Applications, 1--32.
[45]
Irene Rae, Leila Takayama, and Bilge Mutlu. 2013. Inbody experiences: embodiment, control, and trust in robot-mediated communication. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 19211930.
[46]
Ehud Reiter, Ross Turner, Norman Alm, Rolf Black, Martin Dempster, and Annalu Waller. 2009. Using NLG to help language-impaired users tell stories and participate in social dialogues. In Proceedings of the 12th European Workshop on Natural Language Generation (ENLG'09). Association for Computational Linguistics, 1--8.
[47]
Jocelyn Scheirer and Rosalind Picard. 2000. Affective objects. MIT Media Lab Technical Report 524.
[48]
Jerry C. Schnepp, Rosalee J. Wolfe, John C. McDonald, and Jorge A. Toro. 2012. Combining emotion and facial nonmanual signals in synthesized American sign language. In Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility (ASSETS '12). ACM, NY, NY, USA, 249--250.
[49]
Tom Shakespeare. 1994. Cultural representation of disabled people: Dustbins for disavowal? Disability & Society 9, 3 (January 1994), 283--299.
[50]
Kristen Shinohara and Jacob O. Wobbrock. 2011. In the shadow of misperception: assistive technology use and social interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11), 705--714.
[51]
David Sirkin, Gina Venolia, John Tang, George Robertson, Taemie Kim, Kori Inkpen, Mara Sedlins, Bongshin Lee, and Mike Sinclair. 2011. Motion and attention in a kinetic videoconferencing proxy. In IFIP Conference on Human-Computer Interaction. Springer Berlin Heidelberg, 162--180.
[52]
Anselm Strauss and Juliet Corbin. 1998. Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage Publications, Inc.
[53]
Hornbæk. 2016. Happy Moves, Sad Grooves: Using Theories of Biological Motion and Affect to Design Shape-Changing Interfaces. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (DIS '16). ACM, NY, NY, USA, 12821293.
[54]
John C. Tang. 2007. Approaching and leave-taking: Negotiating contact in computer-mediated communication. ACM Transactions on Computer-Human Interaction 14, 1 (May 2007), 1--26.
[55]
John Todman. 2000. Rate and quality of conversations using a text-storage AAC system: Single-case training study. Augmentative and Alternative Communication 16, 3 (January 2000), 164--179.
[56]
Minh Hong Tran, Yun Yang, and Gitesh K. Raikundalia. 2005. Supporting awareness in instant messaging: an empirical study and mechanism design. In Proceedings of the 17th Australia conference on Computer-Human Interaction: Citizens Online: Considerations for Today and the Future (OZCHI '05). Computer-Human Interaction Special Interest Group (CHISIG) of Australia, Narrabundah, Australia, Australia, 1--10.
[57]
Minh Hong Tran, Yun Yang, and Gitesh K. Raikundalia. 2009. Conversational Awareness in TextBased Computer Mediated Communication. In Awareness Systems, 313--333. Springer London.
[58]
Outi Tuisku, Päivi Majaranta, Poika Isokoski, and Kari-Jouko Räihä. 2008. Now Dasher! Dash away!: Longitudinal study of fast text entry by eye gaze. In Proceedings of the 2008 Symposium on Eye Tracking Research & Applications (ETRA '08). ACM, NY, NY, SUA, 19--26.
[59]
Christophe Veaux, Junichi Yamagishi, and Simon King. 2013. Towards personalized synthesized voices for individuals with vocal disabilities: Voice banking and reconstruction. In 4th Workshop on Speech and Language Processing for Assistive Technologies (SPLAT 2013). Association for Computational Linguistics, 107--111.
[60]
Angela Vujic, Thad Starner, and Melody Jackson. 2016. MoodLens: towards improving nonverbal emotional expression with an in-lens fiber optic display. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC '16). ACM, NY, NY, USA, 36--39.
[61]
Annalu Waller, Rolf Black, David A. O'Mara, Helen Pain, Graeme Ritchie, and Ruli Manurung. 2009. Evaluating the STANDUP pun generating software with children with cerebral palsy. ACM Transactions on Accessible Computing (TACCESS) 1, 3 (February 2009), 1--27.
[62]
Peter Washington, Catalin Voss, Nick Haber, Serena Tanaka, Jena Daniels, Carl Feinstein, Terry Wino-grad, and Dennis Wall. 2016. A Wearable Social Interaction Aid for Children with Autism. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16). ACM, NY, NY, USA, 2348--2354.
[63]
Mary Wickenden. 2011. Whose voice is that?: Issues of identity, voice and representation arising in an ethnographic study of the lives of disabled teenagers who use augmentative and alternative communication (AAC). Disability Studies Quarterly 31, 4 (October 2011).
[64]
Amanda Williams, Shelly Farnham, and Scott Counts. 2006. Exploring wearable ambient displays for social awareness. In CHI '06 Extended Abstracts on Human Factors in Computing Systems (CHI EA '06). ACM, NY, NY, USA, 1529--1534.
[65]
Michele A. Williams, Asta Roseway, Chris O'Dowd, Mary Czerwinski, and Meredith Ringel Morris. 2015. SWARM: An Actuated Wearable for Mediating Affect. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '15). ACM, NY, NY, USA, 293--300.
[66]
Bruce Wisenburn and D. Jeffery Higginbotham. 2009. Participant evaluations of rate and communication efficacy of an AAC application using natural language processing. Augmentative and Alternative Communication 25, 2 (January 2009), 78--89.
[67]
Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The aligned rank transform for nonparametric factorial analyses using only anova procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, NY, NY, USA, 143--146.
[68]
Junichi Yamagishi, Christophe Veaux, Simon King, and Steve Renals. 2012. Speech synthesis technologies for individuals with vocal disabilities: Voice banking and reconstruction. Acoustical Science and Technology 33, 1 (2012), 1--5.

Cited By

View all
  • (2025)Design and development of contextualized, parent-mediated communication boards for learners with down syndromeInternational Journal of Research Studies in Education10.5861/ijrse.2025.2585014:5Online publication date: 20-Feb-2025
  • (2024)Exploring Potential of Electromyography-based Avatar Operation Using Residual Muscles of ALS Individuals: Case Study on Avatar DJ PerformanceExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650857(1-7)Online publication date: 11-May-2024
  • (2024)COMPA: Using Conversation Context to Achieve Common Ground in AACProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642762(1-18)Online publication date: 11-May-2024
  • Show More Cited By

Index Terms

  1. Exploring the Design Space of AAC Awareness Displays

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
    May 2017
    7138 pages
    ISBN:9781450346559
    DOI:10.1145/3025453
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 02 May 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Badges

    • Honorable Mention

    Author Tags

    1. aac
    2. accessibility
    3. als
    4. awareness displays
    5. conversational awareness
    6. conversational flow
    7. disability.
    8. emotion expression
    9. nonverbal communication

    Qualifiers

    • Research-article

    Conference

    CHI '17
    Sponsor:

    Acceptance Rates

    CHI '17 Paper Acceptance Rate 600 of 2,400 submissions, 25%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)107
    • Downloads (Last 6 weeks)4
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Design and development of contextualized, parent-mediated communication boards for learners with down syndromeInternational Journal of Research Studies in Education10.5861/ijrse.2025.2585014:5Online publication date: 20-Feb-2025
    • (2024)Exploring Potential of Electromyography-based Avatar Operation Using Residual Muscles of ALS Individuals: Case Study on Avatar DJ PerformanceExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650857(1-7)Online publication date: 11-May-2024
    • (2024)COMPA: Using Conversation Context to Achieve Common Ground in AACProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642762(1-18)Online publication date: 11-May-2024
    • (2024)Breaking Badge: Augmenting Communication with Wearable AAC Smartbadges and DisplaysProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642327(1-25)Online publication date: 11-May-2024
    • (2024)Beyond Repairing with Electronic Speech: Towards Embodied Communication and Assistive TechnologyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642274(1-12)Online publication date: 11-May-2024
    • (2024)An Emotion Translator: Speculative Design By Neurodiverse DyadsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642210(1-18)Online publication date: 11-May-2024
    • (2024)Aligned Co-Design: An Interdependent, Multi-Modal Method for People with Motor and Communication DisabilitiesInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2436038(1-19)Online publication date: 10-Dec-2024
    • (2024)HoloAAC: A Mixed Reality AAC Application for People with Expressive Language DifficultiesVirtual, Augmented and Mixed Reality10.1007/978-3-031-61047-9_20(304-324)Online publication date: 1-Jun-2024
    • (2023)Nonverbal Communication through Expressive ObjectsCommunications of the ACM10.1145/361093967:1(123-131)Online publication date: 21-Dec-2023
    • (2023)Watch Your Language: Using Smartwatches to Support CommunicationProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3608379(1-21)Online publication date: 22-Oct-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media