skip to main content
10.1145/3477087.3478382acmconferencesArticle/Chapter ViewAbstractPublication PagesmobicomConference Proceedingsconference-collections
short-paper

EDGY: on-device paralinguistic privacy protection

Published: 25 October 2021 Publication History

Abstract

Voice user interfaces and assistants are rapidly entering our lives and becoming singular touchpoints spanning our devices. Raw audio signals collected through these devices contain a host of sensitive paralinguistic information (e.g., emotional patterns) that is transmitted to service providers regardless of deliberate or false triggers. We thus encounter a new generation of privacy risks by using these services. To tackle this issue, we have developed EDGY; a configurable, lightweight, disentangled representation learning framework that transforms and filters high-dimensional voice data to identify and selectively filter sensitive attributes at the edge prior to offloading to the cloud. Our results show that EDGY runs in tens of milliseconds with 0.2% relative improvement in ABX score and minimal performance penalties in learning linguistic representations from raw signals on a CPU and single-core ARM processor with no specialized hardware.

References

[1]
Ranya Aloufi, Hamed Haddadi, and David Boyle. Configurable Privacy-Preserving Automatic Speech Recognition. In INTERSPEECH 2021.
[2]
Ranya Aloufi, Hamed Haddadi, and David Boyle. Privacy-Preserving Voice Analysis via Disentangled Representations. In CCSW'20.
[3]
Ranya Aloufi, Hamed Haddadi, and David Boyle. 2020. Paralinguistic Privacy Protection at the Edge. (2020). [arxiv]2011.02930
[4]
Huafeng Jin and Shuo Wang. 2018. Voice-based determination of physical and emotional characteristics of users. (2018).
[5]
Alexander Kozlov, Ivan Lazarevich, Vasily Shamporov, Nikolay Lyalyushkin, and Yury Gorbachev. 2021. Neural Network Compression Framework for Fast Model Inference. In Intelligent Computing.
[6]
Tu Anh Nguyen, Maureen de Seyssel, Patricia Rozé, Morgane Rivière, Evgeny Kharitonov, Alexei Baevski, Ewan Dunbar, and Emmanuel Dupoux. The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modeling. [arxiv]2011.11588
[7]
Jacob Peplinski, Joel Shor, Sachin Joglekar, Jake Garrison, and Shwetak Patel. 2020. FUN! Fast, Universal, Non-Semantic Speech Embeddings. (2020). [arxiv]cs.SD/2011.04609
[8]
Björn Schuller, Stefan Steidl, Anton Batliner, Felix Burkhardt, Laurence Devillers, Christian MüLler, and Shrikanth Narayanan. 2013. Paralinguistics in speech and language—State-of-the-art and the challenge. Computer Speech & Language 27, 1 (2013), 4–39.
[9]
Congzheng Song and Vitaly Shmatikov. Overlearning Reveals Sensitive Attributes. In ICLR'20.
[10]
Benjamin van Niekerk, Leanne Nortje, and Herman Kamper. 2020. Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challenge. (2020). [arxiv]2005.09409

Cited By

View all
  • (2022)Bringing Energy into Utility-Privacy Tradeoff in IoT2022 IEEE International Conference on Smart Computing (SMARTCOMP)10.1109/SMARTCOMP55677.2022.00031(116-123)Online publication date: Jun-2022
  • (2022)A survey of privacy-preserving offloading methods in mobile-edge computingJournal of Network and Computer Applications10.1016/j.jnca.2022.103395203:COnline publication date: 1-Jul-2022

Index Terms

  1. EDGY: on-device paralinguistic privacy protection

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    S3 '21: Proceedings of the 12th ACM Wireless of the Students, by the Students, and for the Students (S3) Workshop
    October 2021
    12 pages
    ISBN:9781450387040
    DOI:10.1145/3477087
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 October 2021

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. disentanglement
    2. model optimization
    3. privacy
    4. speech analysis
    5. voice user interface

    Qualifiers

    • Short-paper

    Conference

    ACM MobiCom '21
    Sponsor:

    Acceptance Rates

    S3 '21 Paper Acceptance Rate 6 of 8 submissions, 75%;
    Overall Acceptance Rate 65 of 93 submissions, 70%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)25
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)Bringing Energy into Utility-Privacy Tradeoff in IoT2022 IEEE International Conference on Smart Computing (SMARTCOMP)10.1109/SMARTCOMP55677.2022.00031(116-123)Online publication date: Jun-2022
    • (2022)A survey of privacy-preserving offloading methods in mobile-edge computingJournal of Network and Computer Applications10.1016/j.jnca.2022.103395203:COnline publication date: 1-Jul-2022

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media