Abstract
Currently, many smart speakers, even social robots, appear on the market to help people's lives become more convenient. Usually, people use smart speakers to check their daily schedule or control home appliances in their house. Many social robots also include smart speakers. They have the common property of being used in voice control machines. Regardless of where the smart speaker is installed and used, when people start a conversation with voice equipment, a security or privacy risk is exposed. Hence, we want to build a speech recognition (SR) that contains the privacy identification information (PII) system in this paper. We call this the SR-PII system. We used a Google Artificial-Intelligence-Yourself (AIY) Voice Kit released from Google to build a simple, smart dialog speaker and included our SR-PII system. In our experiments, we test SR accuracy and the reliability of privacy settings in three environments (quiet, noise, and playing music). We also examine the cloud response and speaker response times during our experiments. The results show that the speaker response is approximately 3.74 s in the cloud environment and approximately 9.04 s from the speaker. We also showed the response accuracy of the speaker, which successfully prevented personal information with the SR-PII system in three environments. The speaker has a response mean time of approximately 8.86 s with 93% mean accuracy in a quiet room, approximately 9.18 s with 89% mean accuracy in a noisy environment, and approximately 9.62 s with 90% mean accuracy in an environment that plays music. We conclude that the SR-PII system can secure private information and that the most important factor affecting the response speed of the speaker is the network connection status. We hope that people can, through our experiments, have some guidelines in building social robots and installing the SR-PII system to protect users’ personal identification information.




Similar content being viewed by others
References
Graves A, Jaitly N (2014) Towards end-to-end speech recognition with recurrent neural networks. In: Proceedings of the International Conference on Machine Learning, pp 1764–1772
Dong L, Xu S, Xu B (2018) Speech-transformer: a no-recurrence sequence-to-sequence model for speech recognition. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp 5884–5888
Hinton G (2012) Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Process Mag 29(6):82–97
Xiong W, Wu L, Alleva F, Droppo J, Huang X, Stolcke A (2018) The Microsoft 2017 conversational speech recognition system. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp 5934–5938
Salazar J, Kirchhoff K, Huang Z (2019) Self-attention networks for connectionist temporal classification in speech recognition. In: Proceedings of the ICASSP - IEEE International Conference on Acoustics, Speech and Signal Processing, pp 7115–7119
Haeb-Umbach R, Watanabe R, Nakatani T, Bacchiani M (2019) Speech processing for digital home assistants. IEEE Signal Process Mag 36(6):111–124
Li B, Sainath T, Narayanan A, Caroselli J (2017) Acoustic modeling for Google Home. In: Interspeech, pp 399–403
Barker J, Watanabe S, Vincent E, Tr-mal J (2018) The fifth CHiME speech separation and recognition challenge: Dataset, task, and baselines. In: Interspeech, pp 1561–1565
Yu D, Deng L (2015) Automatic speech recognition a deep learning approach. Springer, p 4
Li B (2017) Acoustic modeling for google home. In: Proceedings of the Interspeech, pp 399–403
Tian Z, Yi J, Tao J, Bai Y, Wen Z (2019) Self-attention transducers for end-to-end speech recognition. In: Proceedings of the Interspeech, pp 4395–4399
Gannot S, Vincent E, Markovich-Golan S, Ozerov A (2017) A consolidated perspective on multimicrophone speech enhancement and source separation. In: IEEE/ACM transactions on ASLP, vol 25, no 4, pp 692–730
Gangamohan P, Mittal VK, Yegnanarayana B (2012) A flexible analysis synthesis tool (fast) for studying the characteristic features of emotion in speech. In: Consumer Communications and Networking Conference (CCNC), pp 250–254
Yegnanarayana B, Murty K (2009) Event-based instantaneous fundamental frequency estimation from speech signals. In: IEEE transactions on audio speech and language processing, vol 17, no 4, pp 614–624
Krothapalli SR, Yadav J, Sarkar S, Koolagudi SG, Vuppala AK (2012) Neural network-based feature transformation for emotion independent speaker identification. Int J Speech Technol 15(3):335–349
Desai S, Black AW, Yegnanarayana B, Prahallad K (2010) Spectral mapping using artificial neural networks for voice conversion. IEEE Trans Audio Speech Lang Process 18(5):954–964
Saon G (2017) English conversational telephone speech recognition by humans and machines. In: Proceedings of the Interspeech, pp 132–136
Karita S, Soplin NEY, Watanabe S, Delcroix M, Ogawa A, Nakatani T (2019) Improving transformer-based end-to-end speech recognition with connectionist temporal classification and language model integration. In: Proceedings of the Interspeech, pp 1408–1412
Rahulamathavan Y, Sutharsini KR, Ray IG, Lu R, Rajarajan M (2019) Privacy-preserving vector-based speaker verification. In: IEEE/ACM Transactions on the audio speech language process, vol 27, no 3, pp 496–506
European Parliament and Council (2016) Regulation (EU) 2016/679 of the European Parliament and the council of 27 Apr. 2016 on the Protection of Natural Persons With Regard to the Processing of Personal Data and on the Free Movement of Such Data and Repealing Directive 95/46/EC (General Data Protection Regulation
Nautsch A, Jasserand C, Kindt E, Todisco M, Trancoso L, Evans N (2019) The GDPR & speech data: Reflections of legal and technology communities first steps towards a common understanding. In: Proceedings of the Interspeech, pp 3695–3699
Papernot N, McDaniel P, Sinha A, Wellman MP (2018) SoK: security and privacy in machine learning. In: Proceedings of the IEEE European symposium security, privacy, pp 399–414
Povey D, et al (2011) The Kaldi speech recognition toolkit. In: Proceedings of the IEEE workshop automatic speech recognition understanding
Lin P-C, Yankson B, Hung PCK (2020) A prototype of privacy identification system for smart toy dialogue design. In: 17th IEEE International Conference on Networking, Sensing and Control (IEEE ICNSC 2020), Nanjing, China, pp 1–6
Lin P-C, Yankson B, Lu ZH, Hung PCK (2019) Children privacy identification system in LINE Chatbot for smart toys. In: IEEE International Conference on Cloud Computing (IEEE CLOUD 2019), Milan, Italy
Lin P-C, Lin YH, Lai C-A, Chauhan V, Arbai N (2019) Movie recommender system with perceptual engineering in Chatbot. In: The 2nd International Conference on Innovative & Advanced Multidisciplinary Research (ICIAMR 2019), Park Avenue Convention Centre, Singapore, Oct. 12–13
Thomas L, Lothar F, Kilian P, Kai R (2004) Exploitation of public and private WiFi coverage for new business models. In: Lamersdorf W, Tschammer V, Amarger S (eds)
Bahl P, Padmanabhan VN (2000) RADAR: an in-building RF-based user location and tracking system. In: Proceedings of 19th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM’00), vol 2, pp 775–784
Manikanta K, Kiran J, Dinesh B, Sachin K (2015) Decimeter Level Localization Using Wi-Fi. In: Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication. SIGCOMM '15. New York, NY, USA. ACM. pp 269–282
Franzago M (2018) Collaborative model-driven software engineering: a classification framework and a research map. IEEE Trans Softw Eng. https://doi.org/10.1109/TSE.2017.2755039
Storey MA (2017) How social and communication channels shape and challenge a participatory culture in software development. In: IEEE Transactions on the Software Engineering, vol 43, no 2, pp 185–204
Lin B (2016) Why developers are slacking off: understanding how software teams use slack. In: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW 16), pp 333–336
Tian Y (2017) APIBot: question answering bot for API documentation. In: Proceedings of the 32nd IEEE/ACM Conference on the Automated Software Engineering (ASE 17), pp 153–158
Nenkov N, Dimitrov G, Dyachenko Y, Koeva K (2016) Artificial intelligence technologies for personnel learning management systems. In: 2016 IEEE 8th International Conference on Intelligent Systems (IS), pp 189–195. https://doi.org/10.1109/IS.2016.7737420.
Nguyen MH (2017) The latest market research trends and landscape in the growing AI Chatbot Industry. https://www.businessinsider.de/chatbot-market-stats-trends-size-ecosystem-research-2017-10?r=US&IR=T.
Fawthrop A (2019) Voice payments set to drive a new era of commerce for Amazon Pay
Payments pioneer launches Voice Pay transactions (2017) Biometric Technology Today, vol 15, pp 2–3, 05
The Rise of Voice Payment Technology in Banking (2019)
Hattersley L (2018) AIY Projects: create a voice kita with your raspberry Pi. In: The MagPi 2017.
Lee H (2018) Voice user interface projects: build voice-enabled applications using dialogflow for google home and Alexa skills kit for Amazon Echo. Packt Publishing Ltd
Gunawan TS, Mokhtar MN, Kartiwi M, Ismail N, Effendi MR, Qodim H (2020) Development of voice-based smart home security system using google voice kit. In: 2020 6th International Conference on Wireless and Telematics (ICWT), 2020, pp 1–4. https://doi.org/10.1109/ICWT50448.2020.9243633.
Vimalkumar M, Sharma SK, Singh JB, Yogesh K (2021) Okay google, what about my privacy? User’s privacy perceptions and acceptance of voice-based digital assistants. Comput Hum Behav 120:106763. https://doi.org/10.1016/j.chb.2021.106763
Krazit T (2010) Google finding its voice In: CNet. Retrieved from https://www.cnet.com/news/google-finding-its-voice/.
Epstein J, Klinkenberg WD (2001) From eliza to internet: a brief history of computerized assessment. Comput Hum Behav 17(3):295–314. https://doi.org/10.1016/S0747-5632(01)00004-8
Marketer E (2019) US voice assistant users 2019. In: Insider intelligence trends, forecasts and statistics. Retrieved February 1, 2021, from eMarketer website: https://www.emarketer.com/content/us-voice-assistant-users-2019.
Foehr J, Germelmann CC (2020) Alexa, can I trust you? Exploring consumer paths to trust in smart voice-interaction technologies. J Assoc Consum Res 5(2):181–205. https://doi.org/10.1086/707731
Bellman S, Johnson EJ, Kobrin SJ, Lohse GL (2004) International differences in information privacy concerns: a global survey of consumers. Inf Soc 20(5):313–324. https://doi.org/10.1080/01972240490507956
Miltgen CL, Henseler J, Gelhard C, Popovi A (2016) Introducing new products that affect consumer privacy: a mediation model. J Bus Res 69(10):4659–4666. https://doi.org/10.1016/j.jbusres.2016.04.015
Zeng F, Ye Q, Li J, Yang Z (2020) Does self-disclosure matter? A dynamic two-stage perspective for the personalization-privacy paradox. J Bus Res. https://doi.org/10.1016/j.jbusres.2020.02.006
Herrero A, San MH, Garcia-De MLS (2017) Explaining the adoption of social networks sites for sharing user-generated content: a revision of the UTAUT2. Comput Hum Behav 71:209–217. https://doi.org/10.1016/j.chb.2017.02.007
Merhi M, Hone K, Tarhini A (2019) A cross-cultural study of the intention to use mobile banking between Lebanese and British consumers: extending UTAUT2 with security, privacy, and trust”. Technol Soc 59:101. https://doi.org/10.1016/j.techsoc.2019.101151
Kowalczuk P (2018) Consumer acceptance of smart speakers: a mixed-methods approach. J Res Indian Med 12(4):418–431. https://doi.org/10.1108/JRIM-01-2018-0022
Pitardi V, Marriott RH (2021) Alexa, she’s not human, but Unveiling the drivers of consumers’ trust in voice-based artificial intelligence. Psychol Mark. https://doi.org/10.1002/mar.21457
Smith H, Dinev T, Xu H (2011) Information privacy research: an interdisciplinary review. MIS Q 35(4):989–1016
Weisbaum H (2018) Hey Alexa, How Secure Are Voice-Activated Assistants Like You? In: NBC News, https://www.nbcnews.com/tech/security/hey-alexa-how-secure-arevoice-activated-assistants-you-n824566
Song L, Mittal P (2017) Inaudible Voice Commands. In Princeton University, New Jersey, USA
Russakovskii A (2017) Google Is Permanently Nerfing All Home Minis Because Mine Spied on Everything I Said 24/7 [Update x2]. In: Android Police https://www.androidpolice.com/2017/10/10/google-nerfing-home-minis-mine-spied-everything-said-247/
Greatrex C, White M (2017) Parrot Manages to Fool Amazon’s Alexa and Orders His Own GiftBox Without His Owners Knowing. In: Mirror, https://www.mirror.co.uk/news/uk-news/parrot-manages-fool-amazons-alexa-11207953.
Capital F (2016) Voice Recognition: Risks to Our Privacy In: Forbes, https://www.forbes.com/sites/realspin/2016/10/06/voice-recognition-every-single-day-every-word-you-say/#19368ea6786d
Finn RL, Wright D, Friedewald M (2013) Seven types of privacy. In: Gutwirth S, Leenes R, De Hert P, Poullet Y (eds) European Data Protection: Coming of Age. Springer, New York, pp 3–32. https://doi.org/10.1007/978-94-007-5170-5_1
Warren SD, Brandeis LD (1890) The right to privacy. Harward Law Rev 4:193–220. https://doi.org/10.2307/1321160
Westin A (1967) Privacy and freedom. Atheneum Press, Cambridge
Altman I (1975) The environment and social behavior: privacy, personal space, territory, and crowding. Wadsworth Publishing Company, Monterey
Bygrave LA (2002) Data protection law: approaching its rationale, logic and limits. Wolters Kluwer, Alphen aan den Rijn
Solove DJ (2008) Understanding privacy. Harvard University Press, Cambridge
Kaminski ME (2015) Robots in the home: What will we have agreed to? Idaho Law Rev 51:661–677
Yankson B (2021) Continuous improvement process (CIP)-based privacy-preserving framework for smart connected toys. Int J Inf Secur 20:849–869. https://doi.org/10.1007/s10207-020-00535-2
Kaminski ME, Rueben M, Grimm C, Smart WD (2017) Averting robot eyes. Maryland Law Rev. 76:983–1023
Krasnova H, Veltri NF, Günther O (2012) Self-disclosure and privacy calculus on social networking sites: the role of culture. Bus Inform Syst Eng 4:127–135. https://doi.org/10.1007/s12599-012-0216-6
Trepte S, Reinecke L, Ellison NB, Quiring O, Yao MZ, Ziegele M (2017) A cross-cultural perspective on the privacy calculus. Soc Med Soc 3:1–13. https://doi.org/10.1177/2056305116688035
Baxter P, Kennedy J, Senft E, Lemaignan S, Belpaeme T (2016) From characterising three years of HRI to methodology and reporting recommendations. In: 11th ACM/IEEE International Conference on Human–Robot Interaction (HRI) (Christchurch), pp 391–398. https://doi.org/10.1109/HRI.2016.7451777
Lee MK, Tang KP, Forlizzi J, Kiesler S (2011) Understanding users' perception of privacy in human–robot interaction. In: Proceedings of the 6th International Conference on Human–Robot Interaction. ACM, Lausanne, pp 181–182. https://doi.org/10.1145/1957656.1957721
Syrdal DS, Walters ML, Otero N, Koay KL, Dautenhahn K (2007) He knows when you are sleeping – Privacy and the personal robot companion. In: Proceedings of the 2007 AAAI workshop human implications of human–robot interaction, AAAI, Washington DC, pp 28–33
Butler DJ, Huang J, Roesner F, Cakmak M (2015) The privacy-utility tradeoff for remotely teleoperated robots. In: Proceedings of the tenth annual ACM/IEEE International Conference on Human–Robot Interaction. ACM, Portland OR, pp. 27–34. https://doi.org/10.1145/2696454.2696484
Krupp MM, Rueben M, Grimm CM, Smart WD (2017) Privacy and telepresence robotics: What do non-scientists think? In: Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human–Robot Interaction. ACM, Vienna, pp 175–176. https://doi.org/10.1145/3029798.3038384
Rueben M, Bernieri FJ, Grimm CM, Smart WD (2017) Framing effects on privacy concerns about a home telepresence robot. In: Proceedings of the 2017 ACM/IEEE International Conference on Human–Robot Interaction. ACM, Vienna, pp 435–444. https://doi.org/10.1145/2909824.3020218
Lutz C, Schöttler M, Hoffmann CP (2019) The privacy implications of social robots: scoping review and expert interviews. Mobile Media Commun 7:412–434. https://doi.org/10.1177/2050157919843961
Eurobarometer (2015) Special Eurobarometer 427: Autonomous systems
Madden M, Rainie L (2015) Americans' attitudes about privacy, security and surveillance. Pew Internet, Science and Tech Report
Lutz C, Hoffmann CP, Ranzini G (2020) Data capitalism and the user: an exploration of privacy cynicism in Germany. New Media Soc 22:1168–1187. https://doi.org/10.1177/1461444820912544
Acknowledgements
This research was supported by the Ministry of Education, R.O.C., under the grant TEEP@AsiaPlus. The work in this paper was also supported by the Ministry of Science and Technology under Grant No. MOST 109-2221-E-035-063-MY2. The authors would like to express their appreciation to Feng Chia University and the anonymous reviewers for their feedback.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lin, PC., Yankson, B., Chauhan, V. et al. Building a speech recognition system with privacy identification information based on Google Voice for social robots. J Supercomput 78, 15060–15088 (2022). https://doi.org/10.1007/s11227-022-04487-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-022-04487-3