Abstract
We present SocialSense , a collaborative smartphone based speaker and mood identification and reporting system that uses a user’s voice to detect and log his/her speaking and mood episodes. SocialSense collaboratively works with other phones that are running the app present in the vicinity to periodically send/receive speaking and mood vectors to/from other users present in a social interaction setting, thus keeping track of the global speaking episodes of all users with their mood. In addition, it utilizes a novel event-adaptive dynamic classification scheme for speaker identification which updates the speaker classification model every time one or more users enter or leave the scenario, ensuring a most updated classifier based on user presence. Evaluation of using dynamic classifiers shows that SocialSense improves speaker identification accuracy by 30% compared to traditional static speaker identification systems, and a 10% to 43% performance boost under various noisy environments. SocialSense also improves the mood classification accuracy by 4% to 20% compared to the baseline approaches. Energy consumption experiments show that its device daily lifetime is between 10-14 hours.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Neumann, R., Strack, F.: Mood Contagion: The automatic transfer of mood between persons. Journal of Personality and Social Psychology 79(2), 211–223 (2000)
Reynolds, D.A.: Speaker identification and verification using gaussian mixture speaker models. Speech Communication 17(1-2), 91–108 (1995)
Reynolds, D.A., Rose, R.C.: Robust text-independent speaker identification using gaussian mixture speaker models. Transactions on Speech and Audio Processing 3(1), 72–83 (1995)
Luo, C., Chan, M.C.: SocialWeaver: collaborative inference of human conversation networks using smartphones. In: 11th ACM Conference on Embedded Networked Sensor Systems (SenSys), Roma, Italy (2013)
Nakakura, T., Sumi, Y., Nishida, T.: Neary: conversation field detection based on similarity of auditory situation. In: 10th Workshop on Mobile Computing Systems and Applications (HotMobile), Santa Cruz, California, USA (2009)
Lu, H., Brush, B., Priyantha, B., Karlson, A.K., Liu, J.: SpeakerSense: Energy efficient unobtrusive speaker identification on mobile phones. In: IEEE Pervasive Computing and Communication (PerCom), Seattle, Washington, USA (2011)
Rachuri, K., Musolesi, M., Mascolo, C., Rentfrow, P.J., Longworth, C., Aucinas, A.: EmotionSense: A mobile phone based adaptive platform for experimental social psychology research. In: ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp), Copenhagen, Denmark (2010)
Yu, C., Aoki, P.M., Woodruff, A.: Detecting user engagement in everyday conversations. In: 8th International Conference on Spoken Language Processing, South Korea (2004)
Casale, P., Pujol, O., Radeva, P.: Face-to-face social activity detection using data collected with a wearable device. In: 4th Iberian Conference on Pattern Recognition, Portugal (2009)
Miluzzo, E., Lane, N.D., Fodor, K., Peterson, R., Lu, H., Musolesi, M., Eisenman, S.B., Zheng, X., Campbell, A.T.: Sensing meets mobile social networks: the design, implementation and evaluation of the CenceMe application. In: 6th ACM Conference on Embedded Networked Sensor Systems (SenSys), Raleigh, North Carolina, USA (2008)
Choudhury, T.: Sensing and modeling human networks. Ph.D. Thesis, Program in Media Arts and Sciences, Massachusetts Institute of Technology (2004)
Chen, D., Yang, J., Malkin, R., Wactlar, H.D.: Detecting social interactions of the elderly in a nursing home environment. ACM Transactions on Multimedia Computing, Communications and Applications 3(1), 1–22 (2007)
Li, Q., Chen, S., Stankovic, J.A.: Multi-modal in-person interaction monitoring using smartphone and on-body sensors. In: IEEE International Conference on Body Sensor Networks, Cambridge, MA, USA (2013)
Audacity, http://audacity.sourceforge.net/
Kim, J., Lee, S., Narayan, S.S.: An exploratory study of manifolds of emotional speech. In: Acoustics Speech and Signal Processing, Dallas, TX, USA (2010)
Stefanacci, R.G.: How big an issue is depression in assisted living? Assisted Living Consult 4(4), 30–35 (2008)
Miluzzo, E., Cornelius, C.T., Ramaswamy, A., Choudhury, T., Liu, Z., Campbell, A.T.: Darwin phones: the evolution of sensing and inference on mobile phones. In: 8th International Conference on Mobile Systems, Applications, and Services (MobiSys), San Francisco, California, USA (2010)
Xu, C., et al.: Crowd++: unsupervised speaker count with smartphones. In: ACM International Joint Conference on Pervasive and Ubiquitous Computing, Zurich, Switzerland (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Ahmed, M.Y., Kenkeremath, S., Stankovic, J. (2015). SocialSense: A Collaborative Mobile Platform for Speaker and Mood Identification. In: Abdelzaher, T., Pereira, N., Tovar, E. (eds) Wireless Sensor Networks. EWSN 2015. Lecture Notes in Computer Science, vol 8965. Springer, Cham. https://doi.org/10.1007/978-3-319-15582-1_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-15582-1_5
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-15581-4
Online ISBN: 978-3-319-15582-1
eBook Packages: Computer ScienceComputer Science (R0)