skip to main content
10.1145/1814433.1814437acmconferencesArticle/Chapter ViewAbstractPublication PagesmobisysConference Proceedingsconference-collections
research-article

Darwin phones: the evolution of sensing and inference on mobile phones

Published: 15 June 2010 Publication History

Abstract

We present Darwin, an enabling technology for mobile phone sensing that combines collaborative sensing and classification techniques to reason about human behavior and context on mobile phones. Darwin advances mobile phone sensing through the deployment of efficient but sophisticated machine learning techniques specifically designed to run directly on sensor-enabled mobile phones (i.e., smartphones). Darwin tackles three key sensing and inference challenges that are barriers to mass-scale adoption of mobile phone sensing applications: (i) the human-burden of training classifiers, (ii) the ability to perform reliably in different environments (e.g., indoor, outdoor) and (iii) the ability to scale to a large number of phones without jeopardizing the "phone experience" (e.g., usability and battery lifetime). Darwin is a collaborative reasoning framework built on three concepts: classifier/model evolution, model pooling, and collaborative inference. To the best of our knowledge Darwin is the first system that applies distributed machine learning techniques and collaborative inference concepts to mobile phones. We implement the Darwin system on the Nokia N97 and Apple iPhone. While Darwin represents a general framework applicable to a wide variety of emerging mobile sensing applications, we implement a speaker recognition application and an augmented reality application to evaluate the benefits of Darwin. We show experimental results from eight individuals carrying Nokia N97s and demonstrate that Darwin improves the reliability and scalability of the proof-of-concept speaker recognition application without additional burden to users.

References

[1]
Apple iPhone. http://www.apple.com/iphone.
[2]
The CASA project. http://www.casa.umass.edu/.
[3]
FFTW. http://www.fftw.org.
[4]
Kiss FFT. http://sourceforge.net/projects/kissfft.
[5]
Nokia Energy Profiler. http://tinyurl.com/ycostaq.
[6]
Nokia Series. http://europe.nokia.com/nseries/.
[7]
Qt platform. http://qt.nokia.com/.
[8]
Sensorplanet. http://research.nokia.com/research/projects/sensorplanet/index.html.
[9]
T. Abdelzaher, et al. Mobiscopes for Human Spaces. In IEEE Pervasive Computing, volume 6, page 20. IEEE, 2007.
[10]
M. Azizyan, et al. SurroundSense: Mobile Phone Localization via Ambience Fingerprinting. In MobiCom'09, 2009.
[11]
S. Basu. A linked-HMM Model for Robust Voicing and Speech Detection. In ICASSP'03, 2003.
[12]
A. Campbell, S. Eisenman, N. Lane, E. Miluzzo, and R. Peterson. People-Centric Urban Sensing. In WICON'06, 2006.
[13]
A. Campbell, et al. The Rise of People-Centric Sensing. In IEEE Internet Computing Special Issue on Mesh Networks, July-August 2008.
[14]
T. Choudhury, et al. The Mobile Sensing Platform: an Embedded System for Capturing and Recognizing Human Activities. IEEE Pervasive Magazine, Spec. Issue on Activity--Based Computing, 2008.
[15]
S. B. Eisenman. People-Centric Mobile Sensing Networks. Ph.D. Dissertation, October 2008.
[16]
H. Ezzaidi and J. Rouat. Pitch and MFCC Dependent GMM Models for Speaker Identification systems. In Electrical and Computer Engineering, 2004. Canadian Conference on, volume 1, 2004.
[17]
Z. Fang, Z. Guoliang, and S. Zhanjiang. Comparison of Different Implementations of MFCC. J. Comput. Sci. Technol., 16(6), 2001.
[18]
G. Friedland and O. Vinyals. Live Speaker Identification in Conversations. In ACM Multimedia'08, 2008.
[19]
R. Honicky, et al. Increasing the Precision of Mobile Sensing Systems Through Super-Sampling. UrbanSense08, page 31, 2008.
[20]
R. Honicky, et al. N-SMARTS: Networked Suite of Mobile Atmospheric Real-Time Sensors. In Networked Systems for Developing Regions'08, 2008.
[21]
A. Kansal, M. Goraczko, and F. Zhao. Building a Sensor Network of Mobile Phones. In IPSN'07.
[22]
D. Kim, J. Hightower, R. Govindan, and D. Estrin. Discovering Semantically Meaningful Places from Pervasive RF-Beacons. In UbiComp'09, pages 21--30. ACM, 2009.
[23]
T. Kinnunen, T. Kilpeläinen, and P. Fränti. Comparison of Clustering Algorithms in Speaker Identification. In SPC 2000.
[24]
R. Kumar, et al. DFuse: a Framework for Distributed Data Fusion. In SenSys'03, 2003.
[25]
N. Lane, et al. Cooperative Techniques Supporting Sensor-Based People-Centric Inferencing. In Pervasive'08.
[26]
I. Lapidot, H. Guterman, and A. Cohen. Unsupervised Speaker Recognition Based on Competition Between Self-Organizing Maps. IEEE Transactions on Neural Networks, 13(4):877--887, 2002.
[27]
H. Lu, et al. SoundSense: Scalable Sound Sensing for People-Centric Applications on Mobile Phones. MobiSys'09, 2009.
[28]
B. Maeireizo, D. Litman, and R. Hwa. Co-Training for Predicting Emotions with Spoken Dialogue Data. In ACL'04, 2004.
[29]
E. Miluzzo, et al. Sensing Meets Mobile Social Networks: the Design, Implementation and Evaluation of the CenceMe Application. In SenSys'08, 2008.
[30]
P. Mohan, et al. Nericell: Rich Monitoring of Road and Traffic Conditions Using Mobile Smartphones. In SenSys'08, 2008.
[31]
M. Mun, et al. PEIR, the Personal Environmental Impact Report, as a Platform for Participatory Sensing Systems Research. In MobiSys'09, 2009.
[32]
M. Musolesi, et al. Supporting Energy-Efficient Uploading Strategies for Continuous Sensing Applications on Mobile Phones. In Pervasive'10, 2010.
[33]
S. Nath, et al. Synopsis Diffusion for Robust Aggregation in Sensor Networks. In SenSys'04, 2004.
[34]
Nokia. N900. http://maemo.nokia.com/n900.
[35]
NRC. Sensing the World with Mobile Devices. Technical Report, 2008.
[36]
R. Olfati-Saber. Distributed Kalman Filter with Embedded Consensus Filters. In CDC-ECC'05, pages 8179--8184, 2005.
[37]
R. Olfati-Saber. Distributed Kalman Filtering and Sensor Fusion in Sensor Networks. Lecture notes in control and information sciences, 331:157, 2006.
[38]
S. Patil, et al. Serial Data Fusion Using Space-Filling Curves in Wireless Sensor Networks. In IEEE SECON'04, 2004.
[39]
D. Reynolds, et al. Speaker Verification Using Adapted Gaussian Mixture Models. Digital Signal Processing, 10(1-3):19--41, 2000.
[40]
D. Reynolds and R. Rose. Robust Text-Independent Speaker Identification Using Gaussian Mixture Speaker Models. IEEE transactions on Speech and Audio Processing, 1995.
[41]
E. Riloff, J. Wiebe, and T. Wilson. Learning Subjective Nouns Using Extraction Pattern Bootstrapping. In CoNLL'03, 2003.
[42]
E. Shriberg. Higher-Level Features in Speaker Recognition. Lecture Notes in Computer Science, 4343:241, 2007.
[43]
E. Shriberg, et al. Modeling Prosodic Feature Sequences for Speaker Recognition. Speech Communication.
[44]
E. Shriberg and E. Stolcke. The Case for Automatic Higher-Level Features in Forensic Speaker Recognition. In Interspeech'08, 2008.
[45]
N. Shrivastava, et al. Medians and Beyond: new Aggregation Techniques for Sensor Networks. In SenSys'04, pages 239--249. ACM New York, NY, USA, 2004.
[46]
D. Siewiorek, et al. Sensay: a Context-Aware Mobile Phone. In ISWC'03, 2003.
[47]
C. Tadj, et al. GMM Based Speaker Identification Using Training Time Dependent Number of Mixtures. In ICASSP'98, volume 2, 1998.
[48]
Y. Wang, et al. A Framework of Energy Efficient Mobile Sensing for Automatic User State Recognition. In MobiSys'09, 2009.
[49]
L. Xiao, S. Boyd, and S. Lall. A Scheme for Robust Distributed Sensor Fusion Based on Average Consensus. In IPSN';05, 2005.
[50]
D. Yarowsky. Unsupervised Word Sense Disambiguation Rivaling Supervised Methods. In ACL'95.
[51]
S. Zhang, S. Zhang, and B. Xu. A Robust Unsupervised Speaker Clustering of Speech Utterances. In IEEE NLP-KE'05.
[52]
J. Zhao, et al. Computing Aggregates for Monitoring Wireless Sensor Networks. In SNPA'03, 2003.
[53]
M. Zhu. Recall, Precision and Average Precision. In Technical Report, University of Waterloo, 2004.
[54]
X. Zhu. Semi-Supervised Learning Literature Survey. In Computer Sciences TR, University of Wisconsin-Madison, 2008.

Cited By

View all
  • (2022)A Comprehensive Review of Internet of Things: Technology Stack, Middlewares, and Fog/Edge Computing InterfaceSensors10.3390/s2203099522:3(995)Online publication date: 27-Jan-2022
  • (2022)Smartphone-Operated Wireless Chemical Sensors: A ReviewChemosensors10.3390/chemosensors1002005510:2(55)Online publication date: 30-Jan-2022
  • (2021)Application of Machine Learning Classifiers for Predicting Human Activity2021 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT)10.1109/IAICT52856.2021.9532572(39-44)Online publication date: 27-Jul-2021
  • Show More Cited By

Index Terms

  1. Darwin phones: the evolution of sensing and inference on mobile phones

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      MobiSys '10: Proceedings of the 8th international conference on Mobile systems, applications, and services
      June 2010
      382 pages
      ISBN:9781605589855
      DOI:10.1145/1814433
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      In-Cooperation

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 15 June 2010

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. collaborative inference
      2. distributed machine learning
      3. mobile phones
      4. mobile sensing systems

      Qualifiers

      • Research-article

      Conference

      MobiSys'10
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 274 of 1,679 submissions, 16%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)34
      • Downloads (Last 6 weeks)2
      Reflects downloads up to 17 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2022)A Comprehensive Review of Internet of Things: Technology Stack, Middlewares, and Fog/Edge Computing InterfaceSensors10.3390/s2203099522:3(995)Online publication date: 27-Jan-2022
      • (2022)Smartphone-Operated Wireless Chemical Sensors: A ReviewChemosensors10.3390/chemosensors1002005510:2(55)Online publication date: 30-Jan-2022
      • (2021)Application of Machine Learning Classifiers for Predicting Human Activity2021 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT)10.1109/IAICT52856.2021.9532572(39-44)Online publication date: 27-Jul-2021
      • (2020)Zero-Shot Human Activity Recognition Using Non-Visual SensorsSensors10.3390/s2003082520:3(825)Online publication date: 4-Feb-2020
      • (2020)Opportunities and challenges in the collection and analysis of digital phenotyping dataNeuropsychopharmacology10.1038/s41386-020-0771-3Online publication date: 17-Jul-2020
      • (2019)Human Activity Recognition Using Inertial Sensors in a Smartphone: An OverviewSensors10.3390/s1914321319:14(3213)Online publication date: 21-Jul-2019
      • (2019)SoundSemanticsProceedings of the 18th International Conference on Information Processing in Sensor Networks10.1145/3302506.3310402(217-228)Online publication date: 16-Apr-2019
      • (2019)HCFContext: Smartphone Context Inference via Sequential History-based Collaborative Filtering2019 IEEE International Conference on Pervasive Computing and Communications (PerCom10.1109/PERCOM.2019.8767396(1-10)Online publication date: Mar-2019
      • (2019)A Survey on Mobile Crowdsensing Systems: Challenges, Solutions, and OpportunitiesIEEE Communications Surveys & Tutorials10.1109/COMST.2019.291403021:3(2419-2465)Online publication date: Nov-2020
      • (2019)Maximizing MapReduce job speed and reliability in the mobile cloud by optimizing task allocationPervasive and Mobile Computing10.1016/j.pmcj.2019.10108260(101082)Online publication date: Nov-2019
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media