skip to main content
10.1145/1869983.1869992acmconferencesArticle/Chapter ViewAbstractPublication PagessensysConference Proceedingsconference-collections
research-article

The Jigsaw continuous sensing engine for mobile phone applications

Published: 03 November 2010 Publication History

Abstract

Supporting continuous sensing applications on mobile phones is challenging because of the resource demands of long-term sensing, inference and communication algorithms. We present the design, implementation and evaluation of the Jigsaw continuous sensing engine, which balances the performance needs of the application and the resource demands of continuous sensing on the phone. Jigsaw comprises a set of sensing pipelines for the accelerometer, microphone and GPS sensors, which are built in a plug and play manner to support: i) resilient accelerometer data processing, which allows inferences to be robust to different phone hardware, orientation and body positions; ii) smart admission control and on-demand processing for the microphone and accelerometer data, which adaptively throttles the depth and sophistication of sensing pipelines when the input data is low quality or uninformative; and iii) adaptive pipeline processing, which judiciously triggers power hungry pipeline stages (e.g., sampling the GPS) taking into account the mobility and behavioral patterns of the user to drive down energy costs. We implement and evaluate Jigsaw on the Nokia N95 and the Apple iPhone, two popular smartphone platforms, to demonstrate its capability to recognize user activities and perform long term GPS tracking in an energy-efficient manner.

References

[1]
B. Abdesslem et al. Less is more: Energy-efficient mobile sensing with SenseLess. In Proceedings of the 1st ACM workshop on Networking, systems, and applications for mobile handhelds, pages 61--62. ACM, 2009.
[2]
Apple. Introduction to the objective-c 2.0 programming language. Website, 2008.
[3]
Apple. iphone sdk. Website, 2008. http://developer.apple.com/iphone/.
[4]
D. Ashbrook and T. Starner. Using gps to learn significant locations and predict movement across multiple users. Personal Ubiquitous Comput., 7(5):275--286, 2003.
[5]
L. Bao and S. S. Intille. Activity recognition from user annotated acceleration data. In Pervasive, pages 1--17, 2004.
[6]
S. Basu. A linked-HMM model for robust voicing and speech detection. In Acoustics, Speech, and Signal Processing, 2003(ICASSP'03)., volume 1, 2003.
[7]
S. Consolvo, D. McDonald, T. Toscos, M. Chen, J. Froehlich, B. Harrison, P. Klasnja, A. LaMarca, L. LeGrand, R. Libby, et al. Activity sensing in the wild: a field trial of ubifit garden. In Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, pages 1797--1806. ACM, 2008.
[8]
I. Constandache, S. Gaonkar, M. Sayler, R. R. Choudhury, and L. P. Cox. Energy-efficient Localization Via Personal Mobility Profiling. In The First Annual International Conference on Mobile Computing, Applications, and Services, 2009.
[9]
R. A. Howard. Dynamic Programming and Markov Processes. The MIT Press, Cambridge, MA, 1960.
[10]
M. B. Kjæergaard, J. Langdal, T. Godsk, and T. Toftkjær. Entracked: energy-efficient robust position tracking for mobile devices. In MobiSys '09: Proceedings of the 7th international conference on Mobile systems, applications, and services, pages 221--234, New York, NY, USA, 2009.
[11]
J. Lester, T. Choudhury, and G. Borriello. A practical approach to recognizing physical activities. Lecture Notes in Computer Science: Pervasive Computing, pages 1--16, 2006.
[12]
D. Li, I. Sethi, N. Dimitrova, and T. McGee. Classification of general audio data for content-based retrieval. Pattern Recognition Letters, 22(5):533--544, 2001.
[13]
J. Lotters, J. Schipper, P. Veltink, W. Olthuis, and P. Bergveld. Procedure for in-use calibration of triaxial accelerometers in medical applications. Sensors and Actuators A: Physical, 68(1--3):221--228, 1998.
[14]
H. Lu, W. Pan, N. Lane, T. Choudhury, and A. Campbell. SoundSense: scalable sound sensing for people-centric applications on mobile phones. In Proceedings of the 7th international conference on Mobile systems, applications, and services, pages 165--178. ACM New York, NY, USA, 2009.
[15]
M. McKinney and J. Breebaart. Features for audio and music classification. In Proc. ISMIR, pages 151--158, 2003.
[16]
E. Miluzzo, N. Lane, K. Fodor, R. Peterson, H. Lu, M. Musolesi, S. Eisenman, X. Zheng, and A. Campbell. Sensing meets mobile social networks: The design, implementation and evaluation of the CenceMe application. In Proceedings of SenSys08, pages 337--350. ACM New York, NY, USA, 2008.
[17]
D. Mizell. Using gravity to estimate accelerometer orientation. In ISWC '03: Proceedings of the 7th IEEE International Symposium on Wearable Computers, page 252, Washington, DC, USA, 2003. IEEE Computer Society.
[18]
P. Mohan, V. N. Padmanabhan, and R. Ramjee. Nericell: rich monitoring of road and traffic conditions using mobile smartphones. In T. F. Abdelzaher, M. Martonosi, and A. Wolisz, editors, SenSys, pages 323--336. ACM, 2008.
[19]
M. Mun, S. Reddy, K. Shilton, N. Yau, J. Burke, D. Estrin, M. Hansen, E. Howard, R. West, and P. Boda. Peir, the personal environmental impact report, as a platform for participatory sensing systems research. In Proceedings of Mobisys09, pages 55--68. ACM New York, NY, USA, 2009.
[20]
Nokia. Symbian c. Website, 2009. http://www.forum.nokia.com/Technology_Topics/Development_Platforms/Symbian_C.
[21]
D. Peebles, H. Lu, N. Lane, T. Choudhury, and A. Campbell. Community-Guided Learning: Exploiting Mobile Sensor Users to Model Human Behavior. In Proc. of 24th AAAI Conference on Artificial Intelligence, 2010.
[22]
V. Peltonen, J. Tuomi, A. Klapuri, J. Huopaniemi, and T. Sorsa. Computational Auditory Scene Recognition. In IEEE Intl. Conf. on Acoustics Speech and Signal Processing, volume 2. IEEE; 1999, 2002.
[23]
N. Ravi, N. Dandekar, P. Mysore, and M. L. Littman. Activity recognition from accelerometer data. IAAI-05, 2005.
[24]
S. Reddy, J. Burke, D. Estrin, M. Hansen, and M. Srivastava. Determining transportation mode on mobile phones. In Proceedings of The 12th IEEE Int. Symposium on Wearable Computers, 2008.
[25]
E. Scheirer and M. Slaney. Construction and evaluation of a robust multifeature speech/musicdiscriminator. In Acoustics, Speech, and Signal Processing, 1997. ICASSP-97, 1997 IEEE International Conference on, volume 2, 1997.
[26]
D. Smith, L. Ma, and N. Ryan. Acoustic environment as an indicator of social and physical context. Personal and Ubiquitous Computing, 10(4):241--254, 2006.
[27]
M. Spina and V. Zue. Automatic transcription of general audio data: preliminary analyses. In Spoken Language, 1996. ICSLP 96. Proceedings., volume 2, 1996.
[28]
Y. Wang, B. Krishnamachari, Q. Zhao, and M. Annavaram. Towards the Tradeoff between Energy Efficiency and User State Estimation Accuracy in Mobile Sensing. In Proceedings of The First Annual International Conference on Mobile Computing, Applications, and Services, 2009.
[29]
Y. Wang, J. Lin, M. Annavaram, Q. Jacobson, J. Hong, B. Krishnamachari, and N. Sadeh. A framework of energy efficient mobile sensing for automatic user state recognition. In Proceedings of the 7th international conference on Mobile systems, applications, and services, pages 179--192. ACM, 2009.
[30]
I. Witten. Weka: Practical Machine Learning Tools and Techniques with Java Implementations. Dept. of Computer Science, University of Waikato, 1999.
[31]
T. Zhang and C. Kuo. Audio-guided audiovisual data segmentation, indexing, and retrieval. In Proceedings of SPIE, volume 3656, page 316. SPIE, 1998.
[32]
F. Zheng, G. Zhang, and Z. Song. Comparison of Different Implementations of MFCC. Journal of Computer Science and Technology, 16(6):582--589, 2001.

Cited By

View all
  • (2025)MMEdge: Real-time Multimodal Inference via Cross-Modal Speculative Sensing and EncodingProceedings of the 26th International Workshop on Mobile Computing Systems and Applications10.1145/3708468.3715688(132-132)Online publication date: 26-Feb-2025
  • (2023)Preventing Keratoconus through Eye Rubbing Activity Detection: A Machine Learning ApproachElectronics10.3390/electronics1204102812:4(1028)Online publication date: 18-Feb-2023
  • (2023)Automated Face-To-Face Conversation Detection on a Commodity Smartwatch with Acoustic SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108827:3(1-29)Online publication date: 27-Sep-2023
  • Show More Cited By

Index Terms

  1. The Jigsaw continuous sensing engine for mobile phone applications

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      SenSys '10: Proceedings of the 8th ACM Conference on Embedded Networked Sensor Systems
      November 2010
      461 pages
      ISBN:9781450303446
      DOI:10.1145/1869983
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 03 November 2010

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. activity recognition
      2. machine learning
      3. mobile phone sensing
      4. power management

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      Acceptance Rates

      Overall Acceptance Rate 198 of 990 submissions, 20%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)69
      • Downloads (Last 6 weeks)4
      Reflects downloads up to 17 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2025)MMEdge: Real-time Multimodal Inference via Cross-Modal Speculative Sensing and EncodingProceedings of the 26th International Workshop on Mobile Computing Systems and Applications10.1145/3708468.3715688(132-132)Online publication date: 26-Feb-2025
      • (2023)Preventing Keratoconus through Eye Rubbing Activity Detection: A Machine Learning ApproachElectronics10.3390/electronics1204102812:4(1028)Online publication date: 18-Feb-2023
      • (2023)Automated Face-To-Face Conversation Detection on a Commodity Smartwatch with Acoustic SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108827:3(1-29)Online publication date: 27-Sep-2023
      • (2023)WiEdge: Edge Computing for Audio Sensing Applications With Accurate Wireless Link PredictionIEEE Internet of Things Journal10.1109/JIOT.2022.317366810:5(3982-3994)Online publication date: 1-Mar-2023
      • (2023)The design of a mental health service platform for college students based on multi-modal informationMicroprocessors & Microsystems10.1016/j.micpro.2022.10474896:COnline publication date: 1-Feb-2023
      • (2023)A perspective on human activity recognition from inertial motion dataNeural Computing and Applications10.1007/s00521-023-08863-935:28(20463-20568)Online publication date: 31-Jul-2023
      • (2023)Activity Recognition in IoTSelf-Powered Internet of Things10.1007/978-3-031-27685-9_2(13-25)Online publication date: 17-Jun-2023
      • (2022)Edge-Centric Programming for IoT Applications With Automatic Code PartitioningIEEE Transactions on Computers10.1109/TC.2021.312936771:10(2408-2422)Online publication date: 1-Oct-2022
      • (2022)Crowdsourcing Mobile Data for A Passive Indoor Positioning System - The MAA Case Study2022 18th International Conference on Mobility, Sensing and Networking (MSN)10.1109/MSN57253.2022.00024(59-67)Online publication date: Dec-2022
      • (2022)Probabilistic Cascading Classifier for Energy-Efficient Activity Monitoring in WearablesIEEE Sensors Journal10.1109/JSEN.2022.317588122:13(13407-13423)Online publication date: 1-Jul-2022
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media