skip to main content
10.1145/2002259.2002305acmconferencesArticle/Chapter ViewAbstractPublication PagesdebsConference Proceedingsconference-collections
research-article

Scheduling for real-time mobile MapReduce systems

Published: 11 July 2011 Publication History

Abstract

The popularity of portable electronics such as smartphones, PDAs and mobile devices and their increasing processing capabilities has enabled the development of several real-time mobile applications that require low-latency, high-throughput response and scalability. Supporting real-time applications in mobile settings is especially challenging due to limited resources, mobile device failures and the significant quality fluctuations of the wireless medium. In this paper we address the problem of supporting distributed real-time applications in a mobile MapReduce framework under the presence of failures. We present Real-Time Mobile MapReduce (MiscoRT), our system aimed at supporting the execution of distributed applications with real-time response requirements. We propose a two level scheduling scheme, designed for the MapReduce programming model, that effectively predicts application execution times and dynamically schedules application tasks. We have performed extensive experiments on a testbed of Nokia N95 8GB smartphones. We demonstrate that our scheduling system is efficient, has low overhead and performs up to 32% faster than its competitors.

References

[1]
Kin: Its nice to meet you. http://kin.com.
[2]
Nokia energy profiler. http://www.forum.nokia.com/main/resources/user experience/ powermanagement/nokia energy profiler/.
[3]
G.-S. Ahn, M. Musolesi, H. Lu, R. Olfati-Saber, and A. T. Campbell. Metrotrack: Predictive tracking of mobile events using mobile phones. In IEEE DCOSS, June 2010.
[4]
G. Andrienko, N. Andrienko, P. Bak, S. Kisilevich, and D. Keim. Analysis of community-contributed space- and time-referenced data (example of flickr and panoramio photos). In IEEE Symposium on Visual Analytics Science and Technology, Atlantic City, NJ, Oct, 2009.
[5]
H. K. Anna and J. Gerda. A robust decentralized job scheduling approach for mobile peers in ad-hoc grids. In CCGrid. Rio de Janeiro, Brazil, 5 May 2007.
[6]
E. M. Atkins, T. F. Abdelzaher, K. G. Shin, and E. H. Durfee. Planning and resource allocation for hard real-time, fault-tolerant plan execution. Autonomous Agents and Multi-Agent Systems, 4(1-2):57--78, 2001.
[7]
H. Aydin. On fault-sensitive feasibility analysis of real-time task sets. In RTSS, Lisbon, Portugal, pages 426--434, Dec 2004.
[8]
H. Aydin, R. Melhem, and D. Mosse. Optimal scheduling of imprecise computation tasks in the presence of multiple faults. In RTCSA, South Korea, Dec 2000.
[9]
B. Bamba, L. Liu, A. Iyengar, and P. S. Yu. Distributed processing of spatial alarms: A safe region-based approach. In ICDCS, pages 207--214, Washington, DC, USA, 2009.
[10]
V. Berten, J. Goossens, and E. Jeannot. A probabilistic approach for fault tolerant multiprocessor real-time scheduling. IPDPS, Greece, 0:152, 2006.
[11]
J.-J. Chen, C.-Y. Yang, T.-W. Kuo, and S.-Y. Tseng. Real-time task replication for fault tolerance in identical multiprocessor systems. In RTAS, WA, Apr 2007
[12]
M. Cinque, D. Cotroneo, Z. Kalbarczyk, and R. K. Iyer. How do mobile phones fail? a failure data analysis of symbian os smart phones. In DSN, pages 585--594, Washington, DC, USA, 2007.
[13]
D. Cutting. Hadoop core. http://hadoop.apache.org/core/.
[14]
J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In OSDI, San Francisco, CA, USA, pages 137--150, Dec 2004.
[15]
A. J. Dou, V. Kalogeraki, D. Gunopulos, T. Mielikainen, and V. H. Tuulos. Misco: A mapreduce framework for mobile systems. In PETRA 2010, Samos, Greece, June 2010.
[16]
C. Ebeling. An Introduction to Reliability and Maintainability Engineering. McGraw-Hill, 1997.
[17]
P. Emberson and I. Bate. Extending a task allocation algorithm for graceful degradation of real-time distributed embedded systems. In RTSS, Barcelona, Spain, Dec 2008.
[18]
T. Facchinetti, L. Almeida, G. Buttazzo, and C. Marchini. Real-time resource reservation protocol for wireless mobile ad hoc networks. In RTSS, Portugal, Dec 2004.
[19]
T. H. Feng and E. A. Lee. Real-time distributed discrete-event execution with fault tolerance. In RTAS, St. Louis, MO, Apr 2008.
[20]
J. Freyne, A. Brennan, B. Smyth, D. Byrne, A. Smeaton, and G. Jones. Automated murmurns: The social mobile tourist application. In SMW'09 - Social Mobile Web 2009, 2009.
[21]
S. Ghosh, R. Melhem, and D. Mosse. Enhancing real-time schedules to tolerate transient faults. In RTSS, Pisa, Italy, pages 120--129, Dec 1995.
[22]
I. Giurgiu, O. Riva, D. Juric, I. Krivulev, and G. Alonso. Calling the cloud: Enabling mobile phones as interfaces to cloud applications. In Middleware, November 30 - December 4 2009.
[23]
A. S. Gokhale, B. Natarajan, D. C. Schmidt, and J. K. Cross. Towards real-time fault-tolerant corba middleware. Cluster Computing, 7(4):331--346, 2004.
[24]
S. Gopalakrishnan and M. Caccamo. Task partitioning with replication upon heterogeneous multiprocessor systems. RTAS, San Jose, CA, USA, 06.
[25]
B. He, W. Fang, Q. Luo, N. K. Govindaraju, and T. Wang. Mars: a mapreduce framework on graphics processors. In PACT, ON, Canada, Oct 2008.
[26]
T. He, J. A. Stankovic, C. Lu, and T. F. Abdelzaher. SPEED: A stateless protocol for real-time communication in sensor networks. In ICDCS, Tokyo, Japan, May May 2003.
[27]
P. S. Huan Li and K. Ramamritham. Scheduling messages with deadlines in multi-hop real-time sensor networks. In RTAS, pages 415--425, 2005.
[28]
H.-M. Huang and C. Gill. Design and performance of a fault-tolerant real-time corba event service. In ECRTS, Dresden, Germany, pages 33--42, Aug 2006.
[29]
V. Kalogeraki, P. M. Melliar-Smith, and L. E. Moser. Dynamic scheduling of distributed method invocations. RTSS, Orlando, Florida, USA, Nov 2000.
[30]
K. Karenos and V. Kalogeraki. Traffic management in sensor networks with a mobile sink. IEEE TPDS, 21(10):1515--1530, 2010.
[31]
M. Kim and D. Kotz. Extracting a mobility model from real user traces. In INFOCOM, 2006.
[32]
A. Leon-Garcia. Probability and Random Processes for Electrical Engineering (2nd Edition). Prentice Hall, July 1993.
[33]
C. Lu, B. M. Blum, T. F. Abdelzaher, J. A. Stankovic, and T. He. RAP: A real-time communication architecture for large-scale wireless sensor networks. In IEEE RTAS, pages 55--66, San Jose, CA, Sep. 2002.
[34]
P. Melliar-Smith, L. Moser, V. Kalogeraki, and P. Narasimhan. The realize middleware for replication and resource management. In Middleware'98, The Lake District, England, September 1998.
[35]
E. Miluzzo, C. Cornelius, A. Ramaswamy, T. Choudhury, Z. liu, and A. Campbell. Darwin phones: the evolution of sensing and inference on mobile phones. In Mobisys 2010, June 15-18, 2010, San Fransisco, CA, 2010.
[36]
E. Miluzzo, N. D. Lane, K. Fodor, R. A. Peterson, H. Lu, M. Musolesi, S. B. Eisenman, X. Zheng, and A. T. Campbell. Sensing meets mobile social networks: the design, implementation and evaluation of the cenceme application. In SenSys, 2008.
[37]
S. Mishra, P. Elespuru, and S. Shakya. Mapreduce system over heterogeneous mobile devices. In SEUS, Newport Beach, CA, USA, 2009.
[38]
L. Moser, P. Melliar-Smith, P. Narasimhan, L. Tewksbury, and V. Kalogeraki. Eternal: Fault tolerance and live upgrades for distributed object systems. In Proceedings of the IEEE Information Survivability Conference, Hilton Head, SC, Jan 2000.
[39]
Nokia. N95 8gb device details. http://www.forum.nokia.com/devices/N95 8GB.
[40]
C. Ranger, R. Raghuraman, A. Penmetsa, G. Bradski, and C. Kozyrakis. Evaluating mapreduce for multi-core and multiprocessor systems. HPCA, 2007.
[41]
T. Salminen and J. Riekki. Lightweight middleware architecture for mobile phones. In PSC, Las Vegas, NV, Jun 2005.
[42]
A. Thiagarajan, L. Ravindranath, K. LaCurts, S. Madden, H. Balakrishnan, S. Toledo, and J. Eriksson. Vtrack: accurate, energy-aware road traffic delay estimation using mobile phones. In SenSys, USA, 2009. ACM.
[43]
V. Tuulos. Disco. http://discoproject.org/.
[44]
F. Wang, K. Ramamritham, and J. A. Stankovic. Determining redundancy levels for fault tolerant real-time systems. IEEE Trans. Comput., 44(2):292--301, 1995.

Cited By

View all
  • (2016)An SDN-Assisted Framework for Optimal Deployment of MapReduce Functions in WSNsIEEE Transactions on Mobile Computing10.1109/TMC.2015.249658215:9(2165-2178)Online publication date: 1-Sep-2016
  • (2016)Towards Peer-to-Peer Solution for Utilization of Volunteer Resources to Provide Computation-as-a-Service2016 IEEE 30th International Conference on Advanced Information Networking and Applications (AINA)10.1109/AINA.2016.46(1146-1153)Online publication date: Mar-2016
  • (2016)MapReduce Parallel Programming ModelInternational Journal of Parallel Programming10.1007/s10766-015-0395-044:4(832-866)Online publication date: 1-Aug-2016
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
DEBS '11: Proceedings of the 5th ACM international conference on Distributed event-based system
July 2011
418 pages
ISBN:9781450304238
DOI:10.1145/2002259
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. distributed systems
  2. mapreduce
  3. mobile systems
  4. real-time

Qualifiers

  • Research-article

Conference

DEBS '11

Acceptance Rates

DEBS '11 Paper Acceptance Rate 23 of 95 submissions, 24%;
Overall Acceptance Rate 145 of 583 submissions, 25%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2016)An SDN-Assisted Framework for Optimal Deployment of MapReduce Functions in WSNsIEEE Transactions on Mobile Computing10.1109/TMC.2015.249658215:9(2165-2178)Online publication date: 1-Sep-2016
  • (2016)Towards Peer-to-Peer Solution for Utilization of Volunteer Resources to Provide Computation-as-a-Service2016 IEEE 30th International Conference on Advanced Information Networking and Applications (AINA)10.1109/AINA.2016.46(1146-1153)Online publication date: Mar-2016
  • (2016)MapReduce Parallel Programming ModelInternational Journal of Parallel Programming10.1007/s10766-015-0395-044:4(832-866)Online publication date: 1-Aug-2016
  • (2016)Task scheduling for MapReduce in heterogeneous networksCluster Computing10.1007/s10586-015-0503-319:1(197-210)Online publication date: 1-Mar-2016
  • (2016)Intelligent Urban Data Monitoring for Smart CitiesMachine Learning and Knowledge Discovery in Databases10.1007/978-3-319-46131-1_23(177-192)Online publication date: 3-Sep-2016
  • (2015)SAMESFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-014-4138-y9:1(128-141)Online publication date: 1-Feb-2015
  • (2015)Task Scheduling for MapReduce Based on Heterogeneous NetworksHuman Centered Computing10.1007/978-3-319-15554-8_23(278-289)Online publication date: 4-Mar-2015
  • (2014)The impact of resource heterogeneity on the timeliness of hard real-time complex jobsProceedings of the 7th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/2674396.2674469(1-8)Online publication date: 27-May-2014
  • (2014)Exploiting Heterogeneous Data SourcesProceedings of the First International Conference on Applied Algorithms - Volume 832110.1007/978-3-319-04126-1_3(29-36)Online publication date: 13-Jan-2014
  • (2013)Runtimes and optimizations for map reduceIETE Technical Review10.4103/0256-4602.12568030:6(506)Online publication date: 2013

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media