Abstract
MapReduce is a programming model and an associated implementation for processing and generating large data sets. Providing MapReduce as a service is the development future trend. By leveraging the game theory, this paper proposes a scheduling algorithm to deal with the competition for resources between multiple jobs in MapReduce. Firstly, we present a model that could estimate job executing time, and then a utility function of job and an optimization objective are brought forward; thirdly, we present a game model to solve the optimization problem. The proof and the solution are also present. Finally, we implement the algorithm and experiment it in a hadoop cluster. The result shows the present algorithm could schedule jobs rational.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Armbrust, M., et al.: Above the Clouds: A berkeley view of cloud computing. University of California at Berkeley (2009)
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
Mingchu, L., et al.: Grid Resource Allocation Model Based on Incomplete Information Game. Journal of Software 23(2), 428–438 (2012)
Cong, W., et al.: Resource Allocation Model to Achieve Optimize Profit in Virtual Data Center. Journal of Northeastern University 32(11), 1546–1549 (2011)
Hadoop CapacityScheduler, http://hadoop.apache.org/common/docs/current/ capacity_ scheduler . html.
Hadoop FairScheduler, http://hadoop.apache.org/common/docs/current/fair_scheduler.html
Sandholm, T., Lai, K.: Dynamic Proportional Share Scheduling in Hadoop. In: Frachtenberg, E., Schwiegelshohn, U. (eds.) JSSPP 2010. LNCS, vol. 6253, pp. 110–131. Springer, Heidelberg (2010)
Xicheng, D., Ying, W., Huaming, L.: Scheduling Mixed Real-time and Non-real-time Applications in MapReduce Environment. In: 17th IEEE International Conference on Parallel and Distributed Systems (ICPADS), Tainan, pp. 9–16 (2011)
Polo, J., et al.: Performance-Driven Task Co-Scheduling for MapReduce Environments. In: 2010 IEEE/IFIP Network Operations and Management Symposium - NOMS, pp. 373–380 (2010)
Kc, K., Anyanwu, K.: Scheduling Hadoop Jobs to Meet Deadlines. In: Proceedings of the 2010 IEEE 2nd International Conference on Cloud Computing Technology and Science (CloudCom 2010), pp. 388–392 (2010)
You, H., Yang, C., Huang, J.: A load-aware scheduler for MapReduce framework in heterogeneous cloud environments. In: Proceedings of the ACM Symposium on Applied Computing, pp. 127–132 (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wan, C., Wang, C., Yuan, Y., Wang, H. (2013). Game-Based Scheduling Algorithm to Achieve Optimize Profit in MapReduce Environment. In: Huang, DS., Bevilacqua, V., Figueroa, J.C., Premaratne, P. (eds) Intelligent Computing Theories. ICIC 2013. Lecture Notes in Computer Science, vol 7995. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39479-9_28
Download citation
DOI: https://doi.org/10.1007/978-3-642-39479-9_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39478-2
Online ISBN: 978-3-642-39479-9
eBook Packages: Computer ScienceComputer Science (R0)