Skip to main content
SpringerLink
Log in

Resource Scheduling in Data-Centric Systems

  • Chapter
  • First Online:
Handbook on Data Centers

Abstract

Effective resource scheduling is a fundamental issue for achieving high performance in various computer systems. The goal of resource scheduling is to arrange the best location of each resource and determine the most appropriate sequence of job execution, while satisfying certain constraints or optimizations. Although the topic of resource scheduling has been widely investigated for several decades, it is still a research hotspot as new paradigms continue to emerge, such as grid computing, cloud computing, big data analytics, and so on.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    These statistics were released on the year of 2012.

References

  1. Schwiegelshohn, U., Badia, R.M., Bubak, M., Danelutto, M., Dustdar, S., Gagliardi, F., Geiger, A., Hluchy, L., Kranzlmüller, D., Laure, E., et al.: Perspectives on grid computing. Future Generation Computer Systems 26(8) (2010) 1104–1115

    Google Scholar 

  2. Xhafa, F., Abraham, A.: Computational models and heuristic methods for grid scheduling problems. Future generation computer systems 26(4) (2010) 608–621

    Google Scholar 

  3. Armbrust, M., Fox, A., Griffith, R., Joseph, A.D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., et al.: A view of cloud computing. Communications of the ACM 53(4) (2010) 50–58

    Google Scholar 

  4. Foster, I., Zhao, Y., Raicu, I., Lu, S.: Cloud computing and grid computing 360-degree compared. In: Grid Computing Environments Workshop, 2008. GCE’08, Ieee (2008) 1–10

    Google Scholar 

  5. Dittrich, J., Quiané-Ruiz, J.A.: Efficient big data processing in hadoop mapreduce. Proceedings of the VLDB Endowment 5(12) (2012) 2014–2015

    Google Scholar 

  6. Madden, S.: From databases to big data. Internet Computing, IEEE 16(3) (2012) 4–6

    Google Scholar 

  7. Amazon Elastic Compute Cloud: http://aws.amazon.com/ec2/

  8. Irwin, D., Chase, J., Grit, L., Yumerefendi, A., Becker, D., Yocum, K.G.: Sharing networked resources with brokered leases. resource 6 (2006) 6

    Google Scholar 

  9. Ciurana, E.: Developing with Google App Engine. Apress (2009)

    Google Scholar 

  10. Rackspace: http://www.rackspace.com

  11. Windows Azure: http://www.windowsazure.com/

  12. Bryant, R.E.: Data-intensive supercomputing: The case for disc. (2007)

    Google Scholar 

  13. Garg, S.K., Yeo, C.S., Anandasivam, A., Buyya, R.: Environment-conscious scheduling of hpc applications on distributed cloud-oriented data centers. Journal of Parallel and Distributed Computing 71(6) (2011) 732–749

    Article  MATH  Google Scholar 

  14. Gorton, I., Gracio, D.K.: Data-intensive computing: A challenge for the 21st century. Data-Intensive Computing: Architectures, Algorithms, and Applications (2012) 3

    Google Scholar 

  15. White, T.: Hadoop - The Definitive Guide. O’Reilly (2009)

    Google Scholar 

  16. Dean, J., Ghemawat, S.: Mapreduce: Simplified data processing on large clusters. In: OSDI. (2004) 137–150

    Google Scholar 

  17. Chen, Y.: Workload-driven design and evaluation of large- scale data-centric systems (May, 09 2012)

    Google Scholar 

  18. Reiss, C., Tumanov, A., Ganger, G.R., Katz, R.H., Kozuch, M.A.: Heterogeneity and dynamicity of clouds at scale: Google trace analysis. In: SoCC. (2012) 7

    Google Scholar 

  19. Macías, M., Guitart, J.: A genetic model for pricing in cloud computing markets. In: SAC, ACM (2011) 113–118

    Google Scholar 

  20. Niyato, D., Vasilakos, A.V., Zhu, K.: Resource and revenue sharing with coalition formation of cloud providers: Game theoretic approach. In: CCGRID, IEEE (2011) 215–224

    Google Scholar 

  21. Lin, W.Y., Lin, G.Y., Wei, H.Y.: Dynamic auction mechanism for cloud resource allocation. In: CCGRID, IEEE (2010) 591–592

    Google Scholar 

  22. Lucas-Simarro, J.L., Moreno-Vozmediano, R., Montero, R.S., Llorente, I.M.: Dynamic placement of virtual machines for cost optimization in multi-cloud environments. In: HPCS, IEEE (2011) 1–7

    Google Scholar 

  23. Wolf, J., Balmin, A., Rajan, D., Hildrum, K., Khandekar, R., Parekh, S., Wu, K.L., Vernica, R.: On the optimization of schedules for mapreduce workloads in the presence of shared scans. The VLDB Journal 21(5) (2012) 589–609

    Article  Google Scholar 

  24. Chang, H., Kodialam, M.S., Kompella, R.R., Lakshman, T.V., Lee, M., Mukherjee, S.: Scheduling in mapreduce-like systems for fast completion time. In: INFOCOM, IEEE (2011) 3074–3082

    Google Scholar 

  25. Wolf, J.L., Rajan, D., Hildrum, K., Khandekar, R., Kumar, V., Parekh, S., Wu, K.L., Balmin, A.: Flex: A slot allocation scheduling optimizer for mapreduce workloads. In: Middleware. (2010) 1–20

    Google Scholar 

  26. Kliazovich, D., Bouvry, P., Khan, S.U.: DENS: data center energy-efficient network-aware scheduling. Cluster Computing 16(1) (2013) 65–75

    Article  Google Scholar 

  27. Chen, Y., Alspaugh, S., Borthakur, D., Katz, R.H.: Energy efficiency for large-scale mapreduce workloads with significant interactive analysis. In: EuroSys, ACM (2012) 43–56

    Google Scholar 

  28. Wang, L., Khan, S.U.: Review of performance metrics for green data centers: a taxonomy study. The Journal of Supercomputing 63(3) (2013) 639–656

    Article  MathSciNet  Google Scholar 

  29. Wang, L., Khan, S.U., Chen, D., Kolodziej, J., Ranjan, R., Xu, C.Z., Zomaya, A.Y.: Energy-aware parallel task scheduling in a cluster. Future Generation Comp. Syst 29(7) (2013) 1661–1670

    Google Scholar 

  30. Isard, M., Prabhakaran, V., Currey, J., Wieder, U., Talwar, K., Goldberg, A.: Quincy: fair scheduling for distributed computing clusters. In: SOSP, ACM (2009) 261–276

    Google Scholar 

  31. Zaharia, M., Borthakur, D., Sarma, J.S., Elmeleegy, K., Shenker, S., Stoica, I.: Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling. In: EuroSys. (2010) 265–278

    Google Scholar 

  32. Borthakur, D., Gray, J., Sarma, J.S., Muthukkaruppan, K., Spiegelberg, N., Kuang, H., Ranganathan, K., Molkov, D., Menon, A., Rash, S., Schmidt, R., Aiyer, A.S.: Apache hadoop goes realtime at facebook. In: SIGMOD Conference. (2011) 1071–1080

    Google Scholar 

  33. Ousterhout, K., Wendell, P., Zaharia, M., Stoica, I.: Sparrow: Scalable scheduling for sub-second parallel jobs. Technical Report UCB/EECS-2013-29, EECS Department, University of California, Berkeley (April 2013)

    Google Scholar 

  34. Buyya, R., Yeo, C.S., Venugopal, S., Broberg, J., Brandic, I.: Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Future Generation Comp. Syst 25(6) (2009) 599–616

    Google Scholar 

  35. Delimitrou, C., Kozyrakis, C.: Paragon: QoS-aware scheduling for heterogeneous datacenters. In: ASPLOS. (2013) 77–88

    Google Scholar 

  36. Vasic, N., Novakovic, D.M., Miucin, S., Kostic, D., Bianchini, R.: Dejavu: Accelerating resource allocation in virtualized environments architectural support for programming languages and operating systems, (17th ASPLOS'12). In: Proceedings of the 17th International Conference on, ACM Press (2012) 423–436

    Google Scholar 

  37. Zhu, X., Young, D., Watson, B.J., Wang, Z., Rolia, J., Singhal, S., McKee, B., Hyser, C., Gmach, D., Gardner, R., Christian, T., Cherkasova, L.: 1000 islands: an integrated approach to resource management for virtualized data centers. Cluster Computing 12(1) (2009) 45–57

    Article  Google Scholar 

  38. Kale, L.V., Kumar, S., Potnuru, M., DeSouza, J., Bandhakavi, S.: Faucets: Efficient resource allocation on the computational grid. In: Proceedings of the 2004 International Conference on Parallel Processing (33th ICPP'04), Montreal, Quebec, Canada, IEEE Computer Society (August 2004) 396–405

    Google Scholar 

  39. Rodero-Merino, L., Caron, E., Muresan, A., Desprez, F.: Using clouds to scale grid resources: An economic model. Future Generation Computer Systems 28(4) (2012) 633 – 646

    Google Scholar 

  40. Kang, Z., Wang, H.: A novel approach to allocate cloud resource with different performance traits. In: Proceedings of the 2013 IEEE International Conference on Services Computing. SCC '13, Washington, DC, USA, IEEE Computer Society (2013) 128–135

    Google Scholar 

  41. Sim, K.M.: Towards complex negotiation for cloud economy. In: Advances in Grid and Pervasive Computing. Springer (2010) 395–406

    Google Scholar 

  42. Garg, S.K., Vecchiola, C., Buyya, R.: Mandi: a market exchange for trading utility and cloud computing services. The Journal of Supercomputing 64(3) (2013) 1153–1174

    Article  Google Scholar 

  43. Izakian, H., Abraham, A., Ladani, B.T.: An auction method for resource allocation in computational grids. Future Generation Comp. Syst 26(2) (2010) 228–235

    Google Scholar 

  44. Zaman, S., Grosu, D.: Combinatorial auction-based allocation of virtual machine instances in clouds. In: CloudCom, IEEE (2010) 127–134

    Google Scholar 

  45. Samimi, P., Patel, A.: Review of pricing models for grid & cloud computing. In: Computers & Informatics (ISCI), 2011 IEEE Symposium on, IEEE (2011) 634–639

    Google Scholar 

  46. Wang, Q., Ren, K., Meng, X.: When cloud meets ebay: Towards effective pricing for cloud computing. In Greenberg, A.G., Sohraby, K., eds.: INFOCOM, IEEE (2012) 936–944

    Google Scholar 

  47. Meng, X., Isci, C., Kephart, J.O., Zhang, L., Bouillet, E., Pendarakis, D.E.: Efficient resource provisioning in compute clouds via VM multiplexing. In Parashar, M., Figueiredo, R.J.O., Kiciman, E., eds.: ICAC, ACM (2010) 11–20

    Google Scholar 

  48. Zhang, W., Qian, H., Wills, C.E., Rabinovich, M.: Agile resource management in a virtualized data center. In Adamson, A., Bondi, A.B., Juiz, C., Squillante, M.S., eds.: WOSP/SIPEW, ACM (2010) 129–140

    Google Scholar 

  49. Garg, S.K., Gopalaiyengar, S.K., Buyya, R.: SLA-based resource provisioning for heterogeneous workloads in a virtualized cloud datacenter. In Xiang, Y., Cuzzocrea, A., Hobbs, M., Zhou, W., eds.: ICA3PP (1). Volume 7016 of Lecture Notes in Computer Science., Springer (2011) 371–384

    Google Scholar 

  50. Urgaonkar, B., Shenoy, P., Chandra, A., Goyal, P.: Dynamic provisioning of multi-tier internet applications. In: Autonomic Computing, 2005. ICAC 2005. Proceedings. Second International Conference on, IEEE (2005) 217–228

    Google Scholar 

  51. Gong, Z., Gu, X., Wilkes, J.: Press: Predictive elastic resource scaling for cloud systems. In: Network and Service Management (CNSM), 2010 International Conference on, IEEE (2010) 9–16

    Google Scholar 

  52. Padala, P., Hou, K.Y., Shin, K.G., Zhu, X., Uysal, M., Wang, Z., Singhal, S., Merchant, A.: Automated control of multiple virtualized resources. In: Proceedings of the 4th ACM European conference on Computer systems, ACM (2009) 13–26

    Google Scholar 

  53. Xu, J., Zhao, M., Fortes, J., Carpenter, R., Yousif, M.: Autonomic resource management in virtualized data centers using fuzzy logic-based approaches. Cluster Computing 11(3) (2008) 213–227

    Article  Google Scholar 

  54. Gmach, D., Krompass, S., Scholz, A., Wimmer, M., Kemper, A.: Adaptive quality of service management for enterprise services. ACM Transactions on the Web (TWEB) 2(1) (2008) 8

    Google Scholar 

  55. Beloglazov, A., Abawajy, J., Buyya, R.: Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing. Future Generation Computer Systems 28(5) (2012) 755–768

    Article  Google Scholar 

  56. Xiong, K., Perros, H.G.: SLA-based resource allocation in cluster computing systems. In: IPDPS, IEEE (2008) 1–12

    Google Scholar 

  57. Gu, J., Hu, J., Zhao, T., Sun, G.: A new resource scheduling strategy based on genetic algorithm in cloud computing environment. Journal of Computers 7(1) (2012) 42–52

    Article  Google Scholar 

  58. Hu, J., Gu, J., Sun, G., Zhao, T.: A scheduling strategy on load balancing of virtual machine resources in cloud computing environment. In: Parallel Architectures, Algorithms and Programming (PAAP), 2010 Third International Symposium on, IEEE (2010) 89–96

    Google Scholar 

  59. Vavilapalli, V.K., Murthy, A.C., Douglas, C., Agarwal, S., Konar, M., Evans, R., Graves, T., Lowe, J., Shah, H., Seth, S., Saha, B., Curino, C., O’Malley, O., Radia, S., Reed, B., Baldeschwieler, E.: Apache hadoop YARN: Yet another resource negotiator. In: SoCC. (2013)

    Google Scholar 

  60. Zaharia, M., Borthakur, D., Sarma, J.S., Shenker, S., Stoica, I.: Job scheduling for multi-user mapreduce clusters. Technical Report No. UCB/EECS-2009-55, Univ. of Calif., Berkeley, CA (April 2009)

    Google Scholar 

  61. Zhang, X., Zhong, Z., Feng, S., Tu, B., Fan, J.: Improving data locality of mapreduce by scheduling in homogeneous computing environments. In: Parallel and Distributed Processing with Applications (ISPA), 2011 IEEE 9th International Symposium on, IEEE (2011) 120–126

    Google Scholar 

  62. Kc, K., Anyanwu, K.: Scheduling hadoop jobs to meet deadlines. In: Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on, IEEE (2010) 388–392

    Google Scholar 

  63. Tang, Z., Zhou, J., Li, K., Li, R.: MTSD: A task scheduling algorithm for mapreduce base on deadline constraints. In: IPDPS Workshops, IEEE Computer Society (2012) 2012–2018

    Google Scholar 

  64. Schwiegelshohn, U., Tchernykh, A.: Online scheduling for cloud computing and different service levels. In: Proc. 9th High-Performance Grid & Cloud Computing – 9th HPGC'12, Proc. IEEE International Parallel and Distributed Processing Symposium Workshops & PhD Forum (26th IPDPS'12), IEEE Computer Society (2012) 1067–1074

    Google Scholar 

  65. Venugopal, S., Buyya, R.: An scp-based heuristic approach for scheduling distributed data-intensive applications on global grids. Journal of Parallel and Distributed Computing 68(4) (2008) 471–487

    Article  MATH  Google Scholar 

  66. Chang, R.S., Chang, J.S., Lin, P.S.: An ant algorithm for balanced job scheduling in grids. Future Generation Computer Systems 25(1) (2009) 20–27

    Article  Google Scholar 

  67. Kolodziej, J., Khan, S.U., Xhafa, F.: Genetic algorithms for energy-aware scheduling in computational grids. In: P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC), 2011 International Conference on, IEEE (2011) 17–24

    Google Scholar 

  68. Lee, Y.H., Leu, S., Chang, R.S.: Improving job scheduling algorithms in a grid environment. Future generation computer systems 27(8) (2011) 991–998

    Google Scholar 

  69. Samuel, T.K., Baer, T., Brook, R.G., Ezell, M., Kovatch, P.: Scheduling diverse high performance computing systems with the goal of maximizing utilization. In: High Performance Computing (HiPC), 2011 18th International Conference on, IEEE (2011) 1–6

    Google Scholar 

  70. Balman, M.: Failure-awareness and dynamic adaptation in data scheduling (November 14 200–8)

    Google Scholar 

  71. Chowdhury, M., Zaharia, M., Ma, J., Jordan, M.I., Stoica, I.: Managing data transfers in computer clusters with orchestra. In: SIGCOMM, ACM (2011) 98–109

    Google Scholar 

  72. Seo, S., Jang, I., Woo, K., Kim, I., Kim, J.S., Maeng, S.: Hpmr: Prefetching and pre-shuffling in shared mapreduce computation environment. In: Cluster Computing and Workshops, 2009. CLUSTER’09. IEEE International Conference on, IEEE (2009) 1–8

    Google Scholar 

  73. Çatalyürek, Ü.V., Kaya, K., Uçar, B.: Integrated data placement and task assignment for scientific workflows in clouds. In: Proceedings of the fourth international workshop on Data-intensive distributed computing, ACM (2011) 45–54

    Google Scholar 

  74. Xie, J., Yin, S., Ruan, X., Ding, Z., Tian, Y., Majors, J., Manzanares, A., Qin, X.: Improving mapreduce performance through data placement in heterogeneous hadoop clusters. In: Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW), 2010 IEEE International Symposium on, IEEE (2010) 1–9

    Google Scholar 

  75. Zeng, W., Zhao, Y., Ou, K., Song, W.: Research on cloud storage architecture and key technologies. In: Proceedings of the 2nd International Conference on Interaction Sciences: Information Technology, Culture and Human, ACM (2009) 1044–1048

    Google Scholar 

  76. Abad, C.L., Lu, Y., Campbell, R.H.: DARE: Adaptive data replication for efficient cluster scheduling. In: Proc. '11 IEEE International Conference on Cluster Computing (13th CLUSTER'11), Austin, TX, USA, IEEE Computer Society (September 2011) 159–168

    Google Scholar 

  77. Castillo, C., Tantawi, A.N., Arroyo, D., Steinder, M.: Cost-aware replication for dataflows. In: NOMS, IEEE (2012) 171–178

    Google Scholar 

  78. Chervenak, A.L., Deelman, E., Livny, M., Su, M.H., Schuler, R., Bharathi, S., Mehta, G., Vahi, K.: Data placement for scientific applications in distributed environments. In: GRID, IEEE Computer Society (2007) 267–274

    Google Scholar 

  79. Chen, Y., Ganapathi, A.S., Griffith, R., Katz, R.H.: Analysis and lessons from a publicly available google cluster trace. Technical Report UCB/EECS-2010-95, EECS Department, University of California, Berkeley (Jun 2010)

    Google Scholar 

  80. Chen, Y., Ganapathi, A.S., Griffith, R., Katz, R.H.: Towards understanding cloud performance tradeoffs using statistical workload analysis and replay. University of California at Berkeley, Technical Report No. UCB/EECS-2010-81 (2010)

    Google Scholar 

  81. Stuer, G., Vanmechelen, K., Broeckhove, J.: A commodity market algorithm for pricing substitutable grid resources. Future Generation Comp. Syst 23(5) (2007) 688–701

    Google Scholar 

  82. Teng, F., Magoulès, F.: Resource pricing and equilibrium allocation policy in cloud computing. In: CIT, IEEE Computer Society (2010) 195–202

    Google Scholar 

  83. Eymann, T., Reinicke, M., Villanueva, O.A., Vidal, P.A., Freitag, F., Moldes, L.N.: Decentralized resource allocation in application layer networks. In: CCGrid, IEEE (May 12 2003) 645–650

    Google Scholar 

  84. Padala, P., Harrison, C., Pelfort, N., Jansen, E., Frank, M.P., Chokkareddy, C.: OCEAN: The open computation exchange and arbitration network, A market approach to meta computing. In: Proc. 2nd International Symposium on Parallel and Distributed Computing (2nd ISPDC'03), Ljubljana, Slovenia, IEEE Computer Society (October 2003) 185–192

    Google Scholar 

  85. Peterson, L., Anderson, T., Culler, D., Roscoe, T.: PlanetLab: A Blueprint for Introducing Disruptive Technology into the Internet. In: First ACM Workshop on Hot Topics in Networks, Association for Computing Machinery (October 2002) Available from http://www.planet-lab.org/pdn/pdn02-001.pdf.

  86. Ghodsi, A., Zaharia, M., Hindman, B., Konwinski, A., Shenker, S., Stoica, I.: Dominant resource fairness: Fair allocation of multiple resource types. Technical report, University of California, Berkeley (2011)

    Google Scholar 

  87. Mihailescu, M., Teo, Y.M.: Dynamic resource pricing on federated clouds. In: CCGRID, IEEE (2010) 513–517

    Google Scholar 

  88. Dutreilh, X., Rivierre, N., Moreau, A., Malenfant, J., Truck, I.: From data center resource allocation to control theory and back. In: Proc. IEEE International Conference on Cloud Computing (3rd IEEE CLOUD'10). (2010) 410–417

    Google Scholar 

  89. Buyya, R., Garg, S.K., Calheiros, R.N.: SLA-oriented resource provisioning for cloud computing: Challenges, architecture, and solutions. In: Cloud and Service Computing (CSC). (January 21 2012)

    Google Scholar 

  90. Gandhi, A., Chen, Y., Gmach, D., Arlitt, M.F., Marwah, M.: Minimizing data center SLA violations and power consumption via hybrid resource provisioning. In: IGCC, IEEE Computer Society (2011) 1–8

    Google Scholar 

  91. Birke, R., Chen, L.Y., Smirni, E.: Data centers in the cloud: A large scale performance study. In: Proc. 2012 IEEE Fifth International Conference on Cloud Computing (5th IEEE CLOUD'12). (June 2012) 336–343

    Google Scholar 

  92. Gao, Y., Rong, H., Huang, J.Z.: Adaptive grid job scheduling with genetic algorithms. Future Generation Computer Systems 21(1) (2005) 151–161

    Article  Google Scholar 

  93. Fidanova, S.: Simulated annealing for grid scheduling problem. In: Modern Computing, 2006. JVA’06. IEEE John Vincent Atanasoff 2006 International Symposium on, IEEE (2006) 41–45

    Google Scholar 

  94. neng Chen, W., 0003, J.Z.: An ant colony optimization approach to a grid workflow scheduling problem with various qoS requirements. IEEE Transactions on Systems, Man, and Cybernetics, Part C 39(1) (2009) 29–43

    Google Scholar 

  95. Braun, T.D., Siegel, H.J., Beck, N., Bölöni, L., Maheswaran, M., Reuther, A.I., Robertson, J.P., Theys, M.D., Yao, B., Hensgen, D.A., Freund, R.F.: A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. J. Parallel Distrib. Comput 61(6) (2001) 810–837

    Google Scholar 

  96. Dong, F., Akl, S.G.: Scheduling algorithms for grid computing: State of the art and open problems. School of Computing, Queens University, Kingston, Ontario (2006)

    Google Scholar 

  97. Ren, Z., Wan, J., Shi, W., Xu, X., Zhou, M.: Workload analysis, implications and optimization on a production hadoop cluster: A case study on taobao. IEEE Transactions on Services Computing (2013)

    Google Scholar 

  98. Isard, M., Budiu, M., Yu, Y., Birrell, A., Fetterly, D.: Dryad: distributed data-parallel programs from sequential building blocks. In: EuroSys, ACM (2007) 59–72

    Google Scholar 

  99. Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: Proceedings of the 2nd USENIX conference on Hot topics in cloud computing. (2010) 10–10

    Google Scholar 

  100. Sandholm, T., Lai, K.: Dynamic proportional share scheduling in Hadoop. In Frachtenberg, E., Schwiegelshohn, U., eds.: Job Scheduling Strategies for Parallel Processing. Springer Verlag (2010) 110–131

    Google Scholar 

  101. Wang, L., von Laszewski, G., Dayal, J., He, X., Younge, A.J., Furlani, T.R.: Towards thermal aware workload scheduling in a data center. In: ISPAN, IEEE Computer Society (2009) 116–122

    Google Scholar 

  102. Ranganathan, K., Foster, I.T.: Decoupling computation and data scheduling in distributed data-intensive applications. In: HPDC, IEEE Computer Society (2002) 352–358

    Google Scholar 

  103. Guo, D., Li, M., Jin, H., Shi, X., Lu, L.: Managing and aggregating data transfers in data centers (2013)

    Google Scholar 

  104. Al-Fares, M., Radhakrishnan, S., Raghavan, B., Huang, N., Vahdat, A.: Hedera: Dynamic flow scheduling for data center networks. In: NSDI, USENIX Association (2010) 281–296

    Google Scholar 

  105. Sun, N.H., Xing, J., Huo, Z.G., Tan, G.M., Xiong, J., Li, B., Ma, C.: Dawning nebulae: a petaflops supercomputer with a heterogeneous structure. Journal of Computer Science and Technology 26(3) (2011) 352–362

    Article  Google Scholar 

  106. : Top500 list

    Google Scholar 

  107. Lumb, I., Smith, C.: Scheduling attributes and platform lsf. In: Grid resource management. Springer (2004) 171–182

    Google Scholar 

  108. Taobao: http://www.taobao.com

  109. Chaiken, R., Jenkins, B., Larson, P.Å., Ramsey, B., Shakib, D., Weaver, S., Zhou, J.: Scope: easy and efficient parallel processing of massive data sets. Proceedings of the VLDB Endowment 1(2) (2008) 1265–1276

    Article  Google Scholar 

  110. Ghemawat, S., Gobioff, H., Leung, S.T.: The google file system. In: ACM SIGOPS Operating Systems Review. Volume 37., ACM (2003) 29–43

    Google Scholar 

  111. Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: Mass Storage Systems and Technologies (MSST), 2010 IEEE 26th Symposium on, IEEE (2010) 1–10

    Google Scholar 

  112. Agarwal, S., Dunagan, J., Jain, N., Saroiu, S., Wolman, A., Bhogan, H.: Volley: Automated data placement for geo-distributed cloud services. In: NSDI. (2010) 17–32

    Google Scholar 

  113. McKeown, N.: Software-defined networking. INFOCOM keynote talk, Apr (2009)

    Google Scholar 

  114. Liu, D., Lee, Y.H.: Pfair scheduling of periodic tasks with allocation constraints on multiple processors. In: IPDPS. (2004)

    Google Scholar 

  115. Lee, J., Easwaran, A., Shin, I.: LLF schedulability analysis on multiprocessor platforms. In: IEEE Real-Time Systems Symposium. (2010) 25–36

    Google Scholar 

  116. Islam, S., Keung, J., Lee, K., Liu, A.: Empirical prediction models for adaptive resource provisioning in the cloud. Future Generation Computer Systems 28(1) (2012) 155–162

    Article  Google Scholar 

  117. Wilkes, J., Reiss, C.: Details of the clusterdata-2011-1 trace (2011)

    Google Scholar 

Download references

Acknowledgement

We thank Raymond Darnell Lemon for his valuable comments on the early version of this chapter. This research is supported by NSF of Zhejiang (LQ12F02002), NSF of China (No. 61202094), Science and Technology Planning Project of Zhejiang Province (No.2010C13022). Xiaohong Zhang is supported by Ph.D. foundation of Henan Polytechnic University (No. B2012-099). Weisong Shi is in part supported by the Introduction of Innovative R&D team program of Guangdong Province (NO. 201001D0104726115), Hangzhou Dianzi University, and the NSF Career Award CCF-0643521.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China

    Zujie Ren

  2. Shenzhen Institutes of Advanced Technology, Chinese Academy of Science, Shenzhen, China

    Xiaohong Zhang

  3. Department of Computer Science, Wayne State University, Detroit, USA

    Weisong Shi

Authors
  1. Zujie Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaohong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weisong Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zujie Ren .

Editor information

Editors and Affiliations

  1. Department of Electrical and Computer Engineering, North Dakota State University, Fargo, North Dakota, USA

    Samee U. Khan

  2. School of Information Technologies, The University of Sydney, Sydney, New South Wales, Australia

    Albert Y. Zomaya

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media New York

About this chapter

Cite this chapter

Ren, Z., Zhang, X., Shi, W. (2015). Resource Scheduling in Data-Centric Systems. In: Khan, S., Zomaya, A. (eds) Handbook on Data Centers. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-2092-1_46

Download citation

  • DOI: https://doi.org/10.1007/978-1-4939-2092-1_46

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4939-2091-4

  • Online ISBN: 978-1-4939-2092-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation