Extending Hadoop's Yarn Scheduler Load Simulator with a highly realistic network & traffic model | IEEE Conference Publication | IEEE Xplore
Scheduled Maintenance: On Monday, 27 January, the IEEE Xplore Author Profile management portal will undergo scheduled maintenance from 9:00-11:00 AM ET (1400-1600 UTC). During this time, access to the portal will be unavailable. We apologize for any inconvenience.

Extending Hadoop's Yarn Scheduler Load Simulator with a highly realistic network & traffic model


Abstract:

Research on accelerating big-data applications can be divided into job scheduling and flow scheduling. Job scheduling focuses on the timely and spacial placement of jobs ...Show More

Abstract:

Research on accelerating big-data applications can be divided into job scheduling and flow scheduling. Job scheduling focuses on the timely and spacial placement of jobs on execution units. Flow scheduling, on the other hand, concentrates on routing of flows originating from actively running jobs. Although both job scheduling and flow scheduling work on accelerating big-data applications, their view on the problem and the available information is very different. We propose a new simulation tool to evaluate ideas that jointly solve the job and flow scheduling problem for big-data applications. Our tool combines the Yarn Scheduler Load Simulator with the distributed network emulator MaxiNet. With our work, the interdependency between the network and the jobs running on top of it can be included into the evaluation of new ideas, leveraging research on big-data applications with joint job and flow scheduling.
Date of Conference: 13-17 April 2015
Date Added to IEEE Xplore: 04 June 2015
Electronic ISBN:978-1-4799-7899-1
Conference Location: London, UK

References

References is not available for this document.