Market-based on demand scheduling (MBoDS) in co-operative grid environment

Computational and data scientists at universities have different job resource requirements. Most universities maintain a set of shared resources to support these computational needs. Access to these resources are often free and the access policy is First Come First Serve (FCFS). However, FCFS policies on shared resources often lead to sub-optimal value from the organization's point of view as different jobs contribute different values to the users. Furthermore, the set of resources at a single institution may fail to satisfy the diverse needs of the institutions researchers. We argue the solution is differentiated quality of service (QoS) based on the user's willingness to pay to rationalize resource usage and federation of university resources to improve both the size of the resource pool as well as the diversity of resources. The proposed XSEDE Campus Bridging (CB) Shared Virtual Compute Facility (SVCF) [22] provides both. The CB-SVCF will be deployed using existing XSEDE Execution Management Services (EMS) and the XSEDE Global Federated File System (GFFS) [13].

Before deploying the CB-SVCF it is critical to understand and demonstrate the expected benefits to stakeholders under different load, pricing, and priority scenarios. We have developed a simulator to understand these trade-offs. In this paper we present the simulation results with two qualities of service and data traces from two universities for a month. Our result shows that MBoDS outperforms both FCFS and simple priority scheduling policy for a single site both in terms of overall value and in terms of number of high priority jobs started within the predefined threshold.