CARE Resource Broker: A framework for scheduling and supporting virtual resource management
Introduction
Computational grid is a parallel and distributed system with a collection of computers that enables dynamic sharing, selection, aggregation and management of resources for collaborative problem solving [1]. The grid middleware provides protocols and functionalities to the customers and providers for grid [2]. The role of the grid metascheduler is very important in computational grid for the discovery of suitable resources for application execution, resource management and monitoring, and load balancing across grid resources. However, managing grid resources and scheduling applications to suitable resources is really a difficult task. A typical grid resource need to be installed with grid middleware, related software and libraries needed for application execution. In such a scenario, users come with their applications and look for suitable resources in a grid that possesses platform, software and libraries required for application execution. This leads to a situation that though potential computational resources are available but they are not able to execute the application due to the lack of the required execution environment. In addition to this, a grid scheduler can also fail to utilize a computational resource if it possesses a number of CPUs that is less than that of the required application. Summarizing the above, a typical grid scheduler may encounter the following scheduling scenarios.
- •
Application requires a number of CPUs that shall be satisfied by a single cluster.
- •
Application requires a number of CPUs that cannot be met by a single cluster.
- •
Application requires a completely different software environment that no cluster in the grid can provide.
However, the existing grid schedulers such as condor-G, Gridbus broker [3] and Gridway [4] have limited functionality for on-demand provisioning of suitable computing resource for job execution and it cannot schedule jobs to resources for the last two scenarios. Hence, there is a demand to develop a new strategy for grid scheduling that supports dynamic customization of software and operating environment, and dynamic deployment of grid related software across physical grid resources. The virtualization technology can be exploited to complement the grid scheduler to overcome the limitations and manage the application scheduling scenarios identified above.
Virtualization is a promising technology which provides software environments in the form of virtual machines dynamically and they can be seamlessly deployed over virtualization layer of the existing grid resource. Software execution environments can then be configured in the virtual machines to meet application requirements. Further, they provide improved security than grid as virtual machines are isolated from each other and from the host machine. Due to this fact, virtualization technologies are also used for constructing trusted virtual execution environment in P2P grids [5]. With the help of virtualization technology adopted in grid, a grid client can lease a grid resource, deploy grid based application, and customize a complete execution environment resulting in on-demand provisioning of resources. In such scenario, the grid scheduler requires appropriate mechanisms for VM deployment, creation of virtual clusters, and application execution in the virtual clusters. Existing grid schedulers lack mechanisms for dynamic creation of virtual grid resources in remote physical hosts to meet the application execution. Though, several attempts are being made to integrate VM technology with grid, a full fledged grid scheduling framework that supports virtualization technology is still unavailable.
In this paper, we propose and implement a grid metascheduling framework called CARE Resource Broker (CRB) that supports
- •
Application scheduling for scenario in which number of CPUs are not met by a single cluster, and in which required software environment are not available in any of the grid resources,
- •
dynamic creation and deployment of virtual machines and virtual clusters,
- •
design and development of virtual resource information system,
- •
dynamic creation of grid environment by installing a grid middleware on the fly in the virtual cluster, and
- •
dynamic preparation of software environment in that virtual cluster thereby creating a complete execution environment to facilitate application execution.
We also propose Virtual Resource Management Protocol (VRMP) that specifies set of mechanisms to be followed and describe messages to be exchanged while creating and deploying virtual clusters. It has been implemented as services namely Virtual Cluster Service (VCS), Virtual Machine Service (VMS) and Virtual Resource Aggregation Service (VRAS) for virtual resource management and monitoring across grid environment. Such resource brokering and scheduling strategies eventually allows more number of jobs to be scheduled across grid resources while increasing the utilization of underutilized resources in grid.
The rest of the paper is organized as follows: The literature survey related to our proposed research work is described in the Section 2. The Section 3 describes the architecture of CRB and its various components. In Section 4, we describe various scheduling scenarios that CRB investigates and corresponding brokering algorithm is explained. The VRM protocol and, the services VCS, VMS and VRAS are explained in the Section 5. The implementation of CRB and its experimental setup is presented in Section 6. Finally, we conclude our paper in Section 7 highlighting the scope of CRB in on-demand high performance computing.
Section snippets
Related work
There have been many attempts to integrate the Virtualization technology with Grid. We took inspiration from the following research works in this field to devise an architecture for our work.
Virtual workspace
Keahey et al. [6], [7] introduced the concept virtual workspace (VW) that aims to provide a customizable and controllable remote job execution environment for Grid. Virtual Workspaces supports unmanned installation of legacy applications, which can effectively reduce the deployment time. By
CARE Resource Broker
The CRB follows resource oriented metascheduling model [16] aimed at optimizing the utilization of resources in grid environment. Fig. 1 shows OGSA complaint layered architecture of CRB that defines various components for scheduling jobs to physical as well as virtualized grid resources.
Request Handler receives job requests submitted by the users and place it in job pool from where it processes them one by one. Controller is the master component which controls various activities of the CRB such
Resource scheduling algorithm
The brokering algorithm implemented in CRB classifies the application request based on its requirements in to three cases and adopts various brokering strategies to map applications onto a resource for each case of request.
Virtual Resource Management Protocol (VRMP)
Virtual Resource Management Protocol (VRMP) determines the functionalities of VRM. The Protocol specifies set of actions to be performed for creation and deployment of virtual resources in the physical resources. It supports the deployment of grid middleware, set of software libraries needed for job execution dynamically in virtual resources. It proposes several components to be present in VRM such as Virtual Resource Creator (VRC), Network Server to generate IP addresses for virtual machines,
Case study
The CRB has been implemented by the Centre for Advanced Computing Research and Education (CARE), Anna University, India. Our experimental setup consisting of two clusters one with two computing nodes and another one with four computing nodes. Both the clusters have their own head node. Scientific Linux 4.0 with 2.6 kernel is installed; Xen 3.0.2 is used for creating virtual machine and LVM has been installed in every computing node. Pentium IV 3.0.2 GHz, 2 GB of RAM, one hard disk (80 GB, 7200
Conclusion
In this paper, we developed a grid metascheduling framework that supports virtualization technology. It is very obvious that the creation and deployment of virtual cluster in remote grid resources followed by autoinstallation of globus middleware, deployment of software libraries required for application execution in the targeted resource leads to overhead with respect to time taken for completing the execution. However, with pre-configured guest OS images with globus middleware installed in
Acknowledgement
We sincerely thank the Ministry of Communication and Information Technology, Government of India for financially supporting this project as part of the research activities of Centre for Advanced Computing Research and Education of Anna University, India.
Thamarai Selvi Somasundaram is a Professor at Department of Information Technology, MIT Campus of Anna University. She is also the director of CARE and has successfully coordinated several research projects funded by various funding agencies across India. She has more than 27 years of teaching experience and 10 years of research experience. She has won many awards for her excellence in academic contributions. Her research areas include Grid Computing, Virtualization technologies, Neural
References (18)
- et al.
Cloud computing and emerging it platforms: Vision, hype, and reality for delivering computing as the 5th utility
Future Generation Computer Systems
(2009) - et al.
A decentralised model for scheduling independent tasks in federated grids
Future Generation Computer Systems
(2009) - et al.
A recursive architecture for hierarchical grid resource management
Future Generation Computer Systems
(2009) - I. Foster, C. Kesselman, J. Nick, S. Tuecke, The physiology of the grid: An open grid services architecture for...
- et al.
The anatomy of the grid: Enabling scalable virtual organizations
International Journal of Supercomputer Applications
(2001) - Gridbus Broker....
- et al.
A modular meta-scheduling architecture for interfacing with pre-WS and WS grid resource management services
Future Generation Computer Systems
(2007) - Deqing Zou, Weide Zheng, Jinjiu Long, Hai Jin, Xueguang Chen, Constructing trusted virtual execution environment in P2P...
- K. Keahey, I. Foster, T. Freeman, X. Zhang, D. Galron, Virtual Workspaces in the Grid, Europar, 2005, Lisbon,...
Cited by (37)
CLOUDRB: A framework for scheduling and managing High-Performance Computing (HPC) applications in science cloud
2014, Future Generation Computer SystemsCitation Excerpt :The main idea of their proposed work is to optimize the usage of Cloud resources in an efficient manner. CARE Resource Broker (CRB) [32] is a Grid metascheduler or resource broker that is deployed over the Globus Toolkit 4 (GT4) [33] middleware to manage the Grid and Virtualization enabled Grid resources. It addresses several scheduling scenarios encountered when Virtualization technology integrates with the Grid environment.
Resource requirement prediction using clone detection technique
2013, Future Generation Computer SystemsCitation Excerpt :On the basis of this monitoring data, job migration or reconfiguration may be required to maintain the granted QoS level. The CARE Resource Broker (CRB) proposed in [11] describes a scheduling technique based on unavailability of resources and the required execution environment in the resource providers. CRB is a Grid metascheduling framework that supports virtualization technology.
A Grid resource brokering strategy based on resource and network performance in Grid
2012, Future Generation Computer SystemsCitation Excerpt :The proposed architecture is based on OGSA compliant layered architecture which is shown in Fig. 1. In our proposed approach we have used CARE Resource Broker (CRB) for job submission [26]. The CRB is implemented by the Centre for Advanced Computing Research and Education (CARE), Anna University, India.
Efficient resource management for running multiple concurrent jobs in a computational grid environment
2011, Future Generation Computer SystemsCitation Excerpt :The proposed model in [9] considers a brokering mechanism based on benchmarking Grid resources. In contrast with many existing resource brokers [8–11], we present the design and implementation of a resource broker that adopts a hybrid approach to resource brokering. In our approach, resource brokering for the purpose of initial scheduling of a batch of jobs is centralized, whereas resource brokering for rescheduling purpose is performed in distributed manner.
SLA enabled CARE resource broker
2011, Future Generation Computer SystemsCitation Excerpt :GREEN, the distributed matchmaking mechanism that orders the grid resources by executing some benchmarks which are independent of underlying middleware is explained in [37]. In our previous work [5], the proposed framework is capable of creating virtual resources over the existing grid resources across the network. It applies resource scheduling strategies to decide on which resource and the virtual machines to be created.
A load-balanced hybrid heuristic for allocation of batch of tasks in cloud computing environment
2023, International Journal of Pervasive Computing and Communications
Thamarai Selvi Somasundaram is a Professor at Department of Information Technology, MIT Campus of Anna University. She is also the director of CARE and has successfully coordinated several research projects funded by various funding agencies across India. She has more than 27 years of teaching experience and 10 years of research experience. She has won many awards for her excellence in academic contributions. Her research areas include Grid Computing, Virtualization technologies, Neural Network, Cloud Computing, Mobile Computing. She has more than 50 publications in international conferences, 10 publications in renowned journals, and has authored 4 books.
Balachandar R. Amarnath has been with CARE as Senior Research Associate since April 2005 and is doing his Ph.D. in Grid Resource Management at Anna University. He has obtained his Masters in Engineering 2002. He possesses 6 years of research as well as teaching experience. His research interests include Semantic Grid, Virtualization, Desktop Grid, and Cloud Computing. He has more than 10 research publications in renowned international conferences.
R. Kumar is a M.E (CSE) graduate and has been working as Senior Research Associate at CARE since August 2005. He has 6 years of research as well as teaching experience. He has contributed to organize workshop and seminars related to Grid Computing and Web Services in CARE premises. He has more than 10 publications in renowned international conferences to his credit.
P. Balakrishnan has been working as a Senior Research Associate at CARE since October 2005. He is currently pursuing Ph.D. in Policies in grid and virtualization framework at Anna University. He pursued his Masters in Engineering during the year 2004. He has 7 years of research and software development experience. His major research areas include Security, SLA, Virtualization, Desktop Grid and Cloud Computing. He has more than 5 research publications in international conferences.
K. Rajendar has been working as Junior Research Associate at CARE since June 2005. He pursued Bachelor of Engineering in Computer science during the year 2002. He has four years of experience in grid computing. His major research areas include Trust, Security, Virtualization and Cloud Computing. He has more than 5 research publications in international conferences.
R. Rajiv is a B.E (Computer Science & Engg ) graduate and has been working as Project Associate at Centre for Advanced Computing Research and Education since April 2006. He has 4 years of research as well as software development experience. He has contributed in software development and testing of the research components in the area of Grid Computing and Virtualization at CARE. His research areas include Trust, Virtualization, Cloud Computing.
G. Kannan is a B.Tech. Graduate in Information Technology and has been working as Junior Research Associate at CARE since October 2005. He has more than 3 years of research experience in Grid Computing. His research has mainly focussed on Scheduling in Grid Computing and Virtualization. He has contributed to organize workshop and seminars related to Grid Computing and Web Services. He has very good domain knowledge in Grid Computing, Linux Operating System and Web Services. He has more than 5 publications in renowned publications in international conferences to his credit.
G. Rajesh Britto is a B.Tech graduate and has been working as Project Assistant at CATE since June 2007. He has over 3 years of experience out of which around 2 years is in research. He has around 5 publications to his credit. He has contributed in the code development of SLA in grid computing and also has knowledge in Vmware Server and its API’s.
E. Mahendran has been working as a Project Assistant at CARE since May 2005. He pursued Bachelor of Engineering in Computer science during the year 2004. He has 3 years of experience in grid computing. His major research areas include Neural Networks, Virtualization and Cloud Computing. He has more than 5 research publications in international conferences and one journal publication.
B. Madusudhanan is a B.Tech graduate and has been working as Project Assistant at CARE since June 2007. He has 21 months of research experience in Grid Computing and Virtualization Technologies. He has 3 publications in renowned international conferences.