A Framework For Resource Availability Characterization And Online Prediction In The Grids

Production grids integrate today thousands of resources into e-Science platforms. However, the current practice of running yearly tens of millions of single-resource, long-running grid jobs with few fault tolerance capabilities is hampered by the highly dynamic grid resource availability. In additional to resource failures, grids introduce a new vector of resource availability dynamics: the resource sharing policy established by the resource owners. As a result, the availability-aware grid resourcemanagement is a challenging problemfor today’s researchers. To address this problem, we present in this work GriS-Prophet, an integrated system for resource availability monitoring, analysis, and prediction. Using GriS-Prophet’s analysis tools on a long-term availability trace from the Austrian Grid, we characterize the grid resource availability for three resource availability policies. Notably, we show that the three policies lead to very different capabilities for running the typical grid workloads efficiently. We introduce a new resource availability predictor based on Bayesian inference. Last but not least, using GriS-Prophet’s prediction tools we achieve an accuracy of more than 90%; and 75%; in our instance and duration availability predictions respectively.