How to Build an Open Source Render Farm Based on Desktop Grid Computing

Abstract

This paper presents an experiment on how to implement a Grid-based High Performance Computing solution using existing resources typically available in a teaching or research laboratory. A cost-effective solution is proposed based on open source software components, and, where appropriate, our own software solutions, for large scientific applications in the public sector such as universities and research institutes. In such institutions, classical solutions for HPC are often not affordable, yet they usually have at their disposal a large number of machines that can be utilised. The Department of Informatics at University of Sussex, for example, has just installed 150 new Core2 Duo machines across 3 laboratories. By scaling this number up across the whole University, it can result a large potential computing resource for utilization. Typical processor usage rates are often somewhere between 10% and 20% (i.e. user-generated processes) for most machines. This paper proposes a solution that exploits the remaining 80% to 90% processor power through consumption of available computer idle time without disturbing current users. To achieve this goal, the open source Condor High Throughput Computing software was selected and implemented as a desktop Grid computing solution. This paper presents our experiences in finding a solution so that other institutions can develop similar Grid solutions for their own large scientific experiments, taking advantage of their existing resources. The implementation of our solution is analyzed in the context of building a render farm.