Sharing the resources among various users and the lack of a centralized control are two key characteristics of many distributed heterogeneous computing systems. A critical challenge for designing applications in such systems is to coordinate the resources in a decentralized fashion while adapting to the changes in the system. In this paper, we consider the computation of a large set of equal-sized independent tasks. This represents the computation paradigm for a variety of large scale applications such as SETI@home and Monte Carlo simulations. We focus on the performance optimization for a decentralized adaptive task allocation protocol. We develop a bandwidth allocation strategy based on our decentralized task allocation algorithm, and a simple task buffer management policy. Simulation results show that our task allocation protocol achieves close to the optimal system throughput.