Abstract
This paper studies the problem of improving the effectiveness of computing dependent tasks over the Internet. The distributed system is composed of a reliable server that coordinates the computation of a massive number of unreliable workers. It is known that the server cannot always ensure that the result of a task is correct without computing the task itself. This fact has significant impact on computing interdependent tasks. Since the computational capacity of the server may be restricted and so may be the time to complete the computation, the server may be able to compute only selected tasks, without knowing whether the remaining tasks were computed by workers correctly. But an incorrectly computed task may render the results of all dependent tasks incorrect. Thus it may become important for the server to compute judiciously selected tasks, so as to maximize the number of correct results.
In this work we assume that any worker computes correctly with probability p<1. Any incorrectly computed task corrupts all dependent tasks. The goal is to determine which tasks should be computed by the (reliable) server and which by the (unreliable) workers, and when, so as to maximize the expected number of correct results, under a constraint d on the computation time. We show that this optimization problem is NP-hard. Then we study optimal scheduling algorithms for the mesh with the tightest deadline. We present combinatorial arguments that completely describe optimal solutions for two ranges of values of worker reliability p, when p is close to zero and when p is close to one.
Contact author: Grzegorz Malewicz, Department of Computer Science, University of Alabama, 116 Houser Hall, Tuscaloosa, AL 35487-0290, USA, Fax (205) 348-0219.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barak, B., Goldreich, O., Impagliazzo, R., Rudich, S., Sahai, A., Vadhan, S., Yang, K.: On the (Im)possibility of Obfuscating Programs. In: Kilian, J. (ed.) CRYPTO 2001. LNCS, vol. 2139, pp. 1–18. Springer, Heidelberg (2001)
Crescenzi, P., Kann, V. (eds.): A compendium of NP optimization problems, http://www.nada.kth.se/~viggo/wwwcompendium/node173.html
Du, W., Jia, J., Mangal, M., Murugesan, M.: Uncheatable Grid Computing. In: 24th International Conference on Distributed Computing Systems, ICDCS (2004)
Foster, I., Kesselman, C.: The Grid: Blueprint for a New Computing Infrastructure, 2nd edn. Morgan Kaufmann, San Francisco (2004)
Garey, M.R., Johnson, D.S.: Computers and Intractability. Freeman, New York (1979)
Golle, P., Mironov, I.: Uncheatable Distributed Computations. In: Naccache, D. (ed.) CT-RSA 2001. LNCS, vol. 2020, pp. 425–440. Springer, Heidelberg (2001)
Golle, P., Stubblebine, S.: Secure Distributed Computing in a Commercial Environment. In: Syverson, P.F. (ed.) FC 2001. LNCS, vol. 2339, pp. 289–304. Springer, Heidelberg (2002)
The Intel Philanthropic Peer-to-Peer program, http://www.intel.com/cure
Kahney, L.: Cheaters Bow to Peer Pressure. In: Wired News, February 15 (2001), http://www.wired.com/news/technology/0,1282,41838,00.html
Kondo, D., Casanova, H., Wing, E., Berman, F.: Models and Scheduling Mechanisms for Global Computing Applications. In: 16th IEEE International Parallel & Distributed Processing Symposium (2002)
Korpela, E., Werthimer, D., Anderson, D., Cobb, J., Lebofsky, M.: SETI@home - massively distributed computing for seti. Computing in Science & Enginering 3(1), 78–83 (2001)
Malewicz, G.: Parallel Scheduling of Complex Dags under Uncertainty (2005) (submitted for publication)
Malewicz, G., Rosenberg, A.L., Yurkewych, M.: On Scheduling Complex Dags for Internet-Based Computing. In: 19th IEEE International Parallel & Distributed Processing Symposium, IPDPS (2005) (to appear)
Malewicz, G., Rosenberg, A.L.: On batch-scheduling dags for Internet-based computing. Typescript, University of Massachusetts (2004) (submitted for publication)
Malewicz, G., Russell, A., Shvartsman, A.: Distributed Cooperation During the Absence of Communication. In: Herlihy, M.P. (ed.) DISC 2000. LNCS, vol. 1914, pp. 119–133. Springer, Heidelberg (2000)
The Olson Laboratory Fight AIDS@Home project, http://www.fightaidsathome.org
Peeters, R.: The maximum edge biclique problem is NP-complete. Discrete Applied Mathematics 131(3), 651–654 (2003)
Rosenberg, A.L., Yurkewych, M.: Optimal Schedules for Some Common Computation-Dags on the Internet. IEEE Transactions on Computers (2005) (to appear)
Rosenberg, A.L.: On Scheduling Mesh-Structured Computations on the Internet. IEEE Transactions on Computers 53(9) (2004)
Rosenberg, A.L.: Accountable Web-computing. IEEE Transactions on Parallel and Distributed Systems 14(2), 97–106 (2003)
The RSA Factoring By Web project, http://www.npac.syr.edu/factoring
Sarmenta, L.F.G.: Sabotage-tolerance mechanisms for volunteer computing systems. Future Generation Computer Systems 18(4), 561–572 (2002)
SETI@home: Current Total Statistics May 9 (2004), http://setiathome.ssl.berkeley.edu/totals.html
Sun, X.H., Wu, M.: GHS: A performance Prediction and Task Scheduling System for Grid Computing. In: 17th IEEE International Parallel & Distributed Processing Symposium (2003)
Szajda, D., Lawson, B., Owen, J.: Hardening Functions for Large Scale Distributed Computations. In: IEEE Symposium on Security and Privacy, pp. 216–224 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gao, L., Malewicz, G. (2005). Internet Computing of Tasks with Dependencies Using Unreliable Workers. In: Higashino, T. (eds) Principles of Distributed Systems. OPODIS 2004. Lecture Notes in Computer Science, vol 3544. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11516798_32
Download citation
DOI: https://doi.org/10.1007/11516798_32
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-27324-0
Online ISBN: 978-3-540-31584-1
eBook Packages: Computer ScienceComputer Science (R0)