skip to main content
10.1145/1095810.1095816acmconferencesArticle/Chapter ViewAbstractPublication PagessospConference Proceedingsconference-collections
Article

BAR fault tolerance for cooperative services

Published:20 October 2005Publication History

ABSTRACT

This paper describes a general approach to constructing cooperative services that span multiple administrative domains. In such environments, protocols must tolerate both Byzantine behaviors when broken, misconfigured, or malicious nodes arbitrarily deviate from their specification and rational behaviors when selfish nodes deviate from their specification to increase their local benefit. The paper makes three contributions: (1) It introduces the BAR (Byzantine, Altruistic, Rational) model as a foundation for reasoning about cooperative services; (2) It proposes a general three-level architecture to reduce the complexity of building services under the BAR model; and (3) It describes an implementation of BAR-B the first cooperative backup service to tolerate both Byzantine users and an unbounded number of rational users. At the core of BAR-B is an asynchronous replicated state machine that provides the customary safety and liveness guarantees despite nodes exhibiting both Byzantine and rational behaviors. Our prototype provides acceptable performance for our application: our BAR-tolerant state machine executes 15 requests per second, and our BAR-B backup service can back up 100MB of data in under 4 minutes.

References

  1. E. Adar and B. Huberman. Free riding on gnutella. Technical report, Xerox PARC, Aug. 2000.Google ScholarGoogle ScholarCross RefCross Ref
  2. A. Adya, W. Bolosky, M. Castro, R. Chaiken, G. Cermak, J. Douceur, J. Howell, J. Lorch, M. Theimer, and R. Wattenhofer. Farsite: Federated, available, and reliable storage for an incompletely trusted environment. In 5th OSDI, Dec 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. A. Akella, S. Seshan, R. Karp, S. Shenker, and C. Papadimitriou. Selfish behavior and stability of the internet: a game-theoretic analysis of tcp. In Proc. SIGCOMM, pages 117--130. ACM Press, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. R. J. Aumann. Subjectivity and correlation in randomized strategies. Journal of Mathematical Economics, 1(1):67--96, 1974.Google ScholarGoogle ScholarCross RefCross Ref
  5. C. Batten, K. Barr, A. Saraf, and S. Trepetin. pStore: A secure peer-to-peer backup system. Technical Memo MIT-LCS-TM-632, Massachusetts Institute of Technology Laboratory for Computer Science, October 2002.Google ScholarGoogle Scholar
  6. G. Bracha and S. Toueg. Asynchronous consensus and broadcast protocols. J. ACM, 32(4):824--840, 1985. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. T. C. Bressoud and F. B. Schneider. Hypervisor-based fault tolerance. ACM Trans. Comput. Syst., 14(1):80--107, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. M. Burrows, M. Abadi, and R. Needham. A Logic of Authentication. In ACM Trans. Comput. Syst., pages 18--36, Feb. 1990. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. R. Canetti and T. Rabin. Optimal Asynchronous Byzantine Agreement. Technical Report 92-15, TR 92-15, Dept. of Computer Science, Hebrew University, 1992.Google ScholarGoogle Scholar
  10. M. Castro and B. Liskov. Practical Byzantine fault tolerance and proactive recovery. ACM Trans. Comput. Syst., 20(4):398--461, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. J. Chase, B. Chun, Y. Fu, S. Schwab, and A. Vahdat. Sharp: An architecture for secure resource peering. In SOSP, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. The game of chicken. http://www.gametheory.net/Dictionary/Games/GameofChicken.html.Google ScholarGoogle Scholar
  13. B. Cohen. The bittorrent home page. http://bittorrent.com.Google ScholarGoogle Scholar
  14. B. Cohen. Incentives build robustness in bittorrent. In Proc. 2nd IPTPS, 2003.Google ScholarGoogle Scholar
  15. L. Cox and B. Noble. Pastiche: Making backup cheap and easy. In Proc. 5th OSDI, Dec 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. L. P. Cox and B. D. Noble. Samsara: honor among thieves in peer-to-peer storage. In Proc. 19th SOSP, pages 120--132, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. A. K. Dixit and S. Skeath. Games of Strategy. W. W. Norton & Company, 1999.Google ScholarGoogle Scholar
  18. J. R. Douceur. The Sybil attack. In Proc. 1st IPTPS, pages 251--260. Springer-Verlag, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. K. Eliaz. Fault tolerant implementation. Review of Economic Studies, 69:589--610, Aug 2002.Google ScholarGoogle ScholarCross RefCross Ref
  20. J. Feigenbaum, C. H. Papadimitriou, and S. Shenker. Sharing the cost of multicast transmissions. J. Comput. Syst. Sci., 63(1):21--41, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. J. Feigenbaum, R. Sami, and S. Shenker. Mechanism design for policy routing. In Proc. 23rd PODC, pages 11--20. ACM Press, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. J. Feigenbaum and S. Shenker. Distributed algorithmic mechanism design: Recent results and future directions. In Proc. 6th DIALM, pages 1--13. ACM Press, New York, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. M. Feldman, C. Papadimitriou, J. Chuang, and I. Stoica. Free-riding and whitewashing in peer-to-peer systems. In Proc. PINS, pages 228--236. ACM Press, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. M. Fischer, N. Lynch, and M. Paterson. Impossibility of distributed consensus with one faulty process. J. ACM, 32(2):374--382, 1985. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. D. Fudenberg and J. Tirole. Game theory. MIT Press, Aug. 1991.Google ScholarGoogle Scholar
  26. J. Garay and Y. Moses. Fully Polynomial Byzantine Agreement for n>3t Processors in t+1 Rounds. SIAM J. of Computing, 27(1), 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. K. P. Gummadi, R. J. Dunn, S. Saroio, S. D. Gribbl, H. M. Levy, and J. Zahorjan. Measurement, modeling, and analysis of a peer-to-peer file-sharing workload. In Proc. 19th SOSP, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. J. Harsanyi. A general theory of rational behavior in game situations. Econometrica, 34(3):613--634, Jul. 1966.Google ScholarGoogle ScholarCross RefCross Ref
  29. L. Lamport. The part-time parliament. ACM Trans. Comput. Syst., 16(2):133--169, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. L. Lamport, R. Shostak, and M. Pease. The Byzantine generals problem. ACM Trans. Program. Lang. Syst., 4(3):382--401, 1982. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. M. Lillibridge, S. Elnikety, A. Birrell, M. Burrows, and M. Isard. A cooperative internet backup scheme. In USENIX ATC, june 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. M. Loney. Charity gives 40,000 pcs a fresh start. CNET News.com, February 4 2005. http://news.com.com/Charity+gives+403421.html.Google ScholarGoogle Scholar
  33. R. Mahajan, M. Rodrig, D. Wetherall, and J. Zahorjan. Sustaining cooperation in multi-hop wireless networks. In NSDI, May 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. G. J. Mailath. Do people play Nash equilibrium? lessons from evolutionary game theory. Journal of Economic Literature, 36 (September 1998), 1347--1374, 1998.Google ScholarGoogle Scholar
  35. D. Malhotra. Making threats credible. Negotiation, 8(3), Mar. 2005.Google ScholarGoogle Scholar
  36. D. Malkhi and M. Reiter. Byzantine quorum systems. Distributed Computing 11/4, pages 203--213, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. D. Malkhi and M. Reiter. Secure and scalable replication in Phalanx. In Proc. 17th SRDS, Oct 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. P. Maniatis, D. S. H. Rosenthal, M. Roussopoulos, M. Baker, T. Giuli, and Y. Muliadi. Preserving peer replicas by rate-limited sampled voting. In Proc. 19th SOSP, pages 44--59. ACM Press, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. J.-P. Martin, A. S. Aiyer, L. Alvisi, A. Clement, M. Dahlin, and C. Porth. BAR tolerance for cooperative services. Technical Report TR-05-10, Department of Computer Sciences, The University of Texas at Austin, Mar. 2005.Google ScholarGoogle Scholar
  40. N. H. Minsky and V. Ungureanu. Law-governed interaction: a coordination and control mechanism for heterogeneous distributed systems. ACM Trans. Softw. Eng. Methodol., 9(3):273--305, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. J. Nash. Non-cooperative games. The Annals of Mathematics, 54:286--295, Sept 1951.Google ScholarGoogle ScholarCross RefCross Ref
  42. T. W. Ngan, D. Wallach, and P. Druschel. Enforcing fair sharing of peer-to-peer resources. In Proc. 2nd IPTPS, 2003.Google ScholarGoogle ScholarCross RefCross Ref
  43. T.-W. Ngan, D. S. Wallach, and P. Druschel. Incentives-compatible peer-to-peer multicast. In 2nd Workshop on Economics of Peer-to-Peer Systems, 2004.Google ScholarGoogle Scholar
  44. S. J. Nielson, S. A. Crosby, and D. S. Wallach. A taxonomy of rational attacks. In Proc. 4th IPTPS, Feb. 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. N. Nisanb and A. Ronenc. Algorithmic mechanism design. Games and Economic Behavior, 35:166--196, April 2001.Google ScholarGoogle ScholarCross RefCross Ref
  46. N. Ntarmos and P. Triantafillou. Aesop: Altruism-endowed self organizing peers. In Proc. 2nd DBISP2P, August 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. N. I. of~Standards and Technology. Secure hash standard. Technical report, U.S. Department of Commerce, August 2002.Google ScholarGoogle Scholar
  48. C. Papadimitriou. Algorithms, games, and the internet. In Proc. 33rd STOC, pages 749--753. ACM Press, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. M. Reiter. The Rampart toolkit for building high-integrity services. In Dagstuhl Seminar on Dist. Sys., pages 99--110, 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. S. Rhea, P. Eaton, D. Geels, H. Weatherspoon, B. Zhao, and J. Kubiatowicz. Pond: The oceanstore prototype. In FAST, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. R. L. Rivest, A. Shamir, and L. M. Adleman. A method for obtaining digital signatures and public-key cryptosystems (reprint). Commun. ACM, 26(1):96--99, 1983. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. L. Rizzo. Effective erasure codes for reliable computer communication protocols. SIGCOMM Comput. Commun. Rev., 27(2):24--36, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. R. Rodrigues, M. Castro, and B. Liskov. BASE: using abstraction to improve fault tolerance. In Proc. 18th SOSP, pages 15--28. ACM Press, Oct. 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. A. Rowstron and P. Druschel. Storage management and caching in past, a large-scale, persistent peer-to-peer storage utility. In Proc. 18th SOSP, pages 188--201. ACM Press, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. F. B. Schneider. Implementing fault-tolerant services using the state machine approach: a tutorial. ACM Comput. Surv., 22(4):299--319, 1990. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. F. B. Schneider. Distributed Computing (Editor: Sape Mullender), chapter 2, "What Good are Models and What Models are Good?", pages 17--26. ACM Press, second edition, 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. "seti@home". http://setiathome.ssl.berkeley.edu/.Google ScholarGoogle Scholar
  58. J. Shneidman and D. Parkes. Rationality and self-interest in peer to peer networks. In Proc. 2nd IPTPS, 2003.Google ScholarGoogle ScholarCross RefCross Ref
  59. J. Shneidman and D. C. Parkes. Specification faithfulness in networks with rational nodes. In Proc. 23rd PODC, pages 88--97. ACM Press, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. J. Shneidman, D. C. Parkes, and L. Massoulie. Faithfulness in internet algorithms. In Proc. PINS, Portland, USA, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. V. Srinivasan, P. Nuggehalli, C.-F. Chiasserini, and R. R. Rao. Cooperation in wireless ad hoc networks. In INFOCOM, 2003.Google ScholarGoogle ScholarCross RefCross Ref
  62. B. White, J. Lepreau, L. Stoller, R. Ricci, S. Guruprasad, M. Newbold, M. Hibler, C. Barb, and A. Joglekar. An integrated experimental environment for distributed systems and networks. In Proc. 5th OSDI, pages 255--270, Boston, MA, Dec. 2002. USENIX Association. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. J. Yin, J.-P. Martin, A. Venkataramani, L. Alvisi, and M. Dahlin. Separating agreement from execution for Byzantine fault tolerant services. In Proc. 19th SOSP, pages 253--267. ACM Press, Oct. 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. BAR fault tolerance for cooperative services

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        SOSP '05: Proceedings of the twentieth ACM symposium on Operating systems principles
        October 2005
        259 pages
        ISBN:1595930795
        DOI:10.1145/1095810
        • cover image ACM SIGOPS Operating Systems Review
          ACM SIGOPS Operating Systems Review  Volume 39, Issue 5
          SOSP '05
          December 2005
          290 pages
          ISSN:0163-5980
          DOI:10.1145/1095809
          Issue’s Table of Contents

        Copyright © 2005 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 20 October 2005

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • Article

        Acceptance Rates

        Overall Acceptance Rate131of716submissions,18%

        Upcoming Conference

        SOSP '24

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader