skip to main content
10.1145/2554850.2554908acmconferencesArticle/Chapter ViewAbstractPublication PagessacConference Proceedingsconference-collections
research-article

Large-scale message synchronization in challenged networks

Published: 24 March 2014 Publication History

Abstract

In this paper we introduce Dispersy, a message synchronization platform capable of running inside a challenged network. Dispersy uses Bloom filters to let peers advertise their local state and receive missing messages. However, in contrast to previous work, the efficiency and effectiveness of our design allows for deployment in networks of unprecedented scale. We show, through extensive experimental evidence that peers have no difficulties in synchronizing over 100,000 messages.
We integrate in Dispersy a NAT traversal technique able to puncture 77% of NAT-firewalls. Not puncturing these firewalls would prevent up to 64% of peers from receiving any synchronization requests. Implementing a NAT traversal technique proved essential when we used Dispersy to extend the functionalities of a BitTorrent client. To date, over 350,000 users used our modified BitTorrent client which included Dispersy, synchronizing in overlays consisting of more than 500,000 messages.
We emulate an overlay consisting of 1000 unmodified Dispersy peers in order to show the propagation and synchronization speed, bandwidth requirements, churn resilience, and overall throughput of Dispersy. Dispersy is able to synchronize a single message to all peers within 19 synchronization steps, withstand an average session-time of 30 seconds, and achieve an average throughput of 10,000 messages/s.

References

[1]
P. S. Almeida, C. Baquero, N. Preguiça, and D. Hutchison. Scalable bloom filters. Inf. Process. Lett., 101(6): 255--261, Mar. 2007.
[2]
C. Audet, F., Jennings. Network Address Translation (NAT) Behavioral Requirements for Unicast UDP. RFC 4787 (Best Current Practice), 2007.
[3]
B. H. Bloom. Space/time trade-offs in hash coding with allowable errors. Communications of the ACM, 13(7): 422--426, July 1970.
[4]
A. Broder and M. Mitzenmacher. Network applications of bloom filters: A survey. In Internet Mathematics, pages 636--646, 2002.
[5]
P. Congdon, B. Aboba, A. Smith, G. Zorn, and J. Roese. Rfc 3580, ieee 802.1x remote authentication dial in user service, 2003.
[6]
B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking cloud serving systems with ycsb. In Proceedings of the 1st ACM symposium on Cloud computing, SoCC '10, pages 143--154, New York, NY, USA, 2010. ACM.
[7]
A. Demers, D. Greene, C. Hauser, W. Irish, J. Larson, S. Shenker, H. Sturgis, D. Swinehart, and D. Terry. Epidemic algorithms for replicated database maintenance. In Proceedings of the sixth annual ACM Symposium on Principles of distributed computing, PODC '87, pages 1--12, New York, NY, USA, 1987. ACM.
[8]
G. Halkes and J. Pouwelse. Udp nat and firewall puncturing in the wild. In Proceedings of the 10th international IFIP TC 6 conference on Networking - Volume Part II, NETWORKING'11, pages 1--12, Berlin, Heidelberg, 2011. Springer-Verlag.
[9]
H.-Y. Huang, P.-E. Luo, M. Li, D. Li, X. Li, W. Shu, and M.-Y. Wu. Performance evaluation of suvnet with real-time traffic data. Vehicular Technology, IEEE Transactions on, 56(6): 3381--3396, nov. 2007.
[10]
M. Jelasity, S. Voulgaris, R. Guerraoui, A.-M. Kermarrec, and M. van Steen. Gossip-based peer sampling. ACM Trans. Comput. Syst., 25(3), Aug. 2007.
[11]
L. Lamport. Time, clocks, and the ordering of events in a distributed system. Commun. ACM, 21(7): 558--565, July 1978.
[12]
U. Lee, E. Magistretti, M. Gerla, P. Bellavista, and A. Corradi. Dissemination and harvesting of urban data using vehicular sensing platforms. Technical report, 2007.
[13]
A. Muller, N. Evans, C. Grothoff, and S. Kamkar. Autonomous nat traversal. In Peer-to-Peer Computing (P2P), 2010 IEEE Tenth International Conference on, pages 1--4, aug. 2010.
[14]
S. Rhea, D. Geels, T. Roscoe, and J. Kubiatowicz. Handling churn in a dht. In In Proceedings of the USENIX Annual Technical Conference, 2004.
[15]
C. Shannon, D. Moore, and k. claffy. Characteristics of fragmented ip traffic on internet links. In Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement, IMW '01, pages 83--97, New York, NY, USA, 2001. ACM.
[16]
N. Zeilemaker, M. Capotă, A. Bakker, and J. Pouwelse. Tribler: P2p media search and sharing. In Proceedings of the 19th ACM international conference on Multimedia, MM '11, pages 739--742, New York, NY, USA, 2011. ACM.

Cited By

View all
  • (2015)Decentralized credit mining in P2P systems2015 IFIP Networking Conference (IFIP Networking)10.1109/IFIPNetworking.2015.7145334(1-9)Online publication date: May-2015
  • (2014)100 Million DHT replies14-th IEEE International Conference on Peer-to-Peer Computing10.1109/P2P.2014.6934318(1-4)Online publication date: Sep-2014
  • (2014)4P: Performant private peer-to-peer file sharing14-th IEEE International Conference on Peer-to-Peer Computing10.1109/P2P.2014.6934311(1-10)Online publication date: Sep-2014

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SAC '14: Proceedings of the 29th Annual ACM Symposium on Applied Computing
March 2014
1890 pages
ISBN:9781450324694
DOI:10.1145/2554850
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 March 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. NAT traversal
  2. decentralized
  3. deployment
  4. evaluation
  5. synchronization

Qualifiers

  • Research-article

Conference

SAC 2014
Sponsor:
SAC 2014: Symposium on Applied Computing
March 24 - 28, 2014
Gyeongju, Republic of Korea

Acceptance Rates

SAC '14 Paper Acceptance Rate 218 of 939 submissions, 23%;
Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25
The 40th ACM/SIGAPP Symposium on Applied Computing
March 31 - April 4, 2025
Catania , Italy

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2015)Decentralized credit mining in P2P systems2015 IFIP Networking Conference (IFIP Networking)10.1109/IFIPNetworking.2015.7145334(1-9)Online publication date: May-2015
  • (2014)100 Million DHT replies14-th IEEE International Conference on Peer-to-Peer Computing10.1109/P2P.2014.6934318(1-4)Online publication date: Sep-2014
  • (2014)4P: Performant private peer-to-peer file sharing14-th IEEE International Conference on Peer-to-Peer Computing10.1109/P2P.2014.6934311(1-10)Online publication date: Sep-2014

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media