skip to main content
10.1145/3491086.3492470acmconferencesArticle/Chapter ViewAbstractPublication PagesmiddlewareConference Proceedingsconference-collections
research-article

Benchmarking Apache Kafka under network faults

Published: 06 December 2021 Publication History

Abstract

Network faults are often transient and hence hard to detect and difficult to resolve. Our study conducts an analysis of Kafka's network fault tolerance capabilities, one of the widely used distributed stream processing system (DSPS). Across different Kafka configurations, we observed that Kafka is fault-tolerant towards network faults to some degree, and we report observations of its shortcomings. We also define a network fault-tolerance benchmark on which other DSPSs can be evaluated.

References

[1]
J. Karimov, T. Rabl, A. Katsifodimos, R. Samarev, H. Heiskanen, and V. Markl, "Benchmarking distributed stream data processing systems," 2018 IEEE 34th International Conference on Data Engineering (ICDE), Apr 2018. [Online].
[2]
M. A. Lopez, A. G. P. Lobato, and O. C. M. B. Duarte, "A performance comparison of open-source stream processing platforms," 2016 IEEE Global Communications Conference (GLOBECOM), pp. 1--6, 2016.
[3]
G. van Dongen and D. V. D. Poel, "A performance analysis of fault recovery in stream processing frameworks," IEEE Access, vol. 9, pp. 93 745--93 763, 2021.

Cited By

View all
  • (2023)A Survey on Networked Data Streaming With Apache KafkaIEEE Access10.1109/ACCESS.2023.330381011(85333-85350)Online publication date: 2023
  • (2022)On Efficiently Partitioning a Topic in Apache Kafka2022 International Conference on Computer, Information and Telecommunication Systems (CITS)10.1109/CITS55221.2022.9832981(1-8)Online publication date: 13-Jul-2022

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
Middleware '21: Proceedings of the 22nd International Middleware Conference: Demos and Posters
December 2021
23 pages
ISBN:9781450391542
DOI:10.1145/3491086
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

In-Cooperation

  • USENIX Assoc: USENIX Assoc
  • IFIP

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 December 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Kafka
  2. Kafka streams
  3. chaos testing
  4. monitoring

Qualifiers

  • Research-article

Conference

Middleware '21
Sponsor:
Middleware '21: 22nd International Middleware Conference
December 6 - 10, 2021
Virtual Event, Canada

Acceptance Rates

Overall Acceptance Rate 203 of 948 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)55
  • Downloads (Last 6 weeks)3
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)A Survey on Networked Data Streaming With Apache KafkaIEEE Access10.1109/ACCESS.2023.330381011(85333-85350)Online publication date: 2023
  • (2022)On Efficiently Partitioning a Topic in Apache Kafka2022 International Conference on Computer, Information and Telecommunication Systems (CITS)10.1109/CITS55221.2022.9832981(1-8)Online publication date: 13-Jul-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media