Synonyms
Definition
An end-to-end data pipeline benchmark is a standardized suite of data ingestion, data processing, and data queries, arranged in a series of stages, where the output of a previous stage in a pipeline feeds the next stage in pipeline, exercising all the needed system characteristics for commonly constructed data pipeline workloads.
Historical Background
As we witness the rapid transformation in data architecture, where relational database management systems (RDBMS) are being supplemented by large-scale non-relational stores such as Hadoop Distributed File System (HDFS), MongoDB, Apache Cassandra, and Apache HBase, a more fundamental shift is on its way, which would require larger changes to modern data architectures. While the current shift was mandated by business requirements for the connected world, the next wave will be dictated by operational cost optimization, transformative changes in the underlying infrastructure...
Keywords
- Streaming Analytics
- Data Pipeline
- Hadoop Distributed File System (HDFS)
- System Under Test (SUT)
- Standard Performance Evaluation Corporation (SPEC)
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
References
Baru C, Bhandarkar M, Nambiar R, Poess M, Rabl T (2013) Benchmarking big data systems and the big data top100 list. Big Data 1(1):60–64
Ghazal A, Rabl T, Hu M, Raab F, Poess M, Crolotte A, Jacobsen HA (2013) Bigbench: towards an industry standard benchmark for big data analytics. In Proceedings of the 2013 ACM SIGMOD international conference on management of data, SIGMOD’13, ACM, New York, pp 1197–1208
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this entry
Cite this entry
Bhandarkar, M.A. (2018). End-to-End Benchmark. In: Sakr, S., Zomaya, A. (eds) Encyclopedia of Big Data Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-63962-8_112-1
Download citation
DOI: https://doi.org/10.1007/978-3-319-63962-8_112-1
Received:
Accepted:
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-63962-8
Online ISBN: 978-3-319-63962-8
eBook Packages: Springer Reference MathematicsReference Module Computer Science and Engineering