Skip to main content
SpringerLink
Log in

Evaluating New Approaches of Big Data Analytics Frameworks

  • Conference paper
  • First Online:
Business Information Systems (BIS 2015)

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 208))

Included in the following conference series:

Abstract

The big data topic will be one of the leading growth markets in information technology in the next years. One problem in this area is the efficient computation of huge data volumes, especially for complex algorithms in data mining and machine learning tasks. This paper discuss new processing frameworks for big and smart data in distributed environments and presents a benchmark between two frameworks - Apache Flink and Apache Spark - based on a mixed workload with algorithms from different analytic areas with different real-world datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Gartner. Hype Cycle for emerging technologies (2014). http://www.gartner.com/newsroom/id/2819918

  2. Apache Software Foundation. Apache Hadoop NextGen MapReduce (Yarn) (2015). http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html

  3. Lee, K.-H., Lee, Y.-J., Choi, H., Chung, Y.D., Moon, B.: Parallel data processing with MapReduce: a survey. SIGMOD Rec. 40(4), 11–20 (2012)

    Article  Google Scholar 

  4. Apache Software Foundation. Apache Flink (2015). http://flink.apache.org/

  5. Apache Software Foundation. Apache Giraph (2015). http://giraph.apache.org/

  6. Apache Software Foundation. Apache Hama (2015). http://hama.apache.org/

  7. Apache Software Foundation. Apache Spark (2015). http://spark.apache.org/

  8. University of California Irvine. AsterixDB (2015). https://asterixdb.ics.uci.edu

  9. Dato. GraphLab (2015). http://www.graphlab.com/

  10. Elser, B., Montresor, A.: An evaluation study of bigdata frameworks for graph processing. In: Hu, X. et al. (eds.) BigData Conference, pp. 60–67. IEEE (2013)

    Google Scholar 

  11. AmpLab. BigDataBenchmark (2015). https://amplab.cs.berkeley.edu/benchmark/

  12. Alexandrov, A., et al.: The stratosphere platform for big data analytics. VLDB J. 23(6), 939–964 (2014)

    Article  Google Scholar 

  13. Zaharia, M. et al.: Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation. NSDI 2012, pp. 2–2. USENIX Association, San Jose (2012)

    Google Scholar 

  14. Ewen, S., Tzoumas, K., Kaufmann, M., Markl, V.: Spinning fast iterative data flows. Proc. VLDB Endow. 5(11), 1268–1279 (2012)

    Article  Google Scholar 

  15. Warneke, D., Kao, O.: Nephele efficient parallel data processing in the cloud. In: Raicu, I., Foster, I., Zhao, Y. (eds.) MTAGS 2009, p. 110. ACM, New York (2009)

    Google Scholar 

  16. Hueske, F., Peters, M., Sax, M.J., Rheinlnder, A., Bergmann, R., Krettek, A., Tzoumas, K.: Opening the black boxes in data flow optimization. Proc. VLDB Endow. 5(12), 1256–1267 (2012)

    Article  Google Scholar 

  17. Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing. USENIX Association (2010)

    Google Scholar 

  18. Blumenstock, J.E.: Size matters: word count as a measure of quality on wikipedia. In: Proceedings of the 17th international conference on World Wide Web. ACM (2008)

    Google Scholar 

  19. Hartigan, J., Manchek, A.: Algorithm AS 136: a k-means clustering algorithm. In: Applied statistics, pp. 100–108 (1979)

    Google Scholar 

  20. Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank citation ranking: bringing order to the web. http://ilpubs.stanford.edu:8090/422/1/1999-66.pdf

  21. Transaction Processing Performance Council: TPC Express Benchmark H Decision Support-Standard Specification (2014). http://www.tpc.org/tpch/default.asp

  22. Liang, F., Feng, C., Lu, X., Xu, Z.: Performance benefits of DataMPI: a case study with bigdatabench. In: Zhan, J., Rui, H., Weng, C. (eds.) BPOE 2014. LNCS, vol. 8807, pp. 111–123. Springer, Heidelberg (2014)

    Google Scholar 

Download references

Acknowledgements

The work presented in this paper was funded by the German Federal Ministry of Education and Research under the project Competence Center for Scalable Data Services and Solutions Dresden/Leipzig (BMBF 01IS14014B) and by the German Federal Ministry of Economic Affairs and Energy under the project InnOPlan (BMWI 01MD15002E).

Author information

Authors and Affiliations

  1. Information Systems Institute, Leipzig University, Grimmaische Straße 12, 04109, Leipzig, Germany

    Norman Spangenberg, Martin Roth & Bogdan Franczyk

Authors
  1. Norman Spangenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Roth
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bogdan Franczyk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Norman Spangenberg .

Editor information

Editors and Affiliations

  1. Department of Information Systems, Poznań University of Economics, Poznań, Poland

    Witold Abramowicz

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Spangenberg, N., Roth, M., Franczyk, B. (2015). Evaluating New Approaches of Big Data Analytics Frameworks. In: Abramowicz, W. (eds) Business Information Systems. BIS 2015. Lecture Notes in Business Information Processing, vol 208. Springer, Cham. https://doi.org/10.1007/978-3-319-19027-3_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-19027-3_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-19026-6

  • Online ISBN: 978-3-319-19027-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation