Skip to main content
Springer Nature Link
Log in

Accelerating BigBench on Hadoop

  • Conference paper
  • First Online:
Big Data Benchmarking (WBDB 2015, WBDB 2015)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10044))

Included in the following conference series:

  • 879 Accesses

Abstract

Benchmarking Big Data systems is an open challenge. The existing Micro-Benchmarks (e.g. TeraSort) do not present an end-to-end scenario in real world. To solve this issue, a new towards industry standard benchmark for Big Data Analytics called BigBench has been proposed. And with BigBench, we’ve been keeping our collaboration with Apache Open Source Community to work on performance tuning and optimization for Hadoop ecosystem. In this paper, we share our contributions to BigBench, and present our tuning and optimization experience along with the benchmark results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. TPCx-BB. http://www.tpc.org/tpcx-bb/

  2. Ghazal, A., Rabl, T., Hu, M., Raab, F., Poess, M., Crolotte, A., Jacobsen, H.: BigBench: towards an industry standard benchmark for big data analytics. In: SIGMOD (2013)

    Google Scholar 

  3. Hive on Spark. https://issues.apache.org/jira/browse/HIVE-7292

  4. TPC-H. http://www.tpc.org/tpch/

  5. TPC-DS. http://www.tpc.org/tpcds/

  6. Tuning Spark. http://spark.apache.org/docs/latest/tuning.html

  7. Chiba, T., Onodera, T.: Workload characterization and optimization of TPC-H queries on Apache Spark. Computer Science (2015)

    Google Scholar 

  8. Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Google Inc (2004)

    Google Scholar 

  9. Rabl, T., Frank, M., Danisch, M., Gowda, B., Jacobsen, H.-A.: Towards a complete BigBench implementation. In: Rabl, T., Sachs, K., Poess, M., Baru, C., Jacobson, H.-A. (eds.) WBDB 2015. LNCS, vol. 8991, pp. 3–11. Springer, Heidelberg (2015). doi:10.1007/978-3-319-20233-4_1

    Chapter  Google Scholar 

  10. Baru, C., Bhandarkar, M., Curino, C., Danisch, M., Frank, M., Gowda, B., Jacobsen, H., Jie, H., Kumar, D., Nambiar, R., Poess, M., Raab, F., Rabl, T., Ravi, N., Sachs, K., Sen, S., Yi, L., Youn, C.: Discussion of BigBench: a proposed industry standard performance benchmark for big data

    Google Scholar 

  11. Spark Dynamic Allocation. http://spark.apache.org/docs/latest/job-scheduling.html#dynamic-resource-allocation

  12. Blanas, S., Patel, M., Ercegovac, V., Rao, J., Shekita, J., Tian, Y.: A comparison of join algorithms for log processing in MapReduce. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of data, pp. 975–986

    Google Scholar 

  13. Spark Memory Management. http://spark.apache.org/docs/latest/configuration.html#memory-management

  14. Ganelin, I., Orhian, E., Sasaki, K., York, B.: SparkTM: Big Data Cluster Computing in Production. Wiley, New York (2016). Chap. 2

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Intel Corporation, Shanghai, China

    Yan Tang, Jack Chen, Xin Hao, Yi Zhou, Yi Yao & Lifeng Wang

  2. Intel Corporation, Santa Clara, California, USA

    Gowda Bhaskar

Authors
  1. Yan Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gowda Bhaskar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jack Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xin Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yi Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yi Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Lifeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yan Tang , Gowda Bhaskar , Jack Chen , Xin Hao , Yi Zhou , Yi Yao or Lifeng Wang .

Editor information

Editors and Affiliations

  1. Technical University of Berlin, Berlin, Germany

    Tilmann Rabl

  2. Cisco Systems, Inc., San Jose, California, USA

    Raghunath Nambiar

  3. University of California at San Diego, La Jolla, California, USA

    Chaitanya Baru

  4. Ampool, Inc., Santa Clara, California, USA

    Milind Bhandarkar

  5. Oracle Corporation, Redwood Shores, California, USA

    Meikel Poess

  6. Indian Institute of Public Health, Hyderabad, India

    Saumyadipta Pyne

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Tang, Y. et al. (2016). Accelerating BigBench on Hadoop. In: Rabl, T., Nambiar, R., Baru, C., Bhandarkar, M., Poess, M., Pyne, S. (eds) Big Data Benchmarking. WBDB WBDB 2015 2015. Lecture Notes in Computer Science(), vol 10044. Springer, Cham. https://doi.org/10.1007/978-3-319-49748-8_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-49748-8_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-49747-1

  • Online ISBN: 978-3-319-49748-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics