Skip to main content
SpringerLink
Log in

Research of CouchDB Storage Plugin for Big Data Query Engine Apache Drill

  • Conference paper
  • First Online:
Big Data (BigData 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1120))

Included in the following conference series:

Abstract

Currently, the document-oriented database supported by Apache Drill is only MongoDB. However, due to the lack of data model, application interface, security and usability of MongoDB, Apache Drill is limited in querying and processing document data. CouchDB is an emerging document-oriented database. Compared to MongoDB, CouchDB has the advantage of supporting triggers, running in Android and BSD environments, rendering in JSON format, and supporting any language that supports HTTP requests, but CouchDB has low query performance and does not support standard SQL queries. Therefore, the research on the CouchDB storage plugin for Apache Drill makes sense. This paper first researches the basic architecture of CouchDB and Apache Drill and the query flow of Apache Drill, and the ValueVector data structure, then designs and implements CouchDB storage plugin based on Apache Drill’s storage plugin specification and CouchDB’s application programming interface. With a simple configuration, users can use CouchDB as a data source for the Apache Drill query engine. Experiments show that the CouchDB Storage Plugin not only further enhances Apache Drill’s query and management capabilities for document-oriented data, but also enables quick query of CouchDB with SQL and greatly improves CouchDB’s query performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Melnik, S., et al.: Dremel: interactive analysis of web-scale datasets. Proc. VLDB Endow. 3(1–2), 330–339 (2010)

    Article  Google Scholar 

  2. Hausenblas, M., Nadeau, J.: Apache drill: interactive ad-hoc analysis at scale. Big Data 1(2), 100–104 (2013)

    Article  Google Scholar 

  3. Liu, T., Martonosi, M.: Impala: a middleware system for managing autonomic, parallel sensor systems. ACM SIGPLAN Not. 38(10), 107–118 (2003)

    Article  Google Scholar 

  4. Zaharia, M., Chowdhury, M., Franklin, M.J., et al.: Spark: cluster computing with working sets. In: Usenix Conference on Hot Topics in Cloud Computing (2010)

    Google Scholar 

  5. Zaharia, M., et al.: Spark: cluster computing with working sets. In: HotCloud 2010, vol. 10, p. 95 (2010)

    Google Scholar 

  6. Kimball, R., Caserta, J.: The Data Warehouse ETL Toolkit: Practical Techniques for Extracting, Cleaning, Conforming, and Delivering Data. Wiley, Hoboken (2011)

    Google Scholar 

  7. Vora, M.N.: Hadoop-HBase for large-scale data. In: Proceedings of 2011 International Conference on Computer Science and Network Technology, vol. 1, pp. 601–605. IEEE, December 2011

    Google Scholar 

  8. Thusoo, A., et al.: Hive: a warehousing solution over a map-reduce framework. Proc. VLDB Endow. 2(2), 1626–1629 (2009)

    Article  Google Scholar 

  9. Chodorow, K.: MongoDB: The Definitive Guide: Powerful and Scalable Data Storage. O’Reilly Media, Inc., Sebastopol (2013)

    Google Scholar 

  10. CouchDB vs. MongoDB Comparison. https://db-engines.com/en/system/CouchDB%3BCouchbase%3BMongoDB. Accessed 17 Apr 2019

  11. Lamb, J.P., Lew, P.W.: Lotus Notes Network Design. McGraw-Hill, Inc., New York (1996)

    Google Scholar 

  12. Androulaki, E., et al.: Hyperledger fabric: a distributed operating system for permissioned blockchains. In: Proceedings of the Thirteenth EuroSys Conference, p. 30. ACM, April 2018

    Google Scholar 

  13. Using CouchDB—hyperledger-fabricdocs master documentation. https://hyperledger-fabric.readthedocs.io/en/release-1.4/couchdb_tutorial.html. Accessed 15 July 2019

  14. HBase Storage Plugin - Apache Drill. http://drill.apache.org/docs/hbase-storage-plugin/. Accessed 15 July 2019

  15. Hive Storage Plugin - Apache Drill. http://drill.apache.org/docs/hive-storage-plugin/. Accessed 15 July 2019

  16. RDBMS Storage Plugin - Apache Drill. http://drill.apache.org/docs/rdbms-storage-plugin/. Accessed 15 July 2019

  17. MongoDB Storage Plugin - Apache Drill. http://drill.apache.org/docs/mongodb-storage-plugin/. Accessed 15 July 2019

Download references

Author information

Authors and Affiliations

  1. College of Computer Science, Sichuan Normal University, Chengdu, 610101, Sichuan, China

    Yulei Liao & Liang Tan

  2. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China

    Liang Tan

Authors
  1. Yulei Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liang Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liang Tan .

Editor information

Editors and Affiliations

  1. Huazhong University of Science and Technology, Wuhan, China

    Hai Jin

  2. East China Normal University, Shanghai, China

    Xuemin Lin

  3. Chinese Academy of Sciences, Beijing, China

    Xueqi Cheng

  4. Huazhong University of Science and Technology, Wuhan, China

    Xuanhua Shi

  5. National University of Defense Technology, Changsha, China

    Nong Xiao

  6. Nanjing University, Nanjing, China

    Yihua Huang

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liao, Y., Tan, L. (2019). Research of CouchDB Storage Plugin for Big Data Query Engine Apache Drill. In: Jin, H., Lin, X., Cheng, X., Shi, X., Xiao, N., Huang, Y. (eds) Big Data. BigData 2019. Communications in Computer and Information Science, vol 1120. Springer, Singapore. https://doi.org/10.1007/978-981-15-1899-7_16

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-1899-7_16

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-1898-0

  • Online ISBN: 978-981-15-1899-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation