Dimitri Van Landuyt
ABSTRACT
Multi-model databases support different NoSQL data models at once, typically a combination of relational, key-value, document, and graph models. Their expected benefits include increased versatility, reduced installation complexity, improved database performance, and smaller storage footprint. However, these benefits are insufficiently investigated, and current studies lack a comparison between multi-model databases and equivalent polyglot database architectures, i.e. setups that combine different technologies to accomplish the same functionality.
To fill this gap, we conduct a series of benchmarks for the purpose of investigating the efficiency of two multi-model NoSQL databases (ArangoDB and OrientDB) in comparison to an equivalent polyglot baseline. These experiments have been performed with UniBench, a framework specifically designed to evaluate the performance of multi-model databases in terms of query execution time, which was also further extended for the purposes of this study.
The study results indicate that the choice between a multi-model NoSQL database and a combination of NoSQL databases depends on the types of data and queries, the underlying data models and the involved databases. These outcomes emphasize the necessity for database architects to prototype and evaluate alternative storage architectures in function of specific application requirements before committing to a technology or paradigm.
- Jemal Abawajy. 2015. Comprehensive analysis of big data variety landscape. International journal of parallel, emergent and distributed systems 30, 1 (2015), 5--14.Google Scholar
Digital Library
- Veronika Abramova, Jorge Bernardino, and Pedro Furtado. 2014. Experimental Evaluation of NoSQL Databases. International journal of database management systems 6, 3 (2014), 1--16.Google ScholarCross Ref
- A. Buble, L. Bulej, and P. Tuma. 2003. CORBA benchmarking: a course with hidden obstacles. In Proceedings International Parallel and Distributed Processing Symposium. 6 pp.-. Google ScholarCross Ref
- Luca Cabibbo. 2013. ONDM: an Object-NoSQL Datastore Mapper. Faculty of Engineering, Roma Tre University. Retrieved June 15th (2013).Google Scholar
- Cody Coleman, Deepak Narayanan, Daniel Kang, Tian Zhao, Jian Zhang, Luigi Nardi, Peter Bailis, Kunle Olukotun, Chris Ré, and Matei Zaharia. 2017. Dawn-bench: An end-to-end deep learning benchmark and competition. Training 100, 101 (2017), 102.Google Scholar
- Brian F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears. 2010. Benchmarking cloud serving systems with YCSB. In Proceedings of the 1st ACM symposium on Cloud computing. 143--154.Google ScholarDigital Library
- DB-Engines. 2021. Ranking. https://db-engines.com/en/ranking.Google Scholar
- Claudio de Lima and Ronaldo dos Santos Mello. 2015. A workload-driven logical design approach for NoSQL document databases. In Proceedings of the 17th International Conference on Information Integration and Web-based Applications & Services. 1--10.Google ScholarDigital Library
- Priya Dialani. 2022. The Future of Data Revolution will be Unstructured Data. https://www.analyticsinsight.net/the-future-of-data-revolution-will-be-unstructured-data/.Google Scholar
- Ahmad Ghazal, Todor Ivanov, Pekka Kostamaa, Alain Crolotte, Ryan Voong, Mohammed Al-Kateb, Waleed Ghazal, and Roberto V Zicari. 2017. Bigbench V2: the new and improved bigbench. In 2017 IEEE 33rd International Conference on Data Engineering (ICDE). IEEE, 1225--1236.Google ScholarCross Ref
- Zhiqiang Gong, Ping Zhong, and Weidong Hu. 2019. Diversity in machine learning. IEEE Access 7 (2019), 64323--64350.Google ScholarCross Ref
- Katarina Grolinger, Wilson A Higashino, Abhinav Tiwari, and Miriam AM Capretz. 2013. Data management in cloud environments: NoSQL and NewSQL data stores. Journal of Cloud Computing: advances, systems and applications 2, 1 (2013), 1--24.Google ScholarDigital Library
- Rohmat Gunawan, Alam Rahmatulloh, and Irfan Darmawan. 2019. Performance Evaluation of Query Response Time in The Document Stored NoSQL Database. In 2019 16th International Conference on Quality in Research (QIR): International Symposium on Electrical and Computer Engineering. IEEE, 1--6.Google Scholar
- Cláudio Lima and Ronaldo Santos Mello. 2016. On proposing and evaluating a NoSQL document database logical approach. International Journal of Web Information Systems (2016).Google ScholarCross Ref
- David Lion, Adrian Chiu, Hailong Sun, Xin Zhuang, Nikola Grcevski, and Ding Yuan. 2016. Don't Get Caught in the Cold, Warm-up Your JVM: Understand and Eliminate JVM Warm-up Overhead in Data-Parallel Systems. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16). 383--400.Google ScholarDigital Library
- Matteo Lissandrini, Martin Brugnara, and Yannis Velegrakis. 2018. Beyond macrobenchmarks: microbenchmark-based graph database evaluation. Proceedings of the VLDB Endowment 12, 4 (2018), 390--403.Google ScholarDigital Library
- Jiaheng Lu and Irena Holubová. 2019. Multi-model databases: a new journey to handle the variety of data. ACM Computing Surveys (CSUR) 52, 3 (2019), 1--38.Google ScholarDigital Library
- Martin Macak, Matus Stovcik, Barbora Buhnova, and Michal Merjavy. 2020. How well a multi-model database performs against its single-model variants: Benchmarking OrientDB with Neo4j and MongoDB. In 2020 15th Conference on Computer Science and Information Systems (FedCSIS). IEEE, 463--470.Google ScholarCross Ref
- Peter Mattson, Vijay Janapa Reddi, Christine Cheng, Cody Coleman, Greg Diamos, David Kanter, Paulius Micikevicius, David Patterson, Guenther Schmuelling, Hanlin Tang, et al. 2020. MLPerf: An industry standard benchmark suite for machine learning performance. IEEE Micro 40, 2 (2020), 8--16.Google ScholarCross Ref
- Jean Moschetta and Giuliano Casale. 2012. OFBench: An Enterprise Application Benchmark for Cloud Resource Management Studies. In 2012 14th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing. 393--399. Google ScholarDigital Library
- Fábio Roberto Oliveira and Luis del Val Cura. 2016. Performance evaluation of NoSQL multi-model data stores in polyglot persistence applications. In Proceedings of the 20th International Database Engineering & Applications Symposium. 230--235.Google ScholarDigital Library
- Omoruyi Osemwegie, Kennedy Okokpujie, Nsikan Nkordeh, Charles Ndujiuba, John Samuel, and Uzairue Stanley. 2018. Performance Benchmarking of Key-Value Store NoSQL Databases. International Journal of Electrical and Computer Engineering 8, 6 (2018), 5333--5341.Google Scholar
- Diogo Augusto Pereira, Wagner Ourique de Morais, and Edison Pignaton de Freitas. 2018. NoSQL real-time database performance comparison. International Journal of Parallel, Emergent and Distributed Systems 33, 2 (2018), 144--156.Google ScholarCross Ref
- Ewa Płuciennik and Kamil Zgorzałek. 2017. The multi-model databases-a review. In International Conference: Beyond Databases, Architectures and Structures. Springer, 141--152.Google ScholarCross Ref
- Tilmann Rabl, Christoph Brücke, Philipp Härtling, Stella Stars, Rodrigo Escobar Palacios, Hamesh Patel, Satyam Srivastava, Christoph Boden, Jens Meiners, and Sebastian Schelter. 2019. ADABench-Towards an industry standard benchmark for advanced analytics. In Technology Conference on Performance Evaluation and Benchmarking. Springer, 47--63.Google Scholar
- Dharavath Ramesh, Ekaansh Khosla, and Shankar Nayak Bhukya. 2016. Inclusion of e-commerce workflow with NoSQL DBMS: MongoDB document store. In 2016 IEEE international conference on computational intelligence and computing research (ICCIC). IEEE, 1--5.Google ScholarCross Ref
- Vincent Reniers, Ansar Rafique, Dimitri Van Landuyt, and Wouter Joosen. 2017. Object-NoSQL Database Mappers: a benchmark study on the performance overhead. Journal of Internet Services and Applications 8, 1 (2017), 1--16.Google ScholarCross Ref
- Noa Roy-Hubara, Peretz Shoval, and Arnon Sturm. 2022. Selecting databases for Polyglot Persistence applications. Data & Knowledge Engineering 137 (2022), 101950.Google ScholarDigital Library
- Pramod J Sadalage and Martin Fowler. 2013. NoSQL distilled: a brief guide to the emerging world of polyglot persistence. Pearson Education.Google Scholar
- Michael Stonebraker. 2010. Errors in database systems, eventual consistency, and the cap theorem. Communications of the ACM, BLOG@ ACM (2010).Google Scholar
- Uta Störl, Thomas Hauf, Meike Klettke, and Stefanie Scherzinger. 2015. Schemaless NoSQL data stores-Object-NoSQL Mappers to the rescue? Datenbanksysteme für Business, Technologie und Web (BTW 2015) (2015).Google Scholar
- Enqing Tang and Yushun Fan. 2016. Performance Comparison between Five NoSQL Databases. In 2016 7th International Conference on Cloud Computing and Big Data (CCBD). IEEE, 105--109.Google ScholarCross Ref
- Dimitri Van Landuyt, Julien Benaouda, Vincent Reniers, Ansar Rafique, and Wouter Joosen. 2023. A Comparative Performance Evaluation of Multi-Model NoSQL Databases and Polyglot Persistence: study results (SF1, SF10 and SF30). people.cs.kuleuven.be/~dimitri.vanlanduyt/dvanlanduyt_sac-dbdm_2023_results.zip.Google Scholar
- Luís HN Villaça, Leonardo G Azevedo, and Fernanda Baião. 2018. Query strategies on polyglot persistence in microservices. In Proceedings of the 33rd Annual ACM Symposium on Applied Computing. 1725--1732.Google ScholarDigital Library
- Lei Wang, Jianfeng Zhan, Chunjie Luo, Yuqing Zhu, Qiang Yang, Yongqiang He, Wanling Gao, Zhen Jia, Yingjie Shi, Shujie Zhang, et al. 2014. Bigdatabench: A big data benchmark suite from internet services. In 2014 IEEE 20th international symposium on high performance computer architecture (HPCA). IEEE, 488--499.Google ScholarCross Ref
- Chao Zhang. 2022. UniBench - Towards Benchmarking the Multi-Model DBMS. https://github.com/HY-UDBMS/UniBench.Google Scholar
- Chao Zhang and Jiaheng Lu. 2021. Holistic evaluation in multi-model databases benchmarking. Distributed and Parallel Databases 39, 1 (2021), 1--33.Google ScholarDigital Library
- Chao Zhang, Jiaheng Lu, Pengfei Xu, and Yuxing Chen. 2018. UniBench: A benchmark for multi-model database management systems. In Technology Conference on Performance Evaluation and Benchmarking. Springer, 7--23.Google Scholar
Index Terms
- A Comparative Performance Evaluation of Multi-Model NoSQL Databases and Polyglot Persistence
Recommendations
Performance Evaluation of NoSQL Multi-Model Data Stores in Polyglot Persistence Applications
IDEAS '16: Proceedings of the 20th International Database Engineering & Applications SymposiumNoSQL data store systems have recently been introduced as alternatives to traditional relational database management systems. These data stores systems implement simpler and scalable data models that increase the performance and efficiency of a new kind ...
A performance evaluation of NoSQL databases to manage proteomics data
NoSQL databases have recently been introduced as alternatives to traditional relational database management systems because of their capabilities in terms of storing data and query retrieval. Biological datasets can be modelled using various models, for ...
A performance evaluation of NoSQL databases to manage proteomics data
NoSQL databases have recently been introduced as alternatives to traditional relational database management systems because of their capabilities in terms of storing data and query retrieval. Biological datasets can be modelled using various models, for ...
Comments