Big Data DBMS Assessment: A Systematic Mapping Study

Abstract

The tremendous prosperity of big data systems that has occurred in recent years has made its understanding crucial for both research and industrial communities. Big Data is expected to generate an economy of 15 billion euros over the next few years and to have repercussions that will more or less directly change the way in which we live. It is, therefore, important for organizations to have quality Database Management Systems (DBMSs) that will allow them to manage large volumes of data in real time and according to their needs. The last decade has witnessed an explosion of new Database Management Systems (DBMSs) which deal not only with relational Data Bases but also with non-relational Data Bases. Companies need to assess DBMS quality in order, for example, to select which DBMS is most appropriate for their needs. The main research question formulated in this research is, therefore, “What is the state of the art of Big Data DBMS assessment?”, which we attempt to answer by following a well-known methodology called “Systematic Mapping Studies” (SMS). This paper describes an SMS of papers published until May 2016. Five digital libraries were searched, and 19 papers were identified and classified into five dimensions: quality characteristics of Big Data DBMSs, techniques and measures used to assess the quality characteristics, DBMSs whose quality has been measured, evolution over time and research methods utilized. The results indicate that there are several benchmarks, which are principally focused on the performance of MongoDB and Cassandra, and that the interest in Big Data DBMS quality is growing. Nonetheless, more research is needed in order to define and validate a quality model that will bring together all the relevant characteristics of DBMSs for Big Data and their respective measures. This quality model will then be employed as a basis on which to build benchmarks for DBMSs, covering not only the diversity of DBMSs and application scenarios and types of applications, but also diverse and representative real-world data sets.