doi: 10.17706/jsw.16.4.174-181
Architecting an Enterprise Data Lake, A Covid19 Case Study
2,3Mehran University of Engineering and Technology, Jamshoro, Sindh.
Abstract—Data is increasing at an enormous rate every day. Traditionally data has resided in silosacross any organization,so it’s difficult to have a complete picture for data driven business decision making. Data lake addresses the problem of rate of increase of data by providing “schema on read”, better integration and cheaper storage. It also solves the data silos problemby providing a central platform for a variety of data housing needs. However, implementing a data lake becomes challenging as the implementation needs to address the additional needs like metadata management, data discovery, data governance, data lifecycle management, security and centralized access controls mechanisms. This paper intends to provide a comprehensive architecture of data lake to address these challenges. We have also conducted and documented our experiments with publicly available datasets about COVID19 to validate the design and applicability of the proposed architecture for business analytics purposes.
Index Terms—Big data, data lake, data governance, data lake management, serverless architecture.
Cite: Bushra, Mohsin Ali Memon, Salahuddin Saddar, "Architecting an Enterprise Data Lake, A Covid19 Case Study," Journal of Software vol. 16, no. 4, pp. 174-181, 2021.
Copyright © 2021 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0)
General Information
-
Mar 01, 2024 News!
Vol 19, No 1 has been published with online version [Click]
-
Apr 26, 2021 News!
Vol 14, No 4- Vol 14, No 12 has been indexed by IET-(Inspec) [Click]
-
Nov 18, 2021 News!
Papers published in JSW Vol 16, No 1- Vol 16, No 6 have been indexed by DBLP [Click]
-
Jan 04, 2024 News!
JSW will adopt Article-by-Article Work Flow
-
Nov 02, 2023 News!
Vol 18, No 4 has been published with online version [Click]