Abstract
Big data is large volume of data produced on a daily basis. Distributed storage systems (DSS) is environment that handles, manages and stores those data. The main drawbacks are the lack of system storage capacity, the network device failures that can appear anytime, on time data processing and the system efficiency. All above mentioned issues can be overcome by applying different coding techniques for data distribution. Till this moment many coding schemes are proposed by the researchers. Determining the most efficient code for usage is still a tricky question, which yields an adequate comparison strategy for code selection. The basic Dimakis comparison method offers analysis between the codes regarding the parameters storage per node and download bandwidth to be repair one node. Total comparison method includes in the analysis the total number of nodes in the system together with the overall storage and total downloaded bandwidth in the repair process, with notation that the file size for all codes must be same. In this paper, we are proposing new method for comparison, called Normalized, that enables consideration of broader spectrum of parameters and not necessarily the same file size of the proposed codes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Moon, T.K.: Error Correction Coding: Mathematical Methods and Algorithms. Wiley-Interscience, Hoboken (2005)
MacWilliams, F.J., Sloane, N.J.A.: The Theory of Error-Correcting Codes. North Holland Mathematical Library, Amsterdam, The Netherlands (1983)
Rawat, A.S., Tamo, I., Guruswami, V., Efremenko, K.: MDS code constructions with small sub-packetization and near-optimal repair bandwidth. Trans. Inf. Theory IEEE 64, 6506–6525 (2018)
Guruswami, V., Wootters, M.: Repairing Reed-Solomon codes. Trans. Inf. Theory IEEE 63, 5684–5698 (2017)
Weatherspoon, H., Kubiatowicz, J.: Erasure coding vs. replication: a quantitative comparison. In: Proceedings of 1st International Workshop Peer-to-Peer System (IPTPS), pp. 328–338 (2001)
Sathiamoorthy, M., et al.: Xoring elephants: Novel erasure codes for big data. Proc. VLDB Endow. 6, 325–336 (2013)
Memorandum of understanding for the implementation of the COST Action, European Cooperation for Statistics of Network Data Science (2015)
Dimakis*, A.G., Prabhakaran, V., Ramchandran, K.: Decentralized erasure code for distributed storage. Trans. Inf. Theory IEEE/ACM Netw., (2006)
Rawat, A.S., Koyluoglu, O.O., Silberstein, N., Vishwanath, S.: Optimal locally repairable and secure codes for distributed storage system. Info. Theory IEEE Trans. 60, 212–236 (2013)
Rashmi, K.V., Shah, N.B., Kumar, P.V.: Regenerating codes for errors and erasures in distributed storage. In: Proceedings of IEEE International Symposium on Information Theory (ISIT) (2012)
Dimakis, A.G., Godfrey, P.B., Wu, Y., Wainright, M.J., Ramchandran, K.: Network coding for distributed storage systems. IEEE Trans. Inf. Theory 57(8), 5227–5239 (2011)
Dimakis, A.G., Godfrey, P.B., Wainright, M., Ramchadran, K.: Network coding for distributed storage systems. In: Proceedings of 26th IEEE International Conference on Computer Communications, Anchorage, AK, pp. 2000–2008, May 2007
Han, Y.S., Zheng, R., Mow, W.H.: Exact regenerating codes for byzantine fault tolerance in distributed storage. In: INFOCOM Proceedings, pp. 2498–2506 (2012)
Han, Y.S., Pai, H.T., Zheng, R., Varshney, P.K.: Update-efficient regenerating codes with minimum per-node storage. In: Information Theory Proceedings (ISIT), pp. 1436–14406 (2013)
Goparaju, S., Tamo, I., Calderbank, R.: An improved sub-packetization bound for minimum storage regenerating codes. IEEE Information Theory Transactions, pp. 2770–2779 (2014)
Rashmi, K.V., Shah, N.B., Kumar, P.V.: Optimal exact-regenerating codes for distributed storage at the MSR and MBR points via a product-matrix construction. IEEE Trans. Inf. Theory 57(8), 5227–5239 (2011)
Paunkoska, N., Finamore, W., Karamachoski, J., Puncheva, M., Marina, N.: Improving DSS Efficiency with Shortened MSR Codes. ICUMT (2016)
Paunkoska, N., Finamore, W., Marina, N.: Fair Comparison of DSS Codes. In: Future of Information and Communication Conference (FICC 2018) (2018)
Vajha, M., et al.: Clay codes: moulding MDS codes to yield an MSR code. In: Proceedings of the 16th USENIX Conference on File and Storage Technologies (FAST 2018), USENIX Association, USA, pp. 139–153 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Paunkoska (Dimoska), N., Marina, N., Finamore, W. (2021). Normalized Comparison Method for Finding the Most Efficient DSS Code. In: Zitouni, R., Phokeer, A., Chavula, J., Elmokashfi, A., Gueye, A., Benamar, N. (eds) Towards new e-Infrastructure and e-Services for Developing Countries. AFRICOMM 2020. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 361. Springer, Cham. https://doi.org/10.1007/978-3-030-70572-5_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-70572-5_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-70571-8
Online ISBN: 978-3-030-70572-5
eBook Packages: Computer ScienceComputer Science (R0)