Skip to main content
SpringerLink
Log in

Performance impacts of hybrid cloud storage

  • Published:
Computing Aims and scope Submit manuscript

Abstract

Cloud storage has seen unprecedented growth in recent times. Consumer electronics contribute significant amount of data to the cloud and as these devices become an inseparable part of our daily lives, it is critical to have insight into the performance impacts of hybrid storage that involves user equipment. In this work, we present performance analysis of hybrid cloud storage assuming multiple scenarios of failure among storage locations. We also present a comparative analysis between multiple existing storage coding techniques based on chunk formation and varying file sizes. We analytically show a composite distribution that caters for retrieval, processing and re-distribution of chunks. Analytical analysis is also presented and compared with simulated results. Results are useful for design of hybrid cloud storage systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Apple: Use the health app on your iphone or ipod touch. https://support.apple.com/en-us/HT203037. Accessed 05 Jan 2016

  2. Babcock C (2011) When amazon’s cloud turned on itself. http://www.informationweek.com/cloud/infrastructure-as-a-service/post-mortem-when-amazons-cloud-turned-on-itself/d/d-id/1097465. Accessed 03 May 2011

  3. Babcock C (2014) Dropbox Takes Blame For Cloud Outage. http://www.informationweek.com/cloud/cloud-storage/dropbox-takes-blame-for-cloud-outage/d/d-id/1113427. Accessed 16 Jan 2014

  4. Blaum M, Brady J, Bruck J, Menon J (1995) EVENODD: an efficient scheme for tolerating double disk failures in RAID architectures. IEEE Trans Comput 44(2):192–202. doi:10.1109/12.364531

    Article  MATH  Google Scholar 

  5. Blomer J, Kalfane M, Karpinski M, Karp R, Luby M, Zuckerman D (1995) An XOR-based erasure-resilient coding scheme. Technical Report TR-95-048, International Computer Science Institute

  6. Cadambe VR, Jafar SA, Maleki H, Ramchandran K, Suh C (2013) Asymptotic interference alignment for optimal repair of MDS codes in distributed data storage. IEEE Trans Inf Theory 59(5):2974–2987

    Article  MATH  Google Scholar 

  7. Calder B, Wang J, Ogus A, Nilakantan N, Skjolsvold A, McKelvie S, Xu Y, Srivastav S, Wu J, Simitci H, Haridas J, Uddaraju C, Khatri H, Edwards A, Bedekar V, Mainali S, Abbasi R, Agarwal A, ul Haq, MF, ul Haq, MI, Bhardwaj D, Dayanand S, Adusumilli A, McNett M, Sankaran S, Manivannan K, Rigas L (2011) Windows azure storage: a highly available cloud storage service with strong consistency. In: Proceedings of the 23rd ACM symposium on operating systems principles 2011, SOSP 2011, Cascais, 23–26 October 2011, pp 143–157. doi:10.1145/2043556.2043571

  8. Calderón A, Carballeira FG, Sánchez LM, García JD, Fernández J (2009) Fault tolerant file models for parallel file systems: introducing distribution patterns for every file. J Supercomput 47(3):312–334. doi:10.1007/s11227-008-0199-8

    Article  Google Scholar 

  9. Chang F, Dean J, Ghemawat S, Hsieh WC, Wallach DA, Burrows M, Chandra T, Fikes A, Gruber R (2006) Bigtable: a distributed storage system for structured data (awarded best paper!). In: 7th Symposium on operating systems design and implementation (OSDI ’06), 6–8 November , Seattle, pp 205–218. http://www.usenix.org/events/osdi06/tech/chang.html

  10. Claburn T (2009) Gmail outage ’A Big’, says Google. http://www.informationweek.com/applications/gmail-outage-a-big-deal-says-google/d/d-id/1082782?. Accessed 15 Oct 2009

  11. Corbett PF, English R, Goel A, Grcanac T, Kleiman S, Leong J, Sankar S (2004) Row-diagonal parity for double disk failure correction (awarded best paper!). In: Proceedings of the FAST ’04 conference on file and storage technologies, Grand hyatt hotel, San Francisco, 31 March–2 April 2004, pp 1–14. http://www.usenix.org/events/fast04/tech/corbett.html

  12. Dimakis AG, Ramchandran K, Wu Y, Suh C (2011) A survey on network codes for distributed storage. Proc IEEE 99(3):476–489. doi:10.1109/JPROC.2010.2096170

    Article  Google Scholar 

  13. Drago I, Mellia M, Munafo M, Sperotto A, Sadre R, Pras A (2012) Inside dropbox. In: Proceedings of the 2012 ACM conference on internet measurement conference - IMC ’12, ACM Press, New York, p 481. doi:10.1145/2398776.2398827

  14. Ford D, Labelle F, Popovici FI, Stokely M, Truong V, Barroso L, Grimes C, Quinlan S (2010) Availability in globally distributed storage systems. In: 9th USENIX symposium on operating systems design and implementation, OSDI 2010, Vancouver, 4–6 October 2010, Proceedings, pp 61–74. http://www.usenix.org/events/osdi10/tech/full_papers/Ford.pdf

  15. Ghemawat S, Gobioff H, Leung S (2003) The google file system. In: Proceedings of the 19th ACM symposium on operating systems principles 2003, SOSP 2003, Bolton Landing, 19–22 October , pp 29–43. doi:10.1145/945445.945450

  16. Hadoop (2013) HDFS Architecture Guide. https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html. Accessed 04 Aug 2013

  17. Hafner JL (2005) WEAVER codes: highly fault tolerant erasure codes for storage systems. In: Proceedings of the FAST ’05 conference on file and storage technologies, San Francisco, 13–16 December 2005. http://www.usenix.org/events/fast05/tech/hafner_weaver.html

  18. Hafner JL, Deenadhayalan V, Rao KK, Tomlin JA (2005) Matrix methods for lost data reconstruction in erasure codes. In: Proceedings of the FAST ’05 conference on file and storage technologies, San Francisco, 13–16 December 2005. http://www.usenix.org/events/fast05/tech/hafner_matrix.html

  19. Hamming R (1950) Error detecting and error correcting codes. Bell Syst Tech J 29:147–160. doi:10.1002/j.1538-7305.1950.tb00463.x

    Article  MathSciNet  Google Scholar 

  20. Hu Y, Chen HCH, Lee PPC, Tang Y (2012) Nccloud: applying network coding for the storage repair in a cloud-of-clouds. In: Proceedings of the 10th USENIX conference on file and dtorage technologies, FAST 2012, San Jose, 14–17 February 2012, p 21. https://www.usenix.org/conference/fast12/nccloud-applying-network-coding-storage-repair-cloud-clouds

  21. Huang C, Simitci H, Xu Y, Ogus A, Calder B, Gopalan P, Li J, Yekhanin S (2012) Erasure coding in windows azure storage. In: 2012 USENIX annual technical conference, Boston, 13–15 June 2012, pp 15–26. https://www.usenix.org/conference/atc12/technical-sessions/presentation/huang

  22. InFO FAA (2011) The Apple iPad and other suitable tablet computing devices as Electronic flight bags (EFB). Technical report, US Department of Transportation

  23. Iosup A, Ostermann S, Yigitbasi N, Prodan R, Fahringer T, Epema DHJ (2011) Performance analysis of cloud computing services for many-tasks scientific computing. IEEE Trans Parallel Distrib Syst 22(6):931–945. doi:10.1109/TPDS.2011.66

    Article  Google Scholar 

  24. Jiekak S, Kermarrec A, Scouarnec NL, Straub G, van Kempen A (2013) Regenerating codes: a system perspective. Oper Syst Rev 47(2):23–32. doi:10.1145/2506164.2506170

    Article  Google Scholar 

  25. Kelly G (2014) The great Facebook outage of September 2014: social media reaction. http://www.mercurynews.com/storify/ci_26458755/great-facebook-outage-september-2014-social-media-reaction. 10 Sep 2014

  26. Li A, Yang X, Kandula S, Zhang M (2010) Cloudcmp: comparing public cloud providers. In: Proceedings of the 10th ACM SIGCOMM internet measurement conference, IMC 2010, Melbourne, 1–3 November 2010, pp 1–14. doi:10.1145/1879141.1879143

  27. Palankar M, Iamnitchi A, Ripeanu M, Garfinkel S (2013) Amazon S3 for science grids: a viable solution. In: Proceedings of the 2008 international workshop on data-aware distributed computing, pp 55–64. doi:10.1145/1383519.1383526

  28. Pamies-Juarez L, Datta A, Oggier FE (2013) Rapidraid: pipelined erasure codes for fast data archival in distributed storage systems. In: Proceedings of the IEEE INFOCOM 2013, Turin, 14–19 April 2013, pp 1294–1302. doi:10.1109/INFCOM.2013.6566922

  29. Papailiopoulos DS, Luo J, Dimakis AG, Huang C, Li J (2012) Simple regenerating codes: network coding for cloud storage. In: Proceedings of the IEEE INFOCOM 2012, Orlando, 25–30 March, pp 2801–2805. doi:10.1109/INFCOM.2012.6195703

  30. Paxson V, Floyd S (1995) Wide area traffic: the failure of poisson modeling. IEEE/ACM Trans Netw 3(3):226–244. doi:10.1109/90.392383

    Article  Google Scholar 

  31. Persico V, Montieri A, Pescapè A (2016) On the network performance of amazon S3 cloud-storage service. In: 5th IEEE international conference on cloud networking, Cloudnet 2016, Pisa, 3–5 October 2016, pp 113–118. doi:10.1109/CloudNet.2016.16

  32. Plank JS (2013) Erasure codes for storage systems. USENIX LOGIN 38(6):44–50

    Google Scholar 

  33. Plank JS, Blaum M (2014) Sector-disk (sd) erasure codes for mixed failure modes in raid systems. TOS 10(1):4:1–4:17. doi:10.1145/2560013

    Article  Google Scholar 

  34. Reed IS, Solomon G (1960) Polynomial codes over certain finite fields. J Soc Ind Appl Math 8(2):300–304. doi:10.1137/0108018

    Article  MATH  MathSciNet  Google Scholar 

  35. Sathiamoorthy M, Asteris M, Papailiopoulos DS, Dimakis AG, Vadali R, Chen S, Borthakur D (2013) Xoring elephants: novel erasure codes for big data. PVLDB 6(5): 325–336. http://www.vldb.org/pvldb/vol6/p325-sathiamoorthy.pdf

  36. Schroeder B, Gibson GA (2007) Disk failures in the real world: what does an MTTF of 1, 000, 000 hours mean to you? In: 5th USENIX conference on file and storage technologies, FAST 2007, San Jose, 13–16 February 2007, pp 1–16. http://www.usenix.org/events/fast07/tech/schroeder.html

  37. Shieh A, Kandula S, Greenberg AG, Kim C, Saha B (2011) Sharing the data center network. In: Proceedings of the 8th USENIX symposium on networked systems design and implementation, NSDI 2011, Boston, 30 March–1 April 2011. https://www.usenix.org/conference/nsdi11/sharing-data-center-network

  38. Sit E, Haeberlen A, Dabek F, Chun B, Weatherspoon H, Morris R, Kaashoek MF, Kubiatowicz J (2006) Proactive replication for data durability. In: 5th International workshop on peer-to-peer systems, IPTPS 2006, Santa Barbara, 27–28 February 2006. http://www.iptps.org/papers-2006/Sit-tempo.pdf

  39. Teodorescu S, Vernic R (2006) A composite exponential-pareto distribution. Annal Ovidius Univ Constanta 14(1):99–108

    MATH  MathSciNet  Google Scholar 

  40. Wang C, Chow SSM, Wang Q, Ren K, Lou W (2013) Privacy-preserving public auditing for secure cloud storage. IEEE Trans Comput 62(2):362–375. doi:10.1109/TC.2011.245

    Article  MATH  MathSciNet  Google Scholar 

  41. Xiang L, Xu Y, Lui JCS, Chang Q (2010) Optimal recovery of single disk failure in RDP code storage systems. In: SIGMETRICS 2010, proceedings of the 2010 ACM SIGMETRICS international conference on measurement and modeling of computer systems, New York, 14–18 June 2010, pp 119–130. doi:10.1145/1811039.1811054

  42. Xiang L, Xu Y, Lui JCS, Chang Q, Pan Y, Li R (2011) A hybrid approach to failed disk recovery using RAID-6 codes: algorithms and performance evaluation. TOS 7(3):11:1–11:34. doi:10.1145/2027066.2027071

    Article  Google Scholar 

  43. Xu S, Li R, Lee PPC, Zhu Y, Xiang L, Xu Y, Lui JCS (2014) Single disk failure recovery forx-code-based parallel storage systems. IEEE Trans Comput 63(4):995–1007. doi:10.1109/TC.2013.8

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. SEECS, National University of Sciences and Technology, Islamabad, Pakistan

    Muhammad Umar Hameed & Syed Ali Haider

  2. University of Ottawa, Ottawa, Canada

    Burak Kantarci

Authors
  1. Muhammad Umar Hameed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Syed Ali Haider
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Burak Kantarci
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Syed Ali Haider.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hameed, M.U., Haider, S.A. & Kantarci, B. Performance impacts of hybrid cloud storage. Computing 99, 1207–1229 (2017). https://doi.org/10.1007/s00607-017-0560-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00607-017-0560-y

Keywords

Mathematics Subject Classification

Search

Navigation