Abstract
Distributed key-value stores are a scalable alternative for relational databases and file systems. Different systems try to provide the right balance between scalability, consistency and reliability for a certain category of applications. Yet many applications don’t have a single static set of requirements for all the data they create and use. In order to meet different requirements, the applications end up using multiple data stores, each with its own interface, or suffer the unnecessary performance degradation. This paper describes Hop, our attempt to define an interface that allows specification of different consistency and replication levels, and by allowing the assembly of different implementations of the interface, provides the right balance of consistency, scalability and availability for different parts of application data. We describe the Hop interface and evaluate some of the implementations we created, as well as how they can be combined to provide flexible distributed deployments.
LA-UR-13-27636
The rights of this work are transferred to the extent transferable according to title 17 
105 U.S.C.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brightwell, R., Oldfield, R., Maccabe, A.B., Bernholdt, D.E.: Hobbes: composition and virtualization as the foundations of an extreme-scale OS/R. In: Proceedings of the 3rd International Workshop on Runtime and Operating Systems for Supercomputers, ROSS 2013, pp. 2:1–2:8. ACM, New York (2013). http://doi.acm.org/10.1145/2491661.2481427
DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: amazon’s highly available key-value store. In: Proceedings of Twenty-First ACM SIGOPS Symposium on Operating Systems Principles, SOSP 2007, pp. 205–220. ACM, New York (2007). http://doi.acm.org/10.1145/1294261.1294281
Eastlake, D., Fowler, G., Vo, K.P., Noll, L.: The fnv non-cryptographic hash algorithm (2012)
Feinberg, A.: Project voldemort: reliable distributed storage. In: Proceedings of the 10th IEEE International Conference on Data Engineering (2011)
Fitzpatrick, B.: Distributed caching with memcached. Linux J. 2004(124), 5 (2004). http://dl.acm.org/citation.cfm?id=1012889.1012894
Geambasu, R., Levy, A.A., Kohno, T., Krishnamurthy, A., Levy, H.M.: Comet: an active distributed key-value store. In: OSDI. pp. 323–336 (2010)
Ionkov, L., Lang, M., Maltzahn, C.: Drepl: optimizing access to application data for analysis and visualization. In: 2013 IEEE 29th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–11 (2013)
Kubiatowicz, J., Bindel, D., Chen, Y., Czerwinski, S., Eaton, P., Geels, D., Gummadi, R., Rhea, S., Weatherspoon, H., Weimer, W., et al.: Oceanstore: an architecture for global-scale persistent storage. ACM Sigplan Not. 35(11), 190–201 (2000)
Lakshman, A., Malik, P.: Cassandra: a decentralized structured storage system. SIGOPS Oper. Syst. Rev. 44, 35–40 (2010). http://doi.acm.org/10.1145/1773912.1773922
Lamport, L.: Paxos made simple. ACM Sigact News 32(4), 18–25 (2001)
Li, T., Verma, R., Duan, X., Jin, H., Raicu, I.: Exploring distributed hash tables in highend computing. SIGMETRICS Perform. Eval. Rev. 39(3), 128–130 (2011). http://doi.acm.org/10.1145/2160803.2160880
Lloyd, W., Freedman, M.J., Kaminsky, M., Andersen, D.G.: Don’t settle for eventual: scalable causal consistency for wide-area storage with cops. In: Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, pp. 401–416. ACM (2011)
Ongaro, D., Ousterhout, J.: In search of an understandable consensus algorithm
Stoica, I., Morris, R., Karger, D., Kaashoek, M.F., Balakrishnan, H.: Chord: a scalable peer-to-peer lookup service for internet applications. In: ACM SIGCOMM Computer Communication Review, vol. 31, pp. 149–160. ACM (2001)
Acknowledgments
This work was performed at the Ultrascale Systems Research Center in the High Performance Computing Division at Los Alamos National Laboratory, and is supported by the U.S. Department of Energy DE-FC02-06ER25750.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland (outside the US)
About this paper
Cite this paper
Ionkov, L., Lang, M. (2015). Hop: Elastic Consistency for Exascale Data Stores. In: Kunkel, J., Ludwig, T. (eds) High Performance Computing. ISC High Performance 2015. Lecture Notes in Computer Science(), vol 9137. Springer, Cham. https://doi.org/10.1007/978-3-319-20119-1_22
Download citation
DOI: https://doi.org/10.1007/978-3-319-20119-1_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-20118-4
Online ISBN: 978-3-319-20119-1
eBook Packages: Computer ScienceComputer Science (R0)