Abstract
Highly distributed NoSQL key-value data stores are rapidly becoming the favorite choice for hosting modern web applications. Such platforms rely on data partitioning, replication and relaxed consistency to achieve high levels of performance and scalability. However, these design choices often exhibit a trade-off between latency (i.e., Quality of Service (QoS)) and consistency (i.e., Quality of Data (QoD)). In this work, in addition to latency-based SLAs, we also adopt the application tolerance to data staleness as another requirement determining the end-user satisfaction and our goal is to strike a fine balance between both the QoS and QoD provided to the end-user. Towards achieving that goal, we propose a suite of quality-aware schedulers for efficiently allocating the necessary computational resources between the foreground user-queries and the background system-updates at data store nodes. This suite of schedulers features our proposed Freshness/Tardiness (FIT) mechanism, which introduces a novel selective approach for scheduling the execution of queries and updates. Our experimental results show that FIT provides significant improvements in balancing the trade-off between QoS and QoD under both the state-transfer and operation-transfer update propagation models employed in current key-value data stores.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig1_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig2_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig3_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig4_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig5_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig6_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig7_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig8_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig9_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig10_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig11_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig12_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig13_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig14_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig15_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig16_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig17_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig18_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig19_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig20_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig21_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10619-013-7136-4/MediaObjects/10619_2013_7136_Fig22_HTML.gif)
Similar content being viewed by others
References
Abadi, D.: Consistency tradeoffs in modern distributed database system design: cap is only part of the story. Computer 45(2), 37–42 (2012)
Adelberg, B., Garcia-Molina, H., Kao, B.: Applying update streams in a soft real-time database system. In: SIGMOD Conference, pp. 245–256 (1995)
Agrawal, D., Abbadi, A.E., Antony, S., Das, S.: Data management challenges in cloud computing infrastructures. In: DNIS, pp. 1–10 (2010)
Aiyer, A.S., Anderson, E., Li, X., Shah, M.A., Wylie, J.J.: Consistability: describing usually consistent systems. In: HotDep (2008)
Bailis, P., Venkataraman, S., Franklin, M.J., Hellerstein, J.M., Stoica, I.: Probabilistically bounded staleness for practical partial quorums. Proc. VLDB Endow. 5(8), 776–787 (2012)
Bateni, M., Golab, L., Hajiaghayi, M.T., Karloff, H.J.: Scheduling to minimize staleness and stretch in real-time data warehouses. In: SPAA, pp. 29–38 (2009)
Bouzeghoub, M., Peralta, V.: A framework for analysis of data freshness. In: IQIS, pp. 59–67 (2004)
Brinch Hansen, P. (ed.): Classic Operating Systems: from Batch Processing to Distributed Systems. Springer, New York (2000)
Buttazzo, G.C., Spuri, M., Sensini, F.: Value vs. deadline scheduling in overload conditions. In: RTSS, pp. 90–99 (1995)
Cattell, R.: Scalable sql and nosql data stores. SIGMOD Rec. 39(4), 12–27 (2010)
Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows, M., Chandra, T., Fikes, A., Gruber, R.E., Bigtable: A distributed storage system for structured data. ACM Trans. Comput. Syst. 26(2) (2008)
Chen, X., Zhang, X.: Coordinated data prefetching for web contents. Comput. Commun. 28(17), 1947–1958 (2005)
Chi, Y., Moon, H.J., Hacigümüs, H.: Icbs: incremental cost-based scheduling under piecewise linear slas. Proc. VLDB Endow. 4(9), 563–574 (2011)
Chi, Y., Moon, H.J., Hacigümüs, H., Tatemura, J.: Sla-tree: a framework for efficiently supporting sla-based decisions in cloud computing. In: EDBT, pp. 129–140 (2011)
Cho, J., Garcia-Molina, H.: Synchronizing a database to improve freshness. In: SIGMOD Conference, pp. 117–128 (2000)
Cooper, B.F., Ramakrishnan, R., Srivastava, U., Silberstein, A., Bohannon, P., Jacobsen, H.A., Puz, N., Weaver, D., Yerneni, R.: Pnuts: Yahoo!’s hosted data serving platform. Proc. VLDB Endow. 1(2), 1277–1288 (2008)
Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with ycsb. In: SoCC, pp. 143–154 (2010)
DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: amazon’s highly available key-value store. In: SOSP, pp. 205–220 (2007)
Gilbert, S., Lynch, N.A.: Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News 33(2), 51–59 (2002)
Golab, L., Johnson, T., Shkapenyuk, V.: Scheduling updates in a real-time stream warehouse. In: ICDE, pp. 1207–1210 (2009)
Golab, W.M., Li, X., Shah, M.A.: Analyzing consistency properties for fun and profit. In: PODC, pp. 197–206 (2011)
Guirguis, S., Sharaf, M.A., Chrysanthis, P.K., Labrinidis, A., Pruhs, K.: Adaptive scheduling of web transactions. In: ICDE, pp. 357–368 (2009)
Guo, H., Larson, P., Ramakrishnan, R., Goldstein, J.: Relaxed currency and consistency: how to say “good enough” in sql. In: SIGMOD Conference, pp. 815–826 (2004)
Gustafsson, T., Hansson, J.: Dynamic on-demand updating of data in real-time database systems. In: SAC, pp. 846–853 (2004)
Haritsa, J.R., Livny, M., Carey, M.J.: Earliest deadline scheduling for real-time database systems. In: RTSS, pp. 232–242 (1991)
Haritsa, J.R., Carey, M.J., Livny, M.: Value-based scheduling in real-time database systems. VLDB J. 2(2), 117–152 (1993)
Hewitt, E.: Cassandra: the Definitive Guide. O’Reilly Media, Sebastopol (2010)
Kang, K.D., Son, S.H., Stankovic, J.A., Abdelzaher, T.F.: A qos-sensitive approach for timeliness and freshness guarantees in real-time databases. In: ECRTS, pp. 203–212 (2002)
Kang, K.D., Son, S.H., Stankovic, J.A.: Managing deadline miss ratio and sensor data freshness in real-time databases. IEEE Trans. Knowl. Data Eng. 16(10), 1200–1216 (2004)
Labrinidis, A., Roussopoulos, N.: Update propagation strategies for improving the quality of data on the web. In: VLDB, pp. 391–400 (2001)
Labrinidis, A., Qu, H., Xu, J.: Quality contracts for real-time enterprises. In: BIRTE, pp. 143–156 (2006)
Lakshman, A., Malik, P.: Cassandra: a decentralized structured storage system. Oper. Syst. Rev. 44(2), 35–40 (2010)
Li, W., Zhang, W., Liberatore, V., Penkrot, V., Beaver, J., Sharaf, M.A., Roychowdhury, S., Chrysanthis, P.K., Pruhs, K.: An optimized multicast-based data dissemination middleware. In: ICDE, pp. 762–764 (2003)
Pang, H., Carey, M.J., Livny, M.: Multiclass query scheduling in real-time database systems. IEEE Trans. Knowl. Data Eng. 7(4), 533–551 (1995)
Qu, H., Labrinidis, A.: Preference-aware query and update scheduling in web-databases. In: ICDE, pp. 356–365 (2007)
Qu, H., Labrinidis, A., Mossé, D.: Unit: User-centric transaction management in web-database systems. In: ICDE, p. 33 (2006)
Ramakrishnan, R.: Cap and cloud data management. Computer 45(2), 43–49 (2012)
Saito, Y., Shapiro, M.: Optimistic replication. ACM Comput. Surv. 37(1), 42–81 (2005)
Sharaf, M.A., Chrysanthis, P.K.: Facilitating mobile decision making. In: Workshop Mobile Commerce, pp. 45–53 (2002)
Sharaf, M.A., Sismanis, Y., Labrinidis, A., Chrysanthis, P.K., Roussopoulos, N.: Efficient dissemination of aggregate data over the wireless web. In: WebDB, pp. 93–98 (2003)
Sharaf, M.A., Guirguis, S., Labrinidis, A., Pruhs, K., Chrysanthis, P.K.: Poster session: asets: a self-managing transaction scheduler. In: ICDE Workshops, pp. 56–62 (2008)
Sharaf, M.A., Chrysanthis, P.K., Labrinidis, A., Amza, C.: Optimizing i/o-intensive transactions in highly interactive applications. In: SIGMOD Conference, pp. 785–798 (2009)
Sharaf, M.A., Xu, C., Zhou, X.: Finding the silver lining for data freshness on the cloud: [extended abstract]. In: CloudDB, pp. 49–50 (2012)
Silberstein, A., Terrace, J., Cooper, B.F., Ramakrishnan, R.: Feeding frenzy: selectively materializing users’ event feeds. In: SIGMOD Conference, pp. 831–842 (2010)
Sivasubramanian, S., Pierre, G., van Steen, M., Alonso, G.: Analysis of caching and replication strategies for web applications. IEEE Internet Comput. 11(1), 60–66 (2007)
Thiele, M., Bader, A., Lehner, W.: Multi-objective scheduling for real-time data warehouses. In: BTW, pp. 307–326 (2009)
Thomas, R.H.: A majority consensus approach to concurrency control for multiple copy databases. ACM Trans. Database Syst. 4(2), 180–209 (1979)
Wada, H., Fekete, A., Zhao, L., Lee, K., Liu, A.: Data consistency properties and the trade-offs in commercial cloud storage: the consumers’ perspective. In: CIDR, pp. 134–143 (2011)
Xu, C., Sharaf, M.A., Zhou, M., Zhou, A., Zhou, X.: Adaptive query scheduling in key-value data stores. In: DASFAA, vol. 1, pp. 86–100 (2013)
Zhu, Y., Sharaf, M.A., Zhou, X.: Scheduling with freshness and performance guarantees for web applications in the cloud. In: ADC, pp. 133–142 (2011)
Zhu, Y., Yu, P.S., Wang, J.: Recods: replica consistency-on-demand store. In: ICDE, pp. 1360–1363 (2013)
Acknowledgements
The authors would like to thank the anonymous reviewers for their valuable comments and suggestions. This work is partially supported by the Shanghai Knowledge Services Platform for Trustworthy Internet of Things (ZF1213), grants from Australian Research Council (DP110102777, DP120102829), MoST of China (2010CB731402, 2012AA011001, 2012AA011003) and NSFC grants (61021004, 60925008).
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Beng Chin Ooi.
Rights and permissions
About this article
Cite this article
Xu, C., Sharaf, M.A., Zhou, X. et al. Quality-aware schedulers for weak consistency key-value data stores. Distrib Parallel Databases 32, 535–581 (2014). https://doi.org/10.1007/s10619-013-7136-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10619-013-7136-4