Liang Zheng
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Amazon's Elastic Compute Cloud (EC2) uses auction-based spot pricing to sell spare capacity, allowing users to bid for cloud resources at a highly reduced rate. Amazon sets the spot price dynamically and accepts user bids above this price. Jobs with lower bids (including those already running) are interrupted and must wait for a lower spot price before resuming. Spot pricing thus raises two basic questions: how might the provider set the price, and what prices should users bid? Computing users' bidding strategies is particularly challenging: higher bid prices reduce the probability of, and thus extra time to recover from, interruptions, but may increase users' cost. We address these questions in three steps: (1) modeling the cloud provider's setting of the spot price and matching the model to historically offered prices, (2) deriving optimal bidding strategies for different job requirements and interruption overheads, and (3) adapting these strategies to MapReduce jobs with master and slave nodes having different interruption overheads. We run our strategies on EC2 for a variety of job sizes and instance types, showing that spot pricing reduces user cost by 90% with a modest increase in completion time compared to on-demand pricing.
Supplemental Material
- Agmon Ben-Yehuda, O., Ben-Yehuda, M., Schuster, A., and Tsafrir, D. Deconstructing Amazon EC2 spot instance pricing. ACM Trans. on Economics and Computation 1, 3 (2013), 1--16. Google Scholar
Digital Library
- Amazon. EC2 spot instance, 2015. http://aws.amazon.com/ec2/purchasing-options/spot-instances/.Google Scholar
- Ardagna, D., Panicucci, B., and Passacantando, M. A game theoretic formulation of the service provisioning problem in cloud systems. In Proc. of WWW (2011), ACM, pp. 177--186. Google ScholarDigital Library
- Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., et al. A view of cloud computing. Communications of the ACM 53, 4 (2010), 50--58. Google ScholarDigital Library
- Blum, A., Sandholm, T., and Zinkevich, M. Online algorithms for market clearing. Journal of the ACM (JACM) 53, 5 (2006), 845--879. Google ScholarDigital Library
- Buyya, R., Yeo, C. S., and Venugopal, S. Market-oriented cloud computing: Vision, hype, and reality for delivering IT services as computing utilities. In Proc. of IEEE HPCC (2008), IEEE, pp. 5--13. Google ScholarDigital Library
- Common Crawl. Common Crawl Corpus. online at: http://commoncrawl.org/ (2015).Google Scholar
- Dogar, F. R., Karagiannis, T., Ballani, H., and Rowstron, A. Decentralized task-aware scheduling for data center networks. In Proc. of ACM SIGCOMM (2014). Google ScholarDigital Library
- Fang, X., Misra, S., Xue, G., and Yang, D. Smart grid -- The new and improved power grid: A survey. IEEE Communications Surveys & Tutorials 14, 4 (2012), 944--980.Google ScholarCross Ref
- Feng, G., Garg, S., Buyya, R., and Li, W. Revenue maximization using adaptive resource provisioning in cloud computing environments. In Proc. of ACM/IEEE Grid (2012), pp. 192--200. Google ScholarDigital Library
- Friedman, E. J., and Parkes, D. C. Pricing WiFi at Starbucks: Issues in online mechanism design. In Proc. of ACM EC (2003), ACM, pp. 240--241. Google ScholarDigital Library
- Guenter, B., Jain, N., and Williams, C. Managing cost, performance, and reliability tradeoffs for energy-aware server provisioning. In Proc. of IEEE INFOCOM (2011), IEEE, pp. 1332--1340.Google ScholarCross Ref
- Ha, S., Sen, S., Joe-Wong, C., Im, Y., and Chiang, M. TUBE: time-dependent pricing for mobile data. In Proc. of ACM SIGCOMM (2012). Google ScholarDigital Library
- Jain, N., Menache, I., Naor, J. S., and Yaniv, J. Near-optimal scheduling mechanisms for deadline-sensitive jobs in large computing clusters. ACM Trans. on Parallel Computing 2, 1 (2015), 3. Google ScholarDigital Library
- Javadi, B., Thulasiram, R. K., and Buyya, R. Statistical modeling of spot instance prices in public cloud environments. In Proc. of IEEE/ACM UCC (2011), IEEE/ACM, pp. 219--228. Google ScholarDigital Library
- Jia, J., Zhang, Q., Zhang, Q., and Liu, M. Revenue generation for truthful spectrum auction in dynamic spectrum access. In Proc. of ACM MobiHoc (2009), pp. 3--12. Google ScholarDigital Library
- Jin, H., Wang, X., Wu, S., Di, S., and Shi, X. Towards optimized fine-grained pricing of IaaS cloud platform. IEEE Trans. on Cloud Computing (2014), to appear.Google Scholar
- Juan, D.-C., Li, L., Peng, H.-K., Marculescu, D., and Faloutsos, C. Beyond poisson: Modeling inter-arrival time of requests in a datacenter. In Advances in Knowledge Discovery and Data Mining. Springer, 2014, pp. 198--209.Google ScholarCross Ref
- Kang, L., and Parkes, D. C. A decentralized auction framework to promote efficient resource allocation in open computational grids. In Proc. Joint Workshop on The Economics of Networked Systems and Incentive-Based Computing (2007).Google Scholar
- Kutanoglu, E., and Wu, S. D. On combinatorial auction and Lagrangean relaxation for distributed resource scheduling. IIE Trans. 31, 9 (1999), 813--826.Google ScholarCross Ref
- Lampe, U., Hans, R., Seliger, M., and Pauly, M. Pricing in infrastructure clouds--an analytical and empirical examination. In Association for Information Systems Conference (2014).Google Scholar
- Lee, G., Chun, B.-G., and Katz, R. H. Heterogeneity-aware resource allocation and scheduling in the cloud. Proc. of HotCloud (2011), 1--5. Google ScholarDigital Library
- Leonardi, E., Mellia, M., Neri, F., and Ajmone Marsan, M. Bounds on average delays and queue size averages and variances in input-queued cell-based switches. In Proc. of IEEE INFOCOM (2001), vol. 2, IEEE, pp. 1095--1103.Google ScholarCross Ref
- MacKie-Mason, J., and Varian, H. Pricing the Internet. In Public Access to the Internet, B. Kahin and J. Keller, Eds. Prentice-Hall, Englewood Cliffs, NJ, 1995. Google ScholarDigital Library
- Menache, I., Ozdaglar, A., and Shimkin, N. Socially optimal pricing of cloud computing resources. In Proc. of VALUETOOLS (2011), ICST, pp. 322--331. Google ScholarDigital Library
- Nair, J., Subramanian, V. G., and Wierman, A. On competitive provisioning of cloud services. ACM SIGMETRICS Performance Evaluation Review 42, 2 (2014), 30--32. Google ScholarDigital Library
- Ng, C., Parkes, D. C., and Seltzer, M. Virtual worlds: Fast and strategyproof auctions for dynamic resource allocation. In Proc. of ACM EC (2003), ACM, pp. 238--239. Google ScholarDigital Library
- Poola, D., Ramamohanarao, K., and Buyya, R. Fault-tolerant workflow scheduling using spot instances on clouds. Procedia Computer Science 29 (2014), 523--533.Google ScholarCross Ref
- Porter, R. Mechanism design for online real-time scheduling. In Proc. of ACM EC (2004), ACM, pp. 61--70. Google ScholarDigital Library
- Sen, S., Jin, Y., Guérin, R., and Hosanagar, K. Modeling the dynamics of network technology adoption and the role of converters. IEEE/ACM Trans. on Networking 18, 6 (2010), 1793--1805. Google ScholarDigital Library
- Teng, F., and Magoulès, F. A new game theoretical resource allocation algorithm for cloud computing. In Advances in Grid and Pervasive Computing. Springer, 2010, pp. 321--330. Google ScholarDigital Library
- Upadhyaya, P., Balazinska, M., and Suciu, D. How to price shared optimizations in the cloud. Proc. of the VLDB Endowment 5, 6 (2012), 562--573. Google ScholarDigital Library
- Van den Bossche, R., Vanmechelen, K., and Broeckhove, J. Cost-optimal scheduling in hybrid IaaS clouds for deadline constrained workloads. In Proc. of IEEE CLOUD (2010), IEEE, pp. 228--235. Google ScholarDigital Library
- Wang, P., Qi, Y., Hui, D., Rao, L., and Liu, X. Present or future: Optimal pricing for spot instances. In Proc. of IEEE ICDCS (2013). Google ScholarDigital Library
- Wang, Q., Ren, K., and Meng, X. When cloud meets eBay: Towards effective pricing for cloud computing. In Proc. of IEEE INFOCOM (2012), IEEE, pp. 936--944.Google ScholarCross Ref
- Wellman, M. P., MacKie-Mason, J. K., Reeves, D. M., and Swaminathan, S. Exploring bidding strategies for market-based scheduling. In Proc. of ACM EC (2003), ACM, pp. 115--124. Google ScholarDigital Library
- Yi, S., Andrzejak, A., and Kondo, D. Monetary cost-aware checkpointing and migration on Amazon cloud spot instances. IEEE Trans. on Services Computing 5, 4 (2012), 512--524. Google ScholarDigital Library
- Zhang, L., Li, Z., and Wu, C. Dynamic resource provisioning in cloud computing: A randomized auction approach. In Proc. of IEEE INFOCOM (2014).Google ScholarCross Ref
- Zhang, Q., Zhu, Q., and Boutaba, R. Dynamic resource allocation for spot markets in cloud computing environments. In Proc. of IEEE/ACM UCC (2011), IEEE/ACM, pp. 178--185. Google ScholarDigital Library
- Zhou, Y., and Wentzlaff, D. The sharing architecture: sub-core configurability for IaaS clouds. In Proc. of ACM ASPLOS (2014), ACM, pp. 559--574. Google ScholarDigital Library
Index Terms
- How to Bid the Cloud
Recommendations
How to Bid the Cloud
SIGCOMM '15: Proceedings of the 2015 ACM Conference on Special Interest Group on Data CommunicationAmazon's Elastic Compute Cloud (EC2) uses auction-based spot pricing to sell spare capacity, allowing users to bid for cloud resources at a highly reduced rate. Amazon sets the spot price dynamically and accepts user bids above this price. Jobs with ...
Pricing the cloud based on multi-attribute auction mechanism
AbstractCloud computing is a dynamic paradigm which aims to provide on-demand computing resources to a user over the Internet. Most of the existing cloud providers use fixed pricing strategy. However, a few cloud service providers have recently priced ...
The shared assignment game and applications to pricing in cloud computing
AAMAS '14: Proceedings of the 2014 international conference on Autonomous agents and multi-agent systemsWe propose an extension to the Assignment Game [37] in which sellers provide indivisible heterogeneous goods to their buyers. Each good takes up various amounts of resources and each seller has capacity constraints with respect to the total amount of ...
Comments