Takayuki Osogami
ABSTRACT
The performance of a Web system can be greatly improved by tuning its configuration parameters. However, finding the optimal configuration has been a time-consuming task due to the long measurement time needed to evaluate the performance of a given configuration. We propose an algorithm, which we refer to as Quick Optimization via Guessing (QOG), that quickly selects one of nearly best configurations with high probability. The key ideas in QOG are (i) the measurement of a configuration is terminated as soon as the configuration is found to be suboptimal, and (ii) the performance of a configuration is guessed at based on the measured similar configurations, so that the better configurations are more likely to be measured before the others. If the performance of a good configuration has been measured, a poor configuration will be quickly found to be suboptimal with short measurement time. We apply QOG to optimizing the configuration of a real Web system, and find that QOG can drastically reduce the total measurement time needed to select the best configuration. Our experiments also illuminate several interesting properties of QOG specifically when it is applied to optimizing Web systems.
- T. W. Anderson. A modification of the sequential probability ratio test to reduce the sample size. Annals of Mathematical Statistics, 31:165--197, 1960.Google Scholar
Cross Ref
- R. E. Bechhofer, T. J. Santner, and D. M. Goldsman. Design and analysis of experiments for statistical selection, screening, and multiple comparisons. John Wiley & Sons, 1995.Google Scholar
- I. Chung and J. K. Hollingsworth. Automated cluster--based web service performance tuning. In Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing, pages 36--44, June 2004. Google ScholarDigital Library
- L. J. Hong and B. L. Nelson. The tradeoff between sampling and switching: New sequential procedures for indifference-zone selection. IIE Transactions, 37(7):623--634, 2005.Google ScholarCross Ref
- S. Kim and B. L. Nelson. A fully sequential procedure for indifference-zone selection in simulation. ACM Transactions on Modeling and Computer Simulation, 11(3):251--273, 2001. Google ScholarDigital Library
- A. M. Law and W. D. Kelton. Simulation Modeling and Analysis. McGraw-Hill, third edition, 2000. Google ScholarDigital Library
- X. Liu, L. Sha, S. Froehlich, J. L. Hellerstein, and S. Parekh. Online response time optimization of Apache Web server. In Proceedings of the 11th International Workshop on Quality of Service (IWQoS 2003), pages 461--478, June 2003. Google ScholarDigital Library
- T. M. Mitchell. Machine Learning. McGraw-Hill, 1997. Google ScholarDigital Library
- B. L. Nelson, J. Swann, D. Goldsman, and W. Song. Simple procedures for selecting the best simulated system when the number of alternatives is large. Operations Research, 49(6):950--963, 2001. Google ScholarDigital Library
- T. Osogami. Finding probably best systems quickly via simulations. Technical Report RT0684, IBM Tokyo Research Laboratory, 2006.Google ScholarCross Ref
- T. Osogami and T. Itoko. Finding probably better system configurations quickly. In Proceedings of the Joint International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS/PERFORMANCE 2006), pages 264--275, June 2006. Google ScholarDigital Library
- M. Raghavachari, D. Reimer, and R. D. Johnson. The deployer's problem: Configuring application servers for performance and reliability. In Proceedings of the 25th International Conference on Software Engineering, pages 484--489, 2003. Google ScholarDigital Library
- J. R. Swisher, S. H. Jacobson, and E. Yücesan. Discrete-event simulation optimization using ranking, selection, and multiple comparison procedures: A survey. ACM Transactions on Modeling and Computer Simulation, 13(2):134--154, 2003. Google ScholarDigital Library
- B. Xi, Z. Liu, M. Raghavachari, C. H. Xia, and L. Zhang. A smart hill-climbing algorithm for application server configuration. In Proceedings of the 13th International Conference on World Wide Web, pages 287--296, 2004. Google ScholarDigital Library
- Y. Zhang, W. Qu, and A. Liu. Automatic performance tuning for J2EE application server systems. In Proceedings of the 6th International Conference on Web Information Systems Engineering (WISE 2005), pages 520--527, November 2005. Google ScholarDigital Library
Index Terms
- Optimizing system configurations quickly by guessing at the performance
Recommendations
BestConfig: tapping the performance potential of systems via automatic configuration tuning
SoCC '17: Proceedings of the 2017 Symposium on Cloud ComputingAn ever increasing number of configuration parameters are provided to system users. But many users have used one configuration setting across different workloads, leaving untapped the performance potential of systems. A good configuration setting can ...
Optimizing system configurations quickly by guessing at the performance
SIGMETRICS '07 Conference ProceedingsThe performance of a Web system can be greatly improved by tuning its configuration parameters. However, finding the optimal configuration has been a time-consuming task due to the long measurement time needed to evaluate the performance of a given ...
Finding probably better system configurations quickly
SIGMETRICS '06/Performance '06: Proceedings of the joint international conference on Measurement and modeling of computer systemsThe performance of computer and communication systems can in theory be optimized by iteratively finding better system configurations. However, a bottleneck is the time required in simulations/experiments for finding a better system configuration in each ...
Comments