Abstract
User-intensive software, such asWeb andmobile applications, heavily depends on the interactions with large and unknown populations of users. Knowing the preferences and behaviors of these populations is crucial for the success of this class of systems. A/B testing is an increasingly popular technique that supports the iterative development of userintensive software based on controlled experiments performed on live users. However, as currently performed, A/B testing is a time consuming, error prone and costly manual activity. In this paper, we investigate a novel approach to automate A/B testing. More specifically, we rephrase A/B testing as a search-based software engineering problem and we propose an initial approach that supports automated A/B testing through aspect-oriented programming and genetic algorithms.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Genetify, https://github.com/gregdingle/genetify/wiki (accessed February 25, 2014)
JBoss AOP, http://www.jboss.org/jbossaop (accessed February 25, 2014)
The A/B Test: Inside the Technology Thats Changing the Rules of Business, http://www.wired.com/business/2012/04/ff_abtesting (accessed February 25, 2014)
Afzal, W., Torkar, R., Feldt, R.: A systematic review of search-based testing for non-functional system properties. Inf. Softw. Technol. (2009)
Crook, T., Frasca, B., Kohavi, R., Longbotham, R.: Seven pitfalls to avoid when running controlled experiments on the web. In: ACM SIGKDD, KDD 2009, pp. 1105–1114. ACM, New York (2009)
Deb, K.: An efficient constraint handling method for genetic algorithms. Computer Methods in Applied Mechanics and Engineering 186(2-4), 311–338 (2000)
Harman, M., Jones, B.F.: Search-based software engineering. Information and Software Technology 43(14), 833–839 (2001)
Kiczales, G., Lamping, J., Mendhekar, A., Maeda, C., Lopes, C., Loingtier, J.-M., Irwin, J.: Aspect-oriented programming. In: Akşit, M., Matsuoka, S. (eds.) ECOOP 1997. LNCS, vol. 1241, pp. 220–242. Springer, Heidelberg (1997)
Kohavi, R., Deng, A., Frasca, B., Longbotham, R., Walker, T., Xu, Y.: Trustworthy online controlled experiments: Five puzzling outcomes explained. In: ACM SIGKDD, pp. 786–794. ACM (2012)
Kohavi, R., Deng, A., Frasca, B., Walker, T., Xu, Y., Pohlmann, N.: Online controlled experiments at large scale. In: ACM SIGKDD, KDD 2013, pp. 1168–1176. ACM, New York (2013)
Kohavi, R., Henne, R.M., Sommerfield, D.: Practical guide to controlled experiments on the web: Listen to your customers not to the hippo. In: ACM SIGKDD, pp. 959–967. ACM (2007)
Kohavi, R., Longbotham, R.: Unexpected results in online controlled experiments. ACM SIGKDD 12(2), 31–35 (2011)
Koza, J.R.: Genetic programming: On the programming of computers by means of natural selection, vol. 1. MIT Press (1992)
Meffert, K., Rotstan, N., Knowles, C., Sangiorgi, U.: JGAP - Java Genetic Algorithms and Genetic Programming Package (2014)
O’Keeffe, M., Cinneide, M.O.: Search-based software maintenance. In: Proceedings of the 10th European Conference on Software Maintenance and Reengineering, CSMR 2006, p. 10, 260 (March 2006)
Räihä, O.: A survey on search-based software design. Computer Science Review 4(4), 203–249 (2010)
Zhang, Y., Finkelstein, A., Harman, M.: Search based requirements optimisation: Existing work and challenges. In: Rolland, C. (ed.) REFSQ 2008. LNCS, vol. 5025, pp. 88–94. Springer, Heidelberg (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Tamburrelli, G., Margara, A. (2014). Towards Automated A/B Testing. In: Le Goues, C., Yoo, S. (eds) Search-Based Software Engineering. SSBSE 2014. Lecture Notes in Computer Science, vol 8636. Springer, Cham. https://doi.org/10.1007/978-3-319-09940-8_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-09940-8_13
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-09939-2
Online ISBN: 978-3-319-09940-8
eBook Packages: Computer ScienceComputer Science (R0)