John Ahlgren
ABSTRACT
We introduce the Web-Enabled Simulation (WES) research agenda, and describe FACEBOOK's WW system. We describe the application of WW to reliability, integrity and privacy at FACEBOOK1, where it is used to simulate social media interactions on an infrastructure consisting of hundreds of millions of lines of code. The WES agenda draws on research from many areas of study, including Search Based Software Engineering, Machine Learning, Programming Languages, Multi Agent Systems, Graph Theory, Game AI, and AI Assisted Game Play. We conclude with a set of open problems and research challenges to motivate wider investigation.
- Konstantinos Adamopoulos, Mark Harman, and Robert Mark Hierons. 2004. Mutation Testing Using Genetic Algorithms: A Co-evolution Approach. In Genetic and Evolutionary Computation Conference (GECCO 2004), LNCS 3103. Springer, Seattle, Washington, USA, 1338--1349.Google Scholar
- Sheeva Afshan, Phil McMinn, and Mark Stevenson. 2013. Evolving Readable String Test Inputs Using a Natural Language Model to Reduce Human Oracle Cost. In International Conference on Software Testing, Verification and Validation (ICST 2013). 352--361.Google ScholarDigital Library
- Nadia Alshahwan, Xinbo Gao, Mark Harman, Yue Jia, Ke Mao, Alexander Mols, Taijin Tei, and Ilya Zorin. 2018. Deploying Search Based Software Engineering with Sapienz at Facebook (keynote paper). In 10th International Symposium on Search Based Software Engineering (SSBSE 2018). Montpellier, France, 3--45. Springer LNCS 11036.Google ScholarCross Ref
- Andrea Arcuri and Xin Yao. 2010. Co-evolutionary automatic programming for software development. Information Sciences (2010). https://doi.org/doi:10.1016/j.ins.2009.12.019 To appear. Available on line from Elsevier.Google Scholar
- Earl T. Barr, Mark Harman, Phil McMinn, Muzammil Shahbaz, and Shin Yoo. 2015. The Oracle Problem in Software Testing: A Survey. IEEE Transactions on Software Engineering 41, 5 (May 2015), 507--525.Google ScholarDigital Library
- Jonathan Bell, Gail Kaiser, Eric Melski, and Mohan Dattatreya. 2015. Efficient dependency detection for safe Java test acceleration. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. 770--781.Google ScholarDigital Library
- Antonia Bertolino. 2007. Software testing research: Achievements, challenges, dreams. In Future of Software Engineering (FOSE'07). IEEE, 85--103.Google Scholar
- Dan Boneh, Andrew J. Grotto, Patrick McDaniel, and Nicolas Papernot. 2019. How Relevant is the Turing Test in the Age of Sophisbots? arXiv e-prints, Article arXiv:1909.00056 (Aug 2019), arXiv:1909.00056 pages. arXiv:cs.CY/1909.00056Google Scholar
- Mustafa Bozkurt and Mark Harman. 2011. Automatically generating realistic test input from web services. In IEEE 6th International Symposium on Service Oriented System Engineering (SOSE 2011), Jerry Zeyu Gao, Xiaodong Lu, Muhammad Younas, and Hong Zhu (Eds.). IEEE, Irvine, CA, USA, 13--24.Google ScholarDigital Library
- Cagatay Catal and Banu Diri. 2009. A systematic review of software fault prediction studies. Expert systems with applications 36, 4 (2009), 7346--7354.Google Scholar
- Edmund M Clarke Jr, Orna Grumberg, Daniel Kroening, Doron Peled, and Helmut Veith. 2018. Model checking. MIT press.Google ScholarDigital Library
- Paul Clements and Linda Northrop. 2001. Software Product Lines: Practices and Patterns. Addison-Wesley Professional. 608 pages.Google ScholarDigital Library
- K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6 (April 2002), 182--197. Issue 2.Google ScholarDigital Library
- Dirk Draheim, John Grundy, John Hosking, Christof Lutteroth, and Gerald Weber. 2006. Realistic load testing of web applications. In Conference on Software Maintenance and Reengineering (CSMR '06). IEEE, 11-pp.Google ScholarDigital Library
- FACEBOOK, INC. 2020. Community Standards. https://www.facebook.com/communitystandards/Google Scholar
- Dror G. Feitelson, Eitan Frachtenberg, and Kent L. Beck. 2013. Development and Deployment at Facebook. IEEE Internet Computing 17, 4 (2013), 8--17.Google ScholarDigital Library
- Carlos Gavidia-Calderon, Federica Sarro, Mark Harman, and Earl T. Barr. To appear. The Assessor's Dilemma: Improving Bug Repair via Empirical Game Theory. IEEE Transactions on Software Engineering (To appear).Google Scholar
- Sumit Gulwani, Oleksandr Polozov, Rishabh Singh, et al. 2017. Program synthesis. Foundations and Trends in Programming Languages 4, 1-2 (2017), 1--119.Google ScholarCross Ref
- Mark Harman. 2010. How SBSE Can Support Construction and Analysis of Predictive Models (Keynote Paper). In 6th International Conference on Predictive Models in Software Engineering (PROMISE 2010). Timisoara, Romania.Google Scholar
- Mark Harman. 2010. Search Based Software Engineering (Keynote Paper). In 13th International Conference on Fundamental Approaches to Software Engineering R@(FASE 2010). Paphos, Cyprus.Google Scholar
- Mark Harman, Lin Hu, Robert Mark Hierons, Xingyuan Zhang, Malcolm Munro, José Javier Dolado, Mari Carmen Otero, and Joachim Wegener. 2002. A Post-Placement Side-Effect Removal Algorithm. In IEEE International Conference on Software Maintenance (Montreal, Canada). IEEE Computer Society Press, Los Alamitos, California, USA, 2--11.Google ScholarCross Ref
- Mark Harman, Yue Jia, Jens Krinke, Bill Langdon, Justyna Petke, and Yuanyuan Zhang. 2014. Search based software engineering for software product line engineering: a survey and directions for future work. In 18th International Software Product Line Conference (SPLC 14). Florence, Italy, 5--18.Google ScholarDigital Library
- Mark Harman and Bryan F. Jones. 2001. Search Based Software Engineering. Information and Software Technology 43, 14 (Dec. 2001), 833--839.Google ScholarCross Ref
- Mark Harman, Afshin Mansouri, and Yuanyuan Zhang. 2012. Search Based Software Engineering: Trends, Techniques and Applications. Comput. Surveys 45, 1 (November 2012), 11:1-11:61.Google ScholarDigital Library
- Mark Harman and Peter O'Hearn. 2018. From Start-ups to Scale-ups: Opportunities and Open Problems for Static and Dynamic Program Analysis (keynote paper). In 18th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM 2018). Madrid, Spain, 1--23.Google ScholarCross Ref
- Ahmed E Hassan. 2008. The road ahead for mining software repositories. In 2008 Frontiers of Software Maintenance. IEEE, 48--57.Google Scholar
- Michael Hilton, Nicholas Nelson, Timothy Tunnell, Darko Marinov, and Danny Dig. 2017. Trade-offs in continuous integration: assurance, security, and flexibility. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. 197--207.Google ScholarDigital Library
- Leonid Hurwicz and Stanley Reiter. 2006. Designing Economic Mechanisms. Cambridge University Press.Google Scholar
- Eugene Ie, Chih-wei Hsu, Martin Mladenov, Vihan Jain, Sanmit Narvekar, Jing Wang, Rui Wu, and Craig Boutilier. 2019. RecSim: A Configurable Simulation Platform for Recommender Systems. arXiv e-prints, Article arXiv:1909.04847 (Sep 2019), arXiv:1909.04847 pages. arXiv:cs.LG/1909.04847Google Scholar
- Nicholas R Jennings and Michael J Wooldridge. 1996. Software agents. IEE review (1996), 17--20.Google Scholar
- Dongwon Kang, Jinhwan Jung, and Doo-Hwan Bae. 2011. Constraint-based Human Resource Allocation in Software Projects. Software: Practice and Experience 41, 5 (April 2011), 551--577.Google Scholar
- Jack PC Kleijnen. 2005. Supply chain simulation tools and techniques: a survey. International journal of simulation and process modelling 1, 1-2 (2005), 82--89.Google ScholarCross Ref
- Kamolwan Kunanusont, Raluca D Gaina, Jialin Liu, Diego Perez-Liebana, and Simon M Lucas. 2017. The n-tuple bandit evolutionary algorithm for automatic game improvement. In 2017 IEEE Congress on Evolutionary Computation (CEC). IEEE, 2201--2208.Google ScholarCross Ref
- Ada Augusta Lovelace. 1843. Sketch of the Analytical Engine Invented by Charles Babbage By L. F. Menabrea of Turin, officer of the military engineers, with notes by the translator. (1843). Translation with notes on article in italian in Bibliothèque Universelle de Genève, October, 1842, Number 82.Google Scholar
- Qingzhou Luo, Farah Hariri, Lamyaa Eloussi, and Darko Marinov. 2014. An empirical analysis of flaky tests. In 22nd International Symposium on Foundations of Software Engineering (FSE 2014), Shing-Chi Cheung, Alessandro Orso, and Margaret-Anne Storey (Eds.). ACM, Hong Kong, China, 643--653.Google ScholarDigital Library
- Konstantinos Manikas and Klaus Marius Hansen. 2013. Software ecosystems--A systematic literature review. Journal of Systems and Software 86, 5 (2013), 1294--1306.Google ScholarDigital Library
- Ke Mao, Mark Harman, and Yue Jia. 2017. Robotic Testing of Mobile Apps for Truly Black-Box Automation. IEEE Software 34, 2 (2017), 11--16.Google ScholarDigital Library
- Alexandru Marginean, Johannes Bader, Satish Chandra, Mark Harman, Yue Jia, Ke Mao, Alexander Mols, and Andrew Scott. 2019. SapFix: Automated End-to-End Repair at Scale. In International Conference on Software Engineering (ICSE) Software Engineering in Practice (SEIP) track. Montreal, Canada.Google ScholarDigital Library
- William Martin, Federica Sarro, and Mark Harman. 2016. Causal Impact Analysis for App Releases in Google Play. In 24th ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE 2016). Seattle, WA, USA, 435--446.Google Scholar
- William Martin, Federica Sarro, Yue Jia, Yuanyuan Zhang, and Mark Harman. 2017. A Survey of App Store Analysis for Software Engineering. IEEE Transactions on Software Engineering 43, 9 (2017).Google ScholarCross Ref
- Fabien Michel, Jacques Ferber, and Alexis Drogoul. 2018. Multi-Agent Systems and Simulation: A Survey from the Agent Commu-nity's Perspective. In Multi-Agent Systems. CRC Press, 17--66.Google Scholar
- Roger B Myerson. 2013. Game theory. Harvard university press.Google Scholar
- Armin Najafi, Peter C. Rigby, and Weiyi Shang. 2019. Bisecting Commits and Modeling Commit Risk during Testing. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2019). 279--289.Google ScholarDigital Library
- Judea Pearl. 2000. Causality. Cambridge University Press, Cambridge.Google Scholar
- Justyna Petke, Saemundur O. Haraldsson, Mark Harman, William B. Langdon, David R. White, and John R. Woodward. 2018. Genetic Improvement of Software: a Comprehensive Survey. IEEE Transactions on Evolutionary Computation 22, 3 (June 2018), 415--432. https://doi.org/doi:10.1109/TEVC.2017.2693219Google ScholarCross Ref
- Jian Ren, Mark Harman, and Massimiliano Di Penta. 2011. Cooperative Co-evolutionary Optimization on Software Project Staff Assignments and Job Scheduling. In 3rd International Symposium on Search based Software Engineering (SSBSE 2011). 127--141. LNCS Volume 6956.Google Scholar
- Luis G. Serrano. [n.d.]. Grokking Machine Learning. Manning Publications.Google Scholar
- Dan Siroker and Pete Koomen. 2013. A/B testing: The most powerful way to turn clicks into customers. John Wiley & Sons.Google ScholarDigital Library
- Richard S. Sutton (Ed.). 1992. Reinforcement Learning. SECS, Vol. 173. Kluwer Academic Publishers. Reprinted from volume 8(3-4) (1992) of Machine Learning.Google Scholar
- Julian Togelius and Jurgen Schmidhuber. 2008. An experiment in automatic game design. In 2008 IEEE Symposium On Computational Intelligence and Games. IEEE, 111--118.Google ScholarCross Ref
- Oriol Vinyals, Igor Babuschkin, Wojciech M. Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H. Choi, Richard Powell, Timo Ewalds, Petko Georgiev, Junhyuk Oh, Dan Horgan, Manuel Kroiss, Ivo Danihelka, Aja Huang, Laurent Sifre, Trevor Cai, John P. Agapiou, Max Jaderberg, Alexander S. Vezhnevets, Rémi Leblond, Tobias Pohlen, Valentin Dalibard, David Budden, Yury Sulsky, James Molloy, Tom L. Paine, Caglar Gulcehre, Ziyu Wang, Tobias Pfaff, Yuhuai Wu, Roman Ring, Dani Yogatama, Dario Wünsch, Katrina McKinney, Oliver Smith, Tom Schaul, Timothy Lillicrap, Koray Kavukcuoglu, Demis Hassabis, Chris Apps, and David Silver. 2019. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature 575, 7782 (2019), 350--354.Google Scholar
- Michael P. Wellman. 2006. Methods for Empirical Game-Theoretic Analysis. In Proceedings of the 21st AAAI Conference. 1552--1556.Google Scholar
- Douglas Brent West. 2001. Introduction to graph theory. Vol. 2. Prentice hall Upper Saddle River.Google Scholar
- Michael Wooldridge and Nicholas R Jennings. 1994. Agent theories, architectures, and languages: a survey. In International Workshop on Agent Theories, Architectures, and Languages. Springer, 1--39.Google Scholar
- Geogios N Yannakakis. 2012. Game AI revisited. In Proceedings of the 9th conference on Computing Frontiers. 285--292.Google ScholarDigital Library
Index Terms
- WES: Agent-based User Interaction Simulation on Real Infrastructure
Index terms have been assigned to the content through auto-classification.
Recommendations
Appreciation to Referees
On behalf of the Editorial Board, I would like to thank the following people, who acted as Referees during the past year. Editor-in-Chief Adomavicius, Gediminas, New York University Fletcher, Roger, University of Dundee, Scotland Aggarwal, Charu C., ...
The MSOM Society Student Paper Competition: Abstracts of 2006 Winners
The journal is pleased to publish the abstracts of the six finalists of the 2006 Manufacturing and Service Operations Management Society's student paper competition.
The 2006 prize committee was chaired by Erica Plambeck (Stanford University). The other ...
Comments