Jianwei Zheng
Changnan Xiao
Zhenhua Li
ABSTRACT
Recent years have witnessed the profound influence of AI technologies on computer gaming. While grandmaster-level AI robots have largely come true for complex games based on heavy back-end support, in practice many game developers crave for participant AI robots (PARs) that behave like average-level humans with inexpensive infrastructures. Unfortunately, to date there has not been a satisfactory solution that registers large-scale use. In this work, we attempt to develop practical PARs (dubbed ParliRobo) showing acceptably humanoid behaviors with well affordable infrastructures under a challenging scenario-a 3D-FPS (first-person shooter) mobile MMOG with real-time interaction requirements. Based on comprehensive real-world explorations, we eventually enable our attempt through a novel ?transform and polish" methodology. It achieves ultralight implementations of the core system components by non-intuitive yet principled approaches, and meanwhile carefully fixes the probable side effect incurred on user perceptions. Evaluation results from large-scale deployment indicate the close resemblance (96% on average) in biofidelity metrics between ParliRobo and human players; moreover, in 73% mini Turing tests ParliRobo cannot be distinguished from human players.
- Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath. 2017. Deep Reinforcement Learning: A Brief Survey. IEEE Signal Processing Magazine, Vol. 34, 6 (2017), 26--38.Google Scholar
Cross Ref
- Christopher Berner, Greg Brockman, Brooke Chan, et al. 2019. Dota 2 with Large Scale Deep Reinforcement Learning. arxiv: 1912.06680Google Scholar
- Blizzard Entertainment, Inc. 2022. StarCraft II Official Game Site. https://starcraft2.com/.Google Scholar
- Thierry Blu, Philippe Thévenaz, and Michael Unser. 2004. Linear Interpolation Revitalized. IEEE Transactions on Image Processing, Vol. 13, 5 (2004), 710--719.Google ScholarDigital Library
- Noam Brown, Adam Lerer, Sam Gross, and Tuomas Sandholm. 2019. Deep Counterfactual Regret Minimization. In Proc. of PMLR ICML. 793--802.Google Scholar
- Ralph E Carlson and Frederick N Fritsch. 1985. Monotone Piecewise Bicubic Interpolation. SIAM J. Numer. Anal., Vol. 22, 2 (1985), 386--400.Google ScholarDigital Library
- Guillaume Chaslot, Sander Bakkes, Istvan Szita, and Pieter Spronck. 2008. Monte-Carlo Tree Search: A New Framework for Game AI. In Proc. of AAAI, Vol. 4. 216--217.Google Scholar
- Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, and Zhangyang Wang. 2021. The Elastic Lottery Ticket Hypothesis. In Proc. of NeurIPS, Vol. 34. 26609--26621.Google Scholar
- Michele Colledanchise, Ramviyas Parasuraman, and Petter Ögren. 2018. Learning of Behavior Trees for Autonomous Agents. IEEE Transactions on Games, Vol. 11, 2 (2018), 183--189.Google ScholarCross Ref
- Jonathan Frankle and Michael Carbin. 2019. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. In Proc. of ICLR.Google Scholar
- Bent Fuglede and Flemming Topsoe. 2004. Jensen-Shannon Divergence and Hilbert Space Embedding. In Proc. of IEEE ISIT. 31.Google ScholarCross Ref
- Gibney, Elizabeth. 2016. Google AI Algorithm Masters Ancient Game of Go. Nature News, Vol. 529, 7587 (2016), 445.Google Scholar
- Google. 2021. Official Website of Google Protocol Buffers. https://developers.google.com/protocol-buffers.Google Scholar
- David Grelaud, Nicolas Bonneel, Michael Wimmer, et al. 2009. Efficient and Practical Audio-Visual Rendering for Games Using Crossmodal Perception. In Proc. of ACM I3D. 177--182.Google ScholarDigital Library
- Dianyuan Han. 2013. Comparison of Commonly Used Image Interpolation Methods. In Proc. of ICCSEE. 1556--1559.Google ScholarCross Ref
- Johannes Heinrich and David Silver. 2016. Deep Reinforcement Learning from Self-Play in Imperfect-Information Games. arXiv preprint arXiv:1603.01121 (2016).Google Scholar
- Henry Ewins. 2020. Like Animals, Video Game AI Is Stupidly Intelligent. https://www.eurogamer.net/articles/2020-01-09-like-animals-video-game-ai-is-stupidly-intelligent.Google Scholar
- Philip Hingston. 2009. A Turing Test for Computer Game Bots. IEEE Transactions on Computational Intelligence and AI in Games, Vol. 1, 3 (2009), 169--186.Google ScholarCross Ref
- Philip Hingston. 2010. A New Design for a Turing Test for Bots. In Proc. of IEEE CIG. 345--350.Google ScholarCross Ref
- Sean D Holcomb, William K Porter, Shaun V Ault, et al. 2018. Overview on Deepmind and Its Alphago Zero AI. In Proc. of ACM ICBDE. 67--71.Google ScholarDigital Library
- Shiyu Huang, Hang Su, Jun Zhu, and Ting Chen. 2019. Combo-Action: Training Agent for FPS Game with Auxiliary Tasks. In Proc. of AAAI, Vol. 33. 954--961.Google ScholarDigital Library
- Matteo Iovino, Edvards Scukins, Jonathan Styrud, Petter Ögren, and Christian Smith. 2020. A Survey of Behavior Trees in Robotics and AI. arXiv preprint arXiv:2005.05842 (2020).Google Scholar
- Matteo Iovino, Jonathan Styrud, Pietro Falco, and Christian Smith. 2021. Learning Behavior Trees with Genetic Programming in Unpredictable Environments. In Proc. of IEEE ICRA. 4591--4597.Google ScholarDigital Library
- Aditya Jain, Ramta Bansal, Avnish Kumar, and KD Singh. 2015. A Comparative Study of Visual and Auditory Reaction Times on the Basis of Gender and Physical Activity Levels of Medical First Year Students. International Journal of Applied and Basic Medical Research, Vol. 5, 2 (2015), 124.Google ScholarCross Ref
- Aaron Khoo and Robert Zubek. 2002. Applying Inexpensive AI Techniques to Computer Games. IEEE Intelligent Systems, Vol. 17, 4 (2002), 48--53.Google ScholarDigital Library
- KRAFTON, Inc. 2022. PUBG Mobile. https://www.pubgmobile.com/.Google Scholar
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet Classification with Deep Convolutional Neural Networks. In Proc. of NeurIPS, Vol. 25.Google Scholar
- Shaofan Lai, Wei-Shi Zheng, Jian-Fang Hu, and Jianguo Zhang. 2017. Global-Local Temporal Saliency Action Prediction. IEEE Transactions on Image Processing, Vol. 27, 5 (2017), 2272--2285.Google ScholarCross Ref
- Guillaume Lample and Devendra Singh Chaplot. 2017. Playing FPS Games with Deep Reinforcement Learning. In Proc.of AAAI.Google ScholarCross Ref
- Junjie Li, Sotetsu Koyamada, Qiwei Ye, et al. 2020. Suphx: Mastering Mahjong with Deep Reinforcement Learning. arXiv preprint arXiv:2003.13590 (2020).Google Scholar
- Zhenhua Li, Yafei Dai, Guihai Chen, and Yunhao Liu. 2023. Content Distribution for Mobile Internet: A Cloud-Based Approach, Second Edition. Springer Nature Press.Google Scholar
- Chiu-Chou Lin, Wei-Chen Chiu, and I-Chen Wu. 2021. An Unsupervised Video Game Playstyle Metric via State Discretization. In Proc. of PMLR UAI. 215--224.Google Scholar
- Tianyu Liu, Zijie Zheng, Hongchang Li, et al. 2019. Playing Card-Based RTS Games with Deep Reinforcement Learning. In Proc. of IJCAI. 4540--4546.Google ScholarCross Ref
- Andres Marzal and Enrique Vidal. 1993. Computation of Normalized Edit Distance and Applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 15, 9 (1993), 926--932.Google ScholarDigital Library
- Maja J Matarić. 2019. Human-Machine and Human-Robot Interaction for Long-Term User Engagement and Behavior Change. In Proc. of ACM MobiCom. 1--2.Google ScholarDigital Library
- Michael Matuschek. 2022. Using Adaptive AI to Improve the Gaming Experience. https://www.mouser.com/blog/using-adaptive-ai-improve-gaming-experience.Google Scholar
- MIPAV. 2020. Transform: Conformal Mapping Algorithms. https://mipav.cit.nih.gov/pubwiki/index.php/Transform:_Conformal_Mapping_Algorithms.Google Scholar
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, et al. 2013. Playing Atari with Deep Reinforcement Learning. arXiv preprint arXiv:1312.5602 (2013).Google Scholar
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, et al. 2015. Human-Level Control through Deep Reinforcement Learning. Nature, Vol. 518, 7540 (2015), 529--533.Google Scholar
- Moonton. 2022. Mobile Legends: Bang Bang. https://m.mobilelegends.com/.Google Scholar
- Ghulam Muhammad, Yousef A Alotaibi, Mansour Alsulaiman, and Mohammad Nurul Huda. 2010. Environment Recognition Using Selected MPEG-7 Audio Features and Mel-Frequency Cepstral Coefficients. In Proc. of IEEE ICDT. 11--16.Google ScholarDigital Library
- Yury Nahshan, Brian Chmiel, Chaim Baskin, et al. 2021. Loss Aware Post-training Quantization. Machine Learning, Vol. 110, 11 (2021), 3245--3262.Google ScholarDigital Library
- Zeev Nehari. 2012. Conformal Mapping. Courier Corporation.Google Scholar
- Joseph C Osborn and Michael Mateas. 2014. A Game-Independent Play Trace Dissimilarity Metric. In Proc. of FDG.Google Scholar
- Mark Owen Riedl and Alexander Zook. 2013. AI for Game Production. In Proc. of IEEE CIG. 1--8.Google ScholarCross Ref
- John Schulman, Filip Wolski, Prafulla Dhariwal, et al. 2017. Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347 (2017).Google Scholar
- Kun Shao, Zhentao Tang, Yuanheng Zhu, et al. 2019. A Survey of Deep Reinforcement Learning in Video Games. arXiv preprint arXiv:1912.10944 (2019).Google Scholar
- David Silver, Thomas Hubert, Julian Schrittwieser, et al. 2018. A General Reinforcement Learning Algorithm That Masters Chess, Shogi, and Go through Self-Play. Science, Vol. 362, 6419 (2018), 1140--1144.Google Scholar
- Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-scale Image Recognition. arXiv preprint arXiv:1409.1556 (2014).Google Scholar
- Jost Tobias Springenberg, Alexey Dosovitskiy, et al. 2015. Striving for Simplicity: The All Convolutional Net. In Proc. of ICLR.Google Scholar
- TheExpressWire. 2023. 2023-2029 Massive Multiplayer Online (MMO) Games Market Size Detailed Report with Sales and Revenue Analysis | Research by Absolute Reports. https://www.digitaljournal.com/pr/news/2023-2029-massive-multiplayer-online-mmo-games-market-size-detailed-report-with-sales-and-revenue-analysis-research-by-absolute-reports.Google Scholar
- Julian Togelius, Sergey Karakovskiy, Jan Koutník, and Jurgen Schmidhuber. 2009. Super Mario Evolution. In Proc. of IEEE CIG. 156--161.Google ScholarCross Ref
- Alan M Turing. 2012. Computing Machinery and Intelligence (1950). The Essential Turing: the Ideas That Gave Birth to the Computer Age (2012), 433--464.Google ScholarCross Ref
- Valve Corporation. 2022. Dota2 Official Game Site. https://www.dota2.com/home.Google Scholar
- Michael Van Lent, John Laird, Josh Buckman, et al. 1999. Intelligent Agents in Computer Games. In Proc. of AAAI. 929--930.Google Scholar
- Vinyals, Oriol and Babuschkin, Igor and Chung, Junyoung and others. 2019. Alphastar: Mastering the Real-Time Strategy Game Starcraft II. DeepMind Blog (2019), 2.Google Scholar
- Muhammad Abdul Wahab. 2017. Interpolation and Extrapolation. In Proc. Topics Syst. Eng. Winter Term, Vol. 17. 1--6.Google Scholar
- Wikipedia. 2022. Kullback Leibler divergence. https://en.wikipedia.org/wiki/Kullback-Leibler_divergence.Google Scholar
- Deheng Ye, Guibin Chen, Wen Zhang, et al. 2020a. Towards Playing Full MOBA Games with Deep Reinforcement Learning. In Proc. of NeurIPS, Vol. 33. 621--632.Google Scholar
- Deheng Ye, Zhao Liu, Mingfei Sun, et al. 2020b. Mastering Complex Control in MOBA Games with Deep Reinforcement Learning. In Proc. of AAAI, Vol. 34. 6672--6679.Google ScholarCross Ref
- Sule Yildirim and Sindre Berg Stene. 2010. A Survey on the Need and Use of AI in Game Agents. InTech.Google Scholar
- Matthew D Zeiler and Rob Fergus. 2014. Visualizing and Understanding Convolutional Networks. In Proc. of Springer ECCV. 818--833.Google ScholarCross Ref
- Dongbin Zhao, Zhen Zhang, and Yujie Dai. 2012. Self-Teaching Adaptive Dynamic Programming for Gomoku. Neurocomputing, Vol. 78, 1 (2012), 23--29.Google ScholarDigital Library
- Yiren Zhou, Seyed-Mohsen Moosavi-Dezfooli, Ngai-Man Cheung, and Pascal Frossard. 2018. Adaptive Quantization for Deep Neural Network. In Proc. of AAAI.Google ScholarCross Ref
- Martin Zinkevich, Michael Johanson, Michael Bowling, and Carmelo Piccione. 2007. Regret Minimization in Games with Incomplete Information. In Proc. of NeurIPS, Vol. 20.Google Scholar
Index Terms
- ParliRobo: Participant Lightweight AI Robots for Massively Multiplayer Online Games (MMOGs)
Recommendations
The usability of massively multiplayer online roleplaying games: designing for new users
CHI '04: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsThis study examines the usability challenges faced by new players of massively multiplayer online role-playing games (MMORPGs), one of the fastest-growing segments of the video game industry. Played in completely online worlds, these games allow players ...
Massively multiplayer online role-playing games: the past, present, and future
Theoretical and Practical Computer Applications in EntertainmentMassively multiplayer online role-playing games (MMORPGs) are emerging in the computer game industry as a very popular genre. These games have existed since the late 1990s, but in the last few years the market has become increasingly strong. This ...
Interactivity in massively multiplayer online games: a concept explication
ACE '10: Proceedings of the 7th International Conference on Advances in Computer Entertainment TechnologyInteractivity in game studies is a widely-discussed but under-conceptualized concept. Most studies on game interactivity focus on social interaction between players in virtual spaces of Massively Multiplayer Online Games (MMOGs), while the other ...
Comments