research-article

Open access

ParliRobo: Participant Lightweight AI Robots for Massively Multiplayer Online Games (MMOGs)

Authors:

Xudong WuAuthors Info & Claims

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Pages 9093 - 9102

https://doi.org/10.1145/3581783.3613764

Published: 27 October 2023 Publication History

Abstract

Recent years have witnessed the profound influence of AI technologies on computer gaming. While grandmaster-level AI robots have largely come true for complex games based on heavy back-end support, in practice many game developers crave for participant AI robots (PARs) that behave like average-level humans with inexpensive infrastructures. Unfortunately, to date there has not been a satisfactory solution that registers large-scale use. In this work, we attempt to develop practical PARs (dubbed ParliRobo) showing acceptably humanoid behaviors with well affordable infrastructures under a challenging scenario-a 3D-FPS (first-person shooter) mobile MMOG with real-time interaction requirements. Based on comprehensive real-world explorations, we eventually enable our attempt through a novel ?transform and polish" methodology. It achieves ultralight implementations of the core system components by non-intuitive yet principled approaches, and meanwhile carefully fixes the probable side effect incurred on user perceptions. Evaluation results from large-scale deployment indicate the close resemblance (96% on average) in biofidelity metrics between ParliRobo and human players; moreover, in 73% mini Turing tests ParliRobo cannot be distinguished from human players.

References

[1]

Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath. 2017. Deep Reinforcement Learning: A Brief Survey. IEEE Signal Processing Magazine, Vol. 34, 6 (2017), 26--38.

[2]

Christopher Berner, Greg Brockman, Brooke Chan, et al. 2019. Dota 2 with Large Scale Deep Reinforcement Learning. arxiv: 1912.06680

[3]

Blizzard Entertainment, Inc. 2022. StarCraft II Official Game Site. https://starcraft2.com/.

[4]

Thierry Blu, Philippe Thévenaz, and Michael Unser. 2004. Linear Interpolation Revitalized. IEEE Transactions on Image Processing, Vol. 13, 5 (2004), 710--719.

Digital Library

[5]

Noam Brown, Adam Lerer, Sam Gross, and Tuomas Sandholm. 2019. Deep Counterfactual Regret Minimization. In Proc. of PMLR ICML. 793--802.

[6]

Ralph E Carlson and Frederick N Fritsch. 1985. Monotone Piecewise Bicubic Interpolation. SIAM J. Numer. Anal., Vol. 22, 2 (1985), 386--400.

Digital Library

[7]

Guillaume Chaslot, Sander Bakkes, Istvan Szita, and Pieter Spronck. 2008. Monte-Carlo Tree Search: A New Framework for Game AI. In Proc. of AAAI, Vol. 4. 216--217.

[8]

Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, and Zhangyang Wang. 2021. The Elastic Lottery Ticket Hypothesis. In Proc. of NeurIPS, Vol. 34. 26609--26621.

[9]

Michele Colledanchise, Ramviyas Parasuraman, and Petter Ögren. 2018. Learning of Behavior Trees for Autonomous Agents. IEEE Transactions on Games, Vol. 11, 2 (2018), 183--189.

[10]

Jonathan Frankle and Michael Carbin. 2019. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. In Proc. of ICLR.

[11]

Bent Fuglede and Flemming Topsoe. 2004. Jensen-Shannon Divergence and Hilbert Space Embedding. In Proc. of IEEE ISIT. 31.

[12]

Gibney, Elizabeth. 2016. Google AI Algorithm Masters Ancient Game of Go. Nature News, Vol. 529, 7587 (2016), 445.

[13]

Google. 2021. Official Website of Google Protocol Buffers. https://developers.google.com/protocol-buffers.

[14]

David Grelaud, Nicolas Bonneel, Michael Wimmer, et al. 2009. Efficient and Practical Audio-Visual Rendering for Games Using Crossmodal Perception. In Proc. of ACM I3D. 177--182.

Digital Library

[15]

Dianyuan Han. 2013. Comparison of Commonly Used Image Interpolation Methods. In Proc. of ICCSEE. 1556--1559.

[16]

Johannes Heinrich and David Silver. 2016. Deep Reinforcement Learning from Self-Play in Imperfect-Information Games. arXiv preprint arXiv:1603.01121 (2016).

[17]

Henry Ewins. 2020. Like Animals, Video Game AI Is Stupidly Intelligent. https://www.eurogamer.net/articles/2020-01-09-like-animals-video-game-ai-is-stupidly-intelligent.

[18]

Philip Hingston. 2009. A Turing Test for Computer Game Bots. IEEE Transactions on Computational Intelligence and AI in Games, Vol. 1, 3 (2009), 169--186.

[19]

Philip Hingston. 2010. A New Design for a Turing Test for Bots. In Proc. of IEEE CIG. 345--350.

[20]

Sean D Holcomb, William K Porter, Shaun V Ault, et al. 2018. Overview on Deepmind and Its Alphago Zero AI. In Proc. of ACM ICBDE. 67--71.

Digital Library

[21]

Shiyu Huang, Hang Su, Jun Zhu, and Ting Chen. 2019. Combo-Action: Training Agent for FPS Game with Auxiliary Tasks. In Proc. of AAAI, Vol. 33. 954--961.

Digital Library

[22]

Matteo Iovino, Edvards Scukins, Jonathan Styrud, Petter Ögren, and Christian Smith. 2020. A Survey of Behavior Trees in Robotics and AI. arXiv preprint arXiv:2005.05842 (2020).

[23]

Matteo Iovino, Jonathan Styrud, Pietro Falco, and Christian Smith. 2021. Learning Behavior Trees with Genetic Programming in Unpredictable Environments. In Proc. of IEEE ICRA. 4591--4597.

Digital Library

[24]

Aditya Jain, Ramta Bansal, Avnish Kumar, and KD Singh. 2015. A Comparative Study of Visual and Auditory Reaction Times on the Basis of Gender and Physical Activity Levels of Medical First Year Students. International Journal of Applied and Basic Medical Research, Vol. 5, 2 (2015), 124.

[25]

Aaron Khoo and Robert Zubek. 2002. Applying Inexpensive AI Techniques to Computer Games. IEEE Intelligent Systems, Vol. 17, 4 (2002), 48--53.

Digital Library

[26]

KRAFTON, Inc. 2022. PUBG Mobile. https://www.pubgmobile.com/.

[27]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet Classification with Deep Convolutional Neural Networks. In Proc. of NeurIPS, Vol. 25.

[28]

Shaofan Lai, Wei-Shi Zheng, Jian-Fang Hu, and Jianguo Zhang. 2017. Global-Local Temporal Saliency Action Prediction. IEEE Transactions on Image Processing, Vol. 27, 5 (2017), 2272--2285.

[29]

Guillaume Lample and Devendra Singh Chaplot. 2017. Playing FPS Games with Deep Reinforcement Learning. In Proc.of AAAI.

[30]

Junjie Li, Sotetsu Koyamada, Qiwei Ye, et al. 2020. Suphx: Mastering Mahjong with Deep Reinforcement Learning. arXiv preprint arXiv:2003.13590 (2020).

[31]

Zhenhua Li, Yafei Dai, Guihai Chen, and Yunhao Liu. 2023. Content Distribution for Mobile Internet: A Cloud-Based Approach, Second Edition. Springer Nature Press.

[32]

Chiu-Chou Lin, Wei-Chen Chiu, and I-Chen Wu. 2021. An Unsupervised Video Game Playstyle Metric via State Discretization. In Proc. of PMLR UAI. 215--224.

[33]

Tianyu Liu, Zijie Zheng, Hongchang Li, et al. 2019. Playing Card-Based RTS Games with Deep Reinforcement Learning. In Proc. of IJCAI. 4540--4546.

[34]

Andres Marzal and Enrique Vidal. 1993. Computation of Normalized Edit Distance and Applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 15, 9 (1993), 926--932.

Digital Library

[35]

Maja J Matarić. 2019. Human-Machine and Human-Robot Interaction for Long-Term User Engagement and Behavior Change. In Proc. of ACM MobiCom. 1--2.

Digital Library

[36]

Michael Matuschek. 2022. Using Adaptive AI to Improve the Gaming Experience. https://www.mouser.com/blog/using-adaptive-ai-improve-gaming-experience.

[37]

MIPAV. 2020. Transform: Conformal Mapping Algorithms. https://mipav.cit.nih.gov/pubwiki/index.php/Transform:_Conformal_Mapping_Algorithms.

[38]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, et al. 2013. Playing Atari with Deep Reinforcement Learning. arXiv preprint arXiv:1312.5602 (2013).

[39]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, et al. 2015. Human-Level Control through Deep Reinforcement Learning. Nature, Vol. 518, 7540 (2015), 529--533.

[40]

Moonton. 2022. Mobile Legends: Bang Bang. https://m.mobilelegends.com/.

[41]

Ghulam Muhammad, Yousef A Alotaibi, Mansour Alsulaiman, and Mohammad Nurul Huda. 2010. Environment Recognition Using Selected MPEG-7 Audio Features and Mel-Frequency Cepstral Coefficients. In Proc. of IEEE ICDT. 11--16.

Digital Library

[42]

Yury Nahshan, Brian Chmiel, Chaim Baskin, et al. 2021. Loss Aware Post-training Quantization. Machine Learning, Vol. 110, 11 (2021), 3245--3262.

Digital Library

[43]

Zeev Nehari. 2012. Conformal Mapping. Courier Corporation.

[44]

Joseph C Osborn and Michael Mateas. 2014. A Game-Independent Play Trace Dissimilarity Metric. In Proc. of FDG.

[45]

Mark Owen Riedl and Alexander Zook. 2013. AI for Game Production. In Proc. of IEEE CIG. 1--8.

[46]

John Schulman, Filip Wolski, Prafulla Dhariwal, et al. 2017. Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347 (2017).

[47]

Kun Shao, Zhentao Tang, Yuanheng Zhu, et al. 2019. A Survey of Deep Reinforcement Learning in Video Games. arXiv preprint arXiv:1912.10944 (2019).

[48]

David Silver, Thomas Hubert, Julian Schrittwieser, et al. 2018. A General Reinforcement Learning Algorithm That Masters Chess, Shogi, and Go through Self-Play. Science, Vol. 362, 6419 (2018), 1140--1144.

[49]

Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-scale Image Recognition. arXiv preprint arXiv:1409.1556 (2014).

[50]

Jost Tobias Springenberg, Alexey Dosovitskiy, et al. 2015. Striving for Simplicity: The All Convolutional Net. In Proc. of ICLR.

[51]

TheExpressWire. 2023. 2023-2029 Massive Multiplayer Online (MMO) Games Market Size Detailed Report with Sales and Revenue Analysis | Research by Absolute Reports. https://www.digitaljournal.com/pr/news/2023-2029-massive-multiplayer-online-mmo-games-market-size-detailed-report-with-sales-and-revenue-analysis-research-by-absolute-reports.

[52]

Julian Togelius, Sergey Karakovskiy, Jan Koutník, and Jurgen Schmidhuber. 2009. Super Mario Evolution. In Proc. of IEEE CIG. 156--161.

[53]

Alan M Turing. 2012. Computing Machinery and Intelligence (1950). The Essential Turing: the Ideas That Gave Birth to the Computer Age (2012), 433--464.

[54]

Valve Corporation. 2022. Dota2 Official Game Site. https://www.dota2.com/home.

[55]

Michael Van Lent, John Laird, Josh Buckman, et al. 1999. Intelligent Agents in Computer Games. In Proc. of AAAI. 929--930.

[56]

Vinyals, Oriol and Babuschkin, Igor and Chung, Junyoung and others. 2019. Alphastar: Mastering the Real-Time Strategy Game Starcraft II. DeepMind Blog (2019), 2.

[57]

Muhammad Abdul Wahab. 2017. Interpolation and Extrapolation. In Proc. Topics Syst. Eng. Winter Term, Vol. 17. 1--6.

[58]

Wikipedia. 2022. Kullback Leibler divergence. https://en.wikipedia.org/wiki/Kullback-Leibler_divergence.

[59]

Deheng Ye, Guibin Chen, Wen Zhang, et al. 2020a. Towards Playing Full MOBA Games with Deep Reinforcement Learning. In Proc. of NeurIPS, Vol. 33. 621--632.

[60]

Deheng Ye, Zhao Liu, Mingfei Sun, et al. 2020b. Mastering Complex Control in MOBA Games with Deep Reinforcement Learning. In Proc. of AAAI, Vol. 34. 6672--6679.

[61]

Sule Yildirim and Sindre Berg Stene. 2010. A Survey on the Need and Use of AI in Game Agents. InTech.

[62]

Matthew D Zeiler and Rob Fergus. 2014. Visualizing and Understanding Convolutional Networks. In Proc. of Springer ECCV. 818--833.

[63]

Dongbin Zhao, Zhen Zhang, and Yujie Dai. 2012. Self-Teaching Adaptive Dynamic Programming for Gomoku. Neurocomputing, Vol. 78, 1 (2012), 23--29.

Digital Library

[64]

Yiren Zhou, Seyed-Mohsen Moosavi-Dezfooli, Ngai-Man Cheung, and Pascal Frossard. 2018. Adaptive Quantization for Deep Neural Network. In Proc. of AAAI.

[65]

Martin Zinkevich, Michael Johanson, Michael Bowling, and Carmelo Piccione. 2007. Regret Minimization in Games with Incomplete Information. In Proc. of NeurIPS, Vol. 20.

Cited By

Wang YWu WLiu JJiang YHuang TMao JPeng S(2024)Memory-Effect Based QoE Evaluation Method and Guarantee Scheme in APN-Driven Game AccelerationIEEE Transactions on Network Science and Engineering10.1109/TNSE.2024.341739311:5(4774-4792)Online publication date: Sep-2024
https://doi.org/10.1109/TNSE.2024.3417393

Index Terms

ParliRobo: Participant Lightweight AI Robots for Massively Multiplayer Online Games (MMOGs)
1. Information systems
  1. Information systems applications
    1. Multimedia information systems
      1. Massively multiplayer online games

Recommendations

The usability of massively multiplayer online roleplaying games: designing for new users
CHI '04: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

This study examines the usability challenges faced by new players of massively multiplayer online role-playing games (MMORPGs), one of the fastest-growing segments of the video game industry. Played in completely online worlds, these games allow players ...
Massively multiplayer online role-playing games: the past, present, and future
Theoretical and Practical Computer Applications in Entertainment

Massively multiplayer online role-playing games (MMORPGs) are emerging in the computer game industry as a very popular genre. These games have existed since the late 1990s, but in the last few years the market has become increasingly strong. This ...
Interactivity in massively multiplayer online games: a concept explication
ACE '10: Proceedings of the 7th International Conference on Advances in Computer Entertainment Technology

Interactivity in game studies is a widely-discussed but under-conceptualized concept. Most studies on game interactivity focus on social interaction between players in virtual spaces of Massively Multiplayer Online Games (MMOGs), while the other ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

October 2023

9913 pages

ISBN:9798400701085

DOI:10.1145/3581783

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada & MBZUAI, UAE
,
Tao Mei
HiDream.ai, China
,
Rita Cucchiara
University of Modena and Reggio Emilia, Italy
,
Program Chairs:
Marco Bertini
University of Florence, Italy
,
Diana Patricia Tobon Vallejo
Unversidad de Medellin, Colombia
,
Pradeep K. Atrey
University at Albany, State University of New York, USA
,
M. Shamim Hossain
M. Shamim Hossain (King Saud University, KSA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

MM '23

Sponsor:

SIGMM

MM '23: The 31st ACM International Conference on Multimedia

October 29 - November 3, 2023

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
306
Total Downloads

Downloads (Last 12 months)247
Downloads (Last 6 weeks)31

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang YWu WLiu JJiang YHuang TMao JPeng S(2024)Memory-Effect Based QoE Evaluation Method and Guarantee Scheme in APN-Driven Game AccelerationIEEE Transactions on Network Science and Engineering10.1109/TNSE.2024.341739311:5(4774-4792)Online publication date: Sep-2024
https://doi.org/10.1109/TNSE.2024.3417393

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten