Yongsheng Liu
ABSTRACT
Video gaming has gained huge popularity over the last few decades. As reported, there are about 2.9 billion gamers globally. Among all genres, competitive games are one of the most popular ones.
Matchmaking is a core problem for competitive games, which determines the player satisfaction, hence influences the game success. Most matchmaking systems group the queuing players into opposing teams with similar skill levels. The key challenge is to accurately rate the players' skills based on their match performances. There has been an increasing amount of effort on developing such rating systems such as Elo, Glicko.
However, games with different game-plays might have different game modes, which might require an extensive amount of effort for rating system customization. Even though there are many rating system choices and various customization strategies, there is a clear lack of a systematic framework with which different rating systems can be analysed and compared against each other. Such a framework could help game developers to identify the bottlenecks of their matchmaking systems and enhance the performance of their matchmaking systems.
To bridge the gap, we present MMBench, the first benchmark framework for evaluating different rating systems. It serves as a fair means of comparison for different rating systems and enables a deeper understanding of different rating systems. In this paper, we will present how MMBench could benchmark the three major rating systems, Elo, Glicko, Trueskill in the battle modes of 1 vs 1, n vs n, battle royal and teamed battle royal over both real and synthetic datasets.
- Craig Chapple. 2020. PUBG Mobile Tops List of Billion-Dollar Mobile Games in 2020. (2020). https://sensortower.com/blog/pubg-mobile-tops-billion-dollarmobile-games-2020Google Scholar
- Shuo Chen and Thorsten Joachims. 2016. Predicting matchups and preferences in context. (2016), 775--784.Google Scholar
- Arman Dehpanah, Muheeb Faizan Ghori, Jonathan Gemmell, and Bamshad Mobasher. 2021. The evaluation of rating systems in online free-for-all games. (2021), 131--151.Google Scholar
- Olivier Delalleau, Emile Contal, Eric Thibodeau-Laufer, Raul Chandias Ferrari, Yoshua Bengio, and Frank Zhang. 2012. Beyond skill rating: Advanced matchmaking in ghost recon online. IEEE Transactions on Computational Intelligence and AI in Games 4, 3 (2012), 167--177.Google ScholarCross Ref
- Arpad E Elo. 1978. The rating of chessplayers, past and present. BT Batsford Limited.Google Scholar
- Mark E Glickman. 1995. The glicko system. Boston University 16 (1995), 16--17.Google Scholar
- Mark E Glickman. 2012. Example of the Glicko-2 system. Boston University (2012), 1--6.Google Scholar
- Ralf Herbrich, Tom Minka, and Thore Graepel. 2006. TrueSkill?: a Bayesian skill rating system. Advances in neural information processing systems 19 (2006).Google Scholar
- Pei Li. 2021. PUBG Mobile reports 1 billion accumulated downloads since 2018 launch. (2021). https://www.reuters.com/article/us-tencent-videogamesidUSKBN2BH0D1Google Scholar
- Yao Li, Minhao Cheng, Kevin Fujii, Fushing Hsieh, and Cho-Jui Hsieh. 2018. Learning from group comparisons: exploiting higher order interactions. Advances in Neural Information Processing Systems 31 (2018).Google Scholar
- Joshua E Menke and Tony R Martinez. 2008. A Bradley--Terry artificial neural network model for individual ratings in group competitions. Neural computing and Applications 17, 2 (2008), 175--186.Google Scholar
- Tom Minka, Ryan Cleven, and Yordan Zaykov. 2018. Trueskill 2: An improved bayesian skill rating system. Technical Report (2018).Google Scholar
- Newzoo. 2021. Newzoo Global Games Market Report 2021. (2021). https://newzoo.com/insights/articles/global-games-market-to-generate-175--8-billion-in-2021-despite-a-slight-decline-the-market-is-on-track-tosurpass-200-billion-in-2023/Google Scholar
- Lei Zhang, Jun Wu, Zhong-Cun Wang, and Chong-Jun Wang. 2010. A factorbased model for context-sensitive skill rating systems. 2 (2010), 249--255.Google Scholar
Index Terms
- MMBench: The Match Making Benchmark
Recommendations
Playing with Streakiness in Online Games: How Players Perceive and React to Winning and Losing Streaks in League of Legends
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsStreakiness refers to observed tendency towards consecutive appearances of particular patterns. In video games, streakiness is oftentimes inevitable, where a player keeps winning or losing for a short period. However, the phenomenon remains understudied ...
Using Machine Learning to Predict Game Outcomes Based on Player-Champion Experience in League of Legends
FDG '21: Proceedings of the 16th International Conference on the Foundations of Digital GamesLeague of Legends (LoL) is the most widely played multiplayer online battle arena (MOBA) game in the world. An important aspect of LoL is competitive ranked play, which utilizes a skill-based matchmaking system to form fair teams. However, players’ ...
Application context based algorithm for player skill evaluation in MOBA games
NetGames '15: Proceedings of the 2015 International Workshop on Network and Systems Support for GamesThis work is motivated by the challenge to improve player rating systems in Multiplayer Online Battle Arena (MOBA) games by incorporating the application context, as opposed to calculating the player rating based solely on the match outcome (winning or ...
Comments