Abstract
StepMania is a popular open-source clone of a rhythm-based video game. As is common in popular games, there is a large number of community-designed levels. It is often difficult for players and level authors to determine the difficulty level of such community contributions. In this work, we formalize and analyze the difficulty prediction task on StepMania levels as an ordinal regression (OR) task. We standardize a more extensive and diverse selection of this data resulting in five data sets, two of which are extensions of previous work. We evaluate many competitive OR and non-OR models, demonstrating that neural network-based models significantly outperform the state of the art and that StepMania-level data makes for an excellent test bed for deep OR models. We conclude with a user experiment showing our models’ superhuman performance.
B. J. Franks and B. Dinkelmann—Equally contributions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Agarwal, A., Davis, J.T., Ward, T.: Supporting ordinal four-state classification decisions using neural networks. Inf. Technol. Manage. 2(1), 5–26 (2001)
Armstrong, B.G., Sloan, M.: Ordinal regression models for epidemiologic data. Am. J. Epidemiol. 129(1), 191–204 (1989)
Brown, T., et al.: Language models are few-shot learners. Adv. Neural. Inf. Process. Syst. 33, 1877–1901 (2020)
Bürkner, P.C., Vuorre, M.: Ordinal regression models in psychology: a tutorial. Adv. Methods Pract. Psychol. Sci. 2(1), 77–101 (2019)
Caronongan, A.P., Marcos, N.A.: Predicting chart difficulty in rhythm games through classification using chart pattern derived attributes. In: Alfred, R., Iida, H., Haviluddin, H., Anthony, P. (eds.) Computational Science and Technology. LNEE, vol. 724, pp. 193–205. Springer, Singapore (2021). https://doi.org/10.1007/978-981-33-4069-5_17
Cheng, J., Wang, Z., Pollastri, G.: A neural network approach to ordinal regression. In: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pp. 1279–1284. IEEE (2008)
Chiu, S.C., Chen, M.S.: A study on difficulty level recognition of piano sheet music. In: 2012 IEEE International Symposium on Multimedia, pp. 17–23. IEEE (2012)
Diaz, R., Marathe, A.: Soft labels for ordinal regression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4738–4747 (2019)
Donahue, C., Lipton, Z.C., McAuley, J.: Dance dance convolution. In: International Conference on Machine Learning, pp. 1039–1048. PMLR (2017)
Fu, H., Gong, M., Wang, C., Batmanghelich, K., Tao, D.: Deep ordinal regression network for monocular depth estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2002–2011 (2018)
Ghatas, Y., Fayek, M., Hadhoud, M.: A hybrid deep learning approach for musical difficulty estimation of piano symbolic music. Alex. Eng. J. 61(12), 10183–10196 (2022)
Gutiérrez, P.A., Perez-Ortiz, M., Sanchez-Monedero, J., Fernandez-Navarro, F., Hervas-Martinez, C.: Ordinal regression methods: survey and experimental study. IEEE Trans. Knowl. Data Eng. 28(1), 127–146 (2015)
Halina, E., Guzdial, M.: Taikonation: Patterning-focused chart generation for rhythm action games. In: The 16th International Conference on the Foundations of Digital Games (FDG) 2021, pp. 1–10 (2021)
Hooshyar, D., Yousefi, M., Lim, H.: Data-driven approaches to game player modeling: a systematic literature review. ACM Comput. Surv. (CSUR) 50(6), 1–19 (2018)
Jian, H., van den Brink, H.M., Groot, W.: College Education and Social trust, Ph.D. thesis, Maastricht University (1727)
Lin, H.T., Li, L.: Reduction from cost-sensitive ordinal ranking to weighted binary classification. Neural Comput. 24(5), 1329–1367 (2012)
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (2019)
Missura, O., Gärtner, T.: Player modeling for intelligent difficulty adjustment. In: Gama, J., Costa, V.S., Jorge, A.M., Brazdil, P.B. (eds.) DS 2009. LNCS (LNAI), vol. 5808, pp. 197–211. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04747-3_17
Niu, Z., Zhou, M., Wang, L., Gao, X., Hua, G.: Ordinal regression with multiple output CNN for age estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4920–4928 (2016)
Roohi, S., Relas, A., Takatalo, J., Heiskanen, H., Hämäläinen, P.: Predicting game difficulty and churn without players. In: Proceedings of the Annual Symposium on Computer-Human Interaction in Play, pp. 585–593 (2020)
Saad, S.E., Yang, J.: Twitter sentiment analysis based on ordinal regression. IEEE Access 7, 163677–163685 (2019)
Sébastien, V., Ralambondrainy, H., Sébastien, O., Conruyt, N.: Score analyzer: automatically determining scores difficulty level for instrumental e-learning. In: 13th International Society for Music Information Retrieval Conference (ISMIR 2012), pp. 571–576 (2012)
Tsujino, Y., Yamanishi, R.: Dance dance gradation: a generation of fine-tuned dance charts. In: Clua, E., Roque, L., Lugmayr, A., Tuomi, P. (eds.) ICEC 2018. LNCS, vol. 11112, pp. 175–187. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99426-0_15
Tsujino, Y., Yamanishi, R., Yamashita, Y.: Characteristics study of dance-charts on rhythm-based video games. In: 2019 IEEE Conference on Games (CoG), pp. 1–4. IEEE (2019)
Tutz, G.: Ordinal regression: a review and a taxonomy of models. Wiley Interdisc. Rev. Comput. Stat. 14(2), e1545 (2022)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Yannakakis, G.N., Spronck, P., Loiacono, D., André, E.: Player Modeling. In: Lucas, S.M., Mateas, M., Preuss, M., Spronck, P., Togelius, J. (eds.) Artificial and Computational Intelligence in Games, Dagstuhl Follow-Ups, vol. 6, pp. 45–59. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany (2013). https://doi.org/10.4230/DFU.Vol6.12191.45
Acknowledgements
The authors acknowledge support by the Carl-Zeiss Foundation, the DFG awards KL 2698/2-1, KL 2698/5-1, BU 4042/2-1 and BU 4042/1-1, and the BMBF awards 01|S20048, 01|S18051A, 03|B0770E, and 01|S21010C.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Ethical Implications
This work focuses on applications in the entertainment industry. Specifically, the focus is improving the entertainment achieved when playing StepMania, a video game involving physical and mental exertion. The prediction of difficulty is closely related to reducing the churn in games, that is, the rate at which players stop playing a game for various reasons. While not the focus of this work, this does mean there is a chance that this work can be used to make rhythm-based video games more addictive.
Beyond the user experiment performed in Table 4, this work does not involve the use of personal data. For Table 4, we merely collected for pairs of songs played by a particular player which of the two was more difficult. This data is entirely anonymous, and for each rated pair, it is no longer possible to determine which player was the participant rating the pair.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Franks, B.J., Dinkelmann, B., Kloft, M., Fellenz, S. (2023). Ordinal Regression for Difficulty Prediction of StepMania Levels. In: De Francisci Morales, G., Perlich, C., Ruchansky, N., Kourtellis, N., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Applied Data Science and Demo Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14174. Springer, Cham. https://doi.org/10.1007/978-3-031-43427-3_30
Download citation
DOI: https://doi.org/10.1007/978-3-031-43427-3_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43426-6
Online ISBN: 978-3-031-43427-3
eBook Packages: Computer ScienceComputer Science (R0)
