Abstract
This paper describes efforts related to the design, development, and evaluation of a game-based assessment called Awkward Annie. The Awkward Annie game focuses on English language pragmatic skills and represents a novel design that asks players to violate social norms by intentionally selecting the most inappropriate things to say to virtual colleagues. Awkward Annie was evaluated through two studies. Study 1 was a small-scale lab-based study with predominantly non-native English speakers (n = 36). Study 2 was a scaled up extension of the same design but with native English speakers on Amazon Mechanical Turk (n = 328). Both studies explored potential relations between user experience, understanding the play mechanics (i.e., being inappropriate), game performance, and English pragmatic ability. The studies also indicate cultural differences in players comfort with acting inappropriately. The results of these studies benefit both cross-cultural game design and the assessment of socio-pragmatics and pragmalinguistics. Additionally, this research advances the current state of assessment practices by investigating how games could potentially provide an alternative to traditional selected response testing.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Belz, J.A.: The role of computer mediation in the instruction and development of L2 pragmatic ability. Ann. Rev. Appl. Linguist. 27, 45 (2007)
Clyne, M.: Inter-Cultural Communication at Work: Cultural Values in Discourse. Cambridge University Press, Cambridge, UK (1994)
Cole, H., Griffiths, M.D.: Social interactions in massively multi-player online role-playing gamers. Cyber Psychol. Behav. 10(4), 575–583 (2007)
Cornillie, F., Thorne, S.L., Desmet, P.: ReCALL special issue: digital games for language learning: challenges and opportunities. ReCALL 24(03), 243–256 (2012)
Garris, R., Ahlers, R., Driskell, J.E.: Games, motivation, and learning: a research and practice model. Simul. Gaming 33(4), 441–467 (2002)
Gee, J.P.: What video games have to teach us about learning and literacy. Comput. Entertain. 1(1), 20 (2003)
Hoffman, E., John, M., Makany, T.: How do game design frameworks align with learning and assessment design frameworks? Paper presented at the annual meeting for the National Council for Measurement in Education, Philadelphia, PA (2014)
Ishihara, N., Cohen, A.D.: Teaching and Learning Pragmatics: Where Language and Culture Meet. Pearson, Harlow (2010)
Jackson, G.T., McNamara, D.S.: Motivation and performance in a game-based intelligent tutoring system. J. Educ. Psychol. 105, 1036–1049 (2013)
Leech, G.N.: Principles of Pragmatics. Longman, London, England (1983)
Limberg, H.: Principles for pragmatics teaching: apologies in the EFL classroom. ELT J. 69(3), 275–285 (2015)
McDonald, K.K., Hannafin, R.D.: Using web-based computer games to meet the demands of today’s high-stakes testing: a mixed method inquiry. J. Res. Technol. Educ. 35(4), 459–472 (2003)
Pivec, M., Dziabenko, O., Schinnerl, I.: Game-based learning in universities and lifelong learning: “UniGame: social skills and knowledge training” game concept. J. Univ. Comput. Sci. 10(1), 14–26 (2004)
Riddiford, N., Joe, A.: Tracking the development of sociopragmatic skills. TESOL Q. 44(1), 195–205 (2010)
Surface, E.A., Dierdorff, E.C., Watson, A.M.: Special operations language training software measurement of effectiveness study: Tactical Iraqi study final report. Special Operations Forces Language Office, Tampa, FL, USA (2007)
Taguchi, N.: Longitudinal study of higher-order inferential ability in L2 English. In: Byrnes, H., Ortega, L. (eds.) The Longitudinal Study of Advanced L2 Capacities. Lawrence Erlbaum, Mahwah (2008)
Timpe-Laughlin, V., Wain, J., Schmidgall, J.: Defining and operationalizing the construct of pragmatic ability: Review and recommendations. ETS Research Report Series. Princeton, NJ. Educational Testing Service (2015)
Timpe, V.: Assessing Intercultural Language Learning. Peter Lang, Frankfurt aM (2013a)
Timpe, V.: The difficulty with difficulty: the issue of determining task difficulty in TBLA. J. Linguist. Lang. Teach. 4(1), 13–28 (2013)
Yee, N.: Motivations for play in online games. Cyber Psychol. Behav. 9(6), 772–775 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Jackson, G.T., Grace, L., Inglese, P., Wain, J., Hone, R. (2018). Awkward Annie: Game-Based Assessment of English Pragmatic Skills. In: Cheok, A., Inami, M., Romão, T. (eds) Advances in Computer Entertainment Technology. ACE 2017. Lecture Notes in Computer Science(), vol 10714. Springer, Cham. https://doi.org/10.1007/978-3-319-76270-8_54
Download citation
DOI: https://doi.org/10.1007/978-3-319-76270-8_54
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-76269-2
Online ISBN: 978-3-319-76270-8
eBook Packages: Computer ScienceComputer Science (R0)