Abstract
The present work explores the effectiveness of goal setting in the context of adaptive training. Previous research has demonstrated that adaptive training approaches that tailor feedback and difficulty based on task performance lead to better learning outcomes than non-adaptive approaches. Likewise, decades of research on goal setting as an instructional technique has shown that setting achievement goals for trainees to improve also increases learning outcomes. In particular, challenging, specific goals have been found to be more effective than generic “try your best” goals. Bridging these techniques together presents an interesting opportunity to examine the effects of goal setting on performance in a training system that adapts both feedback and difficulty. For example, when a scenario’s difficulty is adapted up, a specific goal may no longer be attainable, which begs the question – do challenging goals improve performance compared to generic goals in a training system that adapts difficulty? In this experiment, 45 college students were trained to perform a complex radar detection task under two goal setting conditions, specific (“try to improve by 25%”) or general (“try your best”). We evaluated performance using a pre- to post-test design across several task measures. Overall, results were mixed, showing advantages for the specific goal condition on some accuracy measures but disadvantages on some timeliness measures compared to the general goal group. Implications for goal setting theory and practical applications for adaptive training are discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kelley, C.R.: What is adaptive training? Hum. Factors 11(6), 547–556 (1969)
Wickens, C.D., Hutchins, S., Carolan, T., Cumming, J.: Effectiveness of part-task training and increasing difficulty training strategies: a meta-analysis approach. Hum. Factors 55(2), 461–470 (2013)
Landsberg, C.R., Mercado, A.D., Van Buskirk, W.L., Lineberry, M., Steinhauser, N.: Evaluation of an adaptive training system for submarine periscope operations. Proc. Hum. Factors Ergon. Soc. 56(1), 2422–2426 (2012)
Marraffino, M.D., Johnson, C.I., Whitmer, D.E., Steinhauser, N.B., Clement, A.: Advise when ready for game plan: adaptive training for JTACs. In: Proceedings of the Interservice/Industry Training, Simulation and Education Conference (2019)
Marraffino, M.D., Schroeder, B.L., Fraulini, N.W., Van Buskirk, W.L., Johnson, C.I.: Adapting training in real time: an empirical test of adaptive training frequency algorithms. Military Psychology (in press)
Van Buskirk, W.L., Fraulini, N.W., Schroeder, B.L., Johnson, C.I., Marraffino, M.D.: Application of theory to the development of an adaptive training system for a submarine electronic warfare task. In: Sottilare, R.A., Schwarz, J. (eds.) HCII 2019. LNCS, vol. 11597, pp. 352–362. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22341-0_28
Locke, E.A., Latham, G.P.: A Theory of Goal Setting & Task Performance. Prentice-Hall, Inc., Hoboken (1990)
Boyce, B.A., Wayda, V.K.: The effects of assigned and self-set goals on task performance. J. Sport Exerc. Psychol. 16(3), 258–269 (1994). https://doi.org/10.1123/jsep.16.3.258
Elston, T.L., Ginis, K.A.M.: The effects of self-set versus assigned goals on exercisers’ self-efficacy for an unfamiliar task. J. Sport Exerc. Psychol. 26(3), 500–504 (2004). https://doi.org/10.1123/jsep.26.3.500
Locke, E.A., Bryan, J.F.: The effects of goal setting, rule-learning, and knowledge of score on performance. Am. J. Psychol. 79(3), 451–457 (1966)
Locke, E.A., Latham, G.P.: New directions in goal setting theory. Curr. Dir. Psychol. Sci. 15(5), 265–268 (2006)
Bouffard-Bouchard, T.: Influence of self-efficacy on performance in a cognitive task. J. Soc. Psycho. 130(3), 353–363 (1990)
Wood, R.E., Mento, A.J., Locke, E.A.: Task complexity as a moderator of goal effects: a meta-analysis. J. Appl. Psychol. 72(3), 416–425 (1987)
Locke, E.A.: Toward a theory of task motivation and incentives. Organ. Behav. Hum. Perform. 3(2), 157–189 (1968)
Latham, G.P., Baldes, J.J.: The practical significance of Locke’s theory of goal setting. J. Appl. Psychol. 60(1), 122–124 (1975)
Kluger, A.N., DeNisi, A.: The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol. Bull. 119(2), 254–284 (1996)
Locke, E.A.: Motivation by goal setting. In: Handbook of Organizational Behavior, pp. 43–56 (2000)
Kimmel, H.D.: Three criteria for the use of one-tailed tests. Psychol. Bull. 54(4), 351–353 (1957)
Field, A.: Discovering Statistics Using IBM SPSS Statistics, 5th edn. SAGE Publications, London (2018)
Neubert, M.J.: The value of feedback and goal setting over goal setting alone and potential moderators of this effect: a meta-analysis. Hum. Perform. 11(4), 321–335 (1998)
Astwood, R.S., Van Buskirk, W.L., Cornejo, J., Dalton, J.: The impact of different feedback types on decision-making in simulation based training environments. In: Proceedings of the 52nd Annual Meeting of the Human Factors and Ergonomics Society, vol 52, pp. 2062–2066. Santa Monica, CA: Human Factors and Ergonomics Society (2008)
Buff, W.L., Campbell, G.E.: What to do or what not to do?: Identifying the content of effective feedback. In: Proceedings of the 46th Annual Meeting of the Human Factors and Ergonomics Society, vol. 46, pp. 2074–2078. Santa Monica, CA: Human Factors and Ergonomics Society (2002)
Cohen, V.B.: A reexamination of feedback in computer-based instruction: Implications for instructional design. Educ. Technol. 25(1), 33–37 (1985)
Acknowledgments
We gratefully acknowledge Dr. Kip Krebs and the Office of Naval Research who sponsored this work (Funding Doc# N0001420WX00540). We also wish to thank Robert Veira, Jason Hochreiter, Marc Prince, and Jacob May for their significant contributions during testbed development, and Matthew Pierce and Jacob Entinger for their assistance with data collection. Presentation of this material does not constitute or imply its endorsement, recommendation, or favoring by the U.S. Navy or Department of Defense (DoD). The opinions of the authors expressed herein do not necessarily state or reflect those of the U.S. Navy or DoD. NAWCTSD Public Release 21-ORL007 Distribution Statement A – Approved for public release; distribution is unlimited.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply
About this paper
Cite this paper
Schroeder, B.L., Fraulini, N.W., Van Buskirk, W.L., Johnson, C.I., Marraffino, M.D. (2021). Setting Goals in Adaptive Training: Can Learners Improve with a Moving Target?. In: Sottilare, R.A., Schwarz, J. (eds) Adaptive Instructional Systems. Design and Evaluation. HCII 2021. Lecture Notes in Computer Science(), vol 12792. Springer, Cham. https://doi.org/10.1007/978-3-030-77857-6_30
Download citation
DOI: https://doi.org/10.1007/978-3-030-77857-6_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77856-9
Online ISBN: 978-3-030-77857-6
eBook Packages: Computer ScienceComputer Science (R0)