skip to main content
10.1145/503376.503400acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
Article

Adaptive testing: effects on user performance

Published:20 April 2002Publication History

ABSTRACT

This study examines the effects of interface adaptation on user performance in HCI and CMC. No studies to date have explored the psychological effects of a combination of software performance monitoring and adaptation. This combination is the focus of the present study. Two competing possible effects of adaptive interfaces are presented: 1) Social facilitation, according to which users with high task confidence should perform better, and users with low task confidence should perform less well because their performance is monitored by the interface; and 2) "choking", according to which users with high task confidence should perform less well, and users with low task confidence should perform better because the interface adapts to their performance. A 2 (adaptive vs. non-adaptive) x 2 (high user task confidence vs. low task confidence) x 2 (HCI vs. CMC) laboratory experiment was conducted. Results indicate that for CMC, the social facilitation explanation holds true, while results for HCI were consistent with the "choking" explanation. Implications for the theory and design of adaptive interfaces are discussed

References

  1. Aiello, J. & Kolb, K. Electronic performance monitoring and social context. Journal of Applied Psychology, 7 (1993), 537--548.Google ScholarGoogle Scholar
  2. Aiello, J. & Svec, C. Computer monitoring of work performance: Extending the social facilitation framework to electronic presence. Journal of Applied Social Psychology, 23 (1993), 537--548.Google ScholarGoogle ScholarCross RefCross Ref
  3. Barnard, J.J. Using Iitem Response Theory for Test Construction. Human Sciences Research Council, Pretoria, South Africa, 1990.Google ScholarGoogle Scholar
  4. Cooper, A. The Inmates are Running the Asylum: Why High-Tech Products Drive us Crazy and How to Restore the Sanity. SAMS Macmillan Computer Publishing, Indianapolis, IN, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. De Ayala, R.J., Dodd, B.G., Koch, W.R. A simulation and comparison of flexilevel and Bayesian computerized adaptive testing. Journal of Educational Measurement 27 (1990), 227--229.Google ScholarGoogle ScholarCross RefCross Ref
  6. De Beer, M. & Visser, D. Comparability of paper and pencil and computerized versions of the GSAT. South African Journal of Psychology 1 (1998), 37--42.Google ScholarGoogle Scholar
  7. Etzioni O. & Weld, D. A softbot-based interface to the Internet, in: Huhns, M.N., & Singh, M.P. (eds.) Readings in Agents. Morgan Kauffman Publishers, San Francisco, CA, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Geen, R.G. Evaluation apprehension and social facilitation: A reply to Sanders. Journal of Experimental Social Psychology 17 (1981), 252--256.Google ScholarGoogle ScholarCross RefCross Ref
  9. Graydon, J., et al. The effects of personality on social facilitation whilst performing a sports related task. Journal of Personal and Individual Differences 19, 2 (1995), 265--267.Google ScholarGoogle ScholarCross RefCross Ref
  10. Griffith, T.L. Monitoring and performance: A comparison of computer and supervisor monitoring. Journal of Applied Social Psychology 23 (1993), 549--572.Google ScholarGoogle ScholarCross RefCross Ref
  11. Guerin, B. Mere presence effects in humans: A review. Journal of Experimental and Social Psychology 22 (1986), 38--77.Google ScholarGoogle ScholarCross RefCross Ref
  12. Guerin, B, & Innes, J. Social facilitation and social monitoring: A new look at Zajonc's mere presence hypothesis. British Journal of Social Psychology 21 (1982), 7--18.Google ScholarGoogle ScholarCross RefCross Ref
  13. Harkins, S. Social loafing and social facilitation. Journal of Experimental Social Psychology 23 (1987), 1--18.Google ScholarGoogle ScholarCross RefCross Ref
  14. Henly, S., Klebe, K.J., McBride, J.R. & Cudeck, R. Adaptive and conventional versions of the DAT: the first complete test battery comparison. Applied Psychological Measurement 13 (1989), 363--371.Google ScholarGoogle ScholarCross RefCross Ref
  15. Hindus, D., Ackerman, M.S., Mainwaring, S. & Starr, B. Thunderwire: A field study of an audio-only media space, in Proceedings of CSCW '96 (November 1996), ACM Press, 238--247. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Huhns, M.N., & Singh, M.P. (eds.). Readings in Agents. Morgan Kauffman Publishers, San Francisco, CA, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Isbister, K., & Layton, T. Agents: What (or who) are they?, in Nielsen, J. (ed.) Advances in Human-Computer Interaction (Norwood, N.J, 1995), Ablex Publishing Corporation, 67--86. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Langley, P. User modeling in adaptive interfaces, in Proceedings of the Seventh International Conference on User Modeling (Banff, Alberta, 1997), Springer, 357--370. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Lashkari, Y. et al. Collaborative interface agents. In: Huhns, M.N., & Singh, M.P. (eds.). Readings in Agents. Morgan Kauffman Publishers, San Francisco, CA, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Martin, S. & Knight, J.M. Social facilitation effects resulting from locus of control using human and computer experimenters. Computers in Human Behavior 1 (1986), 123--130.Google ScholarGoogle ScholarCross RefCross Ref
  21. Minsky, M. A conversation with Marvin Minsky. Communications of the ACM 7 (1994), 23--29. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Picard, R. Does HAL cry digital tears: Emotions and computers, in Stork, D.G. (ed.) HAL's Legacy. MIT Press, Cambridge, MA, 1997.Google ScholarGoogle Scholar
  23. Reeves, B. and Nass, C. The Media Equation. How People treat Computers, Television, and New Media like Real People and Places. Cambridge University Press, New York, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Rich, C. & Sidner, C. Collagen: When agents collaborate with people. In: Huhns, M.N., & Singh, M.P. (eds.). Readings in Agents. Morgan Kauffman Publishers, San Francisco, CA, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Rocklin, T.R., et al. Effects and underlying mechanisms of self-adapted testing. Journal of Educational Psychology 87, 1 (1995), 103--116.Google ScholarGoogle ScholarCross RefCross Ref
  26. Roos, L., Wise, S.L., & Plake, B. The role of item feedback in self adapted testing. Educational and Psychological Measurement 57, 1 (1997), 85--98.Google ScholarGoogle ScholarCross RefCross Ref
  27. Sanders, G. Driven by distraction: An integrative review of social facilitation theory and research. Journal of Experimental Social Psychology 17 (1981), 227--251.Google ScholarGoogle ScholarCross RefCross Ref
  28. Selker, T. Coach: A teaching agent that learns. Communications of the ACM 7 (1994) 2--12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Smith, M.J., et al. Employee stress and health complaints in jobs with and without electronic performance monitoring. Applied Ergonomics 23 (1992), 17--27.Google ScholarGoogle ScholarCross RefCross Ref
  30. Stanton, J. & Barnes-Farrell, J. Effects of electronic performance monitoring. Journal of Applied Psychology 6 (1996), 738--745.Google ScholarGoogle ScholarCross RefCross Ref
  31. Trumbly, J.E., Arnett, K.P., & Johnson, P.C. Productivity gains via an adaptive user interface. Journal of Human-Computer Studies 40 (1994), 63--81. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Westin, A. Two key factors that belong in a macroergonomic analysis of electronic monitoring. Applied Ergonomics 23 (1992), 35--42.Google ScholarGoogle ScholarCross RefCross Ref
  33. Winograd, T. Profile: Microsoft Bob, in: Winograd, T. (ed.). Bringing Design to Software. ACM Press, New York, 1996. Google ScholarGoogle Scholar
  34. Zajonc, R. Social facilitation. Science 149 (1965), 269--275.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Adaptive testing: effects on user performance

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in
        • Published in

          cover image ACM Conferences
          CHI '02: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
          April 2002
          478 pages
          ISBN:1581134533
          DOI:10.1145/503376
          • Conference Chair:
          • Dennis Wixon

          Copyright © 2002 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 20 April 2002

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • Article

          Acceptance Rates

          CHI '02 Paper Acceptance Rate61of414submissions,15%Overall Acceptance Rate6,199of26,314submissions,24%

          Upcoming Conference

          CHI '24
          CHI Conference on Human Factors in Computing Systems
          May 11 - 16, 2024
          Honolulu , HI , USA

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader