Skip to main content
SpringerLink
Log in

Item Ordering Biases in Educational Data

  • Conference paper
  • First Online:
Artificial Intelligence in Education (AIED 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11625))

Included in the following conference series:

Abstract

Data collected in a learning system are biased by order in which students solve items. This bias makes data analysis difficult and when not properly addressed, it may lead to misleading conclusions. We provide clear illustrations of the problem using simulated data and discuss methods for analyzing the scope of the problem in real data from a learning system. We present the data collection problem as a variant of the explore-exploit tradeoff and analyze several algorithms for addressing this tradeoff.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.umimeto.org/, available only in Czech.

  2. 2.

    https://en.robomise.cz/.

  3. 3.

    https://github.com/adaptive-learning/simulations-aied2019.

References

  1. Aleven, V., McLaughlin, E.A., Glenn, R.A., Koedinger, K.R.: Instruction based on adaptive learning technologies. In: Handbook of Research on Learning and Instruction, Routledge (2016)

    Google Scholar 

  2. Audibert, J.Y., Munos, R., Szepesvári, C.: Exploration-exploitation tradeoff using variance estimates in multi-armed bandits. Theoret. Comput. Sci. 410(19), 1876–1902 (2009)

    Article  MathSciNet  Google Scholar 

  3. Clement, B., Roy, D., Oudeyer, P.-Y., Lopes, M.: Multi-armed bandits for intelligent tutoring systems. J. Educ. Data Min. 7(2), 20–48 (2015)

    Google Scholar 

  4. González-Brenes, J., Huang, Y., Brusilovsky, P.: General features in knowledge tracing: applications to multiple subskills, temporal item response theory, and expert knowledge. In: Proceedings of Educational Data Mining, pp. 84–91 (2014)

    Google Scholar 

  5. Hofmann, H., Wickham, H., Kafadar, K.: Letter-value plots: boxplots for large data. J. Comput. Graph. Stat. 26(3), 469–477 (2017)

    Article  MathSciNet  Google Scholar 

  6. Jarušek, P., Pelánek, R.: Analysis of a simple model of problem solving times. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds.) ITS 2012. LNCS, vol. 7315, pp. 379–388. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-30950-2_49

    Chapter  Google Scholar 

  7. Käser, T., Koedinger, K.R., Gross, M.: Different parameters - same prediction: an analysis of learning curves. In: Proceedings of Educational Data Mining, pp. 52–59 (2014)

    Google Scholar 

  8. Lan, A.S., Baraniuk, R.G.: A contextual bandits framework for personalized learning action selection. In: Proceedings of Educational Data Mining, pp. 424–429 (2016)

    Google Scholar 

  9. Liu, Y.E., Mandel, T., Brunskill, E., Popovic, Z.: Trading off scientific knowledge and user learning with multi-armed bandits. In: Proceedings of Educational Data Mining, pp. 161–168 (2014)

    Google Scholar 

  10. Murray, R.C., et al.: Revealing the learning in learning curves. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 473–482. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39112-5_48

    Chapter  Google Scholar 

  11. Nixon, T., Fancsali, S., Ritter, S.: The complex dynamics of aggregate learning curves. In: Proceedings of Educational Data Mining (2013)

    Google Scholar 

  12. Papoušek, J., Stanislav, V., Pelánek, R.: Evaluation of an adaptive practice system for learning geography facts. In: Gasevic, D., Lynch, G., Dawson, S., Drachsler, H., Rosé, C.P. (eds.) Proceedings of Learning Analytics & Knowledge, pp. 40–47. ACM (2016)

    Google Scholar 

  13. Pavlik, P.I., Yudelson, M., Koedinger, K.R.: A measurement model of microgenetic transfer for improving instructional outcomes. Int. J. Artif. Intell. Educ. 25(3), 346–379 (2015)

    Article  Google Scholar 

  14. Pelánek, R.: Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques. User Model. User-Adap. Inter. 27(3), 313–350 (2017)

    Article  Google Scholar 

  15. Pelánek, R.: The details matter: methodological nuances in the evaluationof student models. User Model. User-Adap. Inter. (2018)

    Google Scholar 

  16. Pelánek, R., Jarušek, P.: Student modeling based on problem solving times. Int. J. Artif. Intell. Educ. 25(4), 493–519 (2015)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Masaryk University, Brno, Czech Republic

    Jaroslav Čechák & Radek Pelánek

Authors
  1. Jaroslav Čechák
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Radek Pelánek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jaroslav Čechák .

Editor information

Editors and Affiliations

  1. University of Sao Paulo, Sao Paulo, Brazil

    Seiji Isotani

  2. University of Malaga, Málaga, Spain

    Eva Millán

  3. Carnegie Mellon University, Pittsburgh, PA, USA

    Amy Ogan

  4. DePaul University, Chicago, IL, USA

    Peter Hastings

  5. Carnegie Mellon University, Pittsburgh, PA, USA

    Bruce McLaren

  6. University College London, London, UK

    Rose Luckin

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Čechák, J., Pelánek, R. (2019). Item Ordering Biases in Educational Data. In: Isotani, S., Millán, E., Ogan, A., Hastings, P., McLaren, B., Luckin, R. (eds) Artificial Intelligence in Education. AIED 2019. Lecture Notes in Computer Science(), vol 11625. Springer, Cham. https://doi.org/10.1007/978-3-030-23204-7_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-23204-7_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-23203-0

  • Online ISBN: 978-3-030-23204-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation