Skip to main content
SpringerLink
Log in

Measuring the Quality of Assessment Using Questions Generated from the Semantic Web

  • Conference paper
  • First Online:
Artificial Intelligence in Education (AIED 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10947))

Included in the following conference series:

  • 5970 Accesses

Abstract

This article describes a new feature of the adaptive assessment system SIETTE that allows for the static and dynamic generation of questions from tables of data for knowledge assessment. Almost the same approach can be used to generate questions from data collected in a spreadsheet, a database query, or a semantic web query using SPARQL. The main problem faced with question generation is ensuring that the questions are valid for assessment. For this reason, most of the existing systems propose to use this mechanism only for low-stakes assessments. In this paper, we propose a methodology to control question generation quality and measure the impact of potential invalid instances on the final score as well as recommend some strategies to overcome these problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chaudhri, V.K., Clark, P.E., Overholtzer, A., Spaulding, A.: Question generation from a knowledge base. In: Janowicz, K., Schlobach, S., Lambrix, P., Hyvönen, E. (eds.) EKAW 2014. LNCS (LNAI), vol. 8876, pp. 54–65. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-13704-9_5

    Chapter  Google Scholar 

  2. Le, N.-T., Kojiri, T., Pinkwart, N.: Automatic question generation for educational applications – the state of art. In: van Do, T., Thi, H.A.L., Nguyen, N.T. (eds.) Advanced Computational Methods for Knowledge Engineering. AISC, vol. 282, pp. 325–338. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-06569-4_24

    Chapter  Google Scholar 

  3. Unger, C., Bühmann, L., Lehmann, J., Ngonga Ngomo, A.C., Gerber, D., Cimiano, P.: Template-based question answering over RDF data. In: Proceedings of the 21st International Conference on World Wide Web, WWW 2012, pp. 639. ACM Press, New York (2012)

    Google Scholar 

  4. Tamura, Y., Takase, Y., Hayashi, Y., Nakano, Y.I.: Generating quizzes for history learning based on wikipedia articles. In: Zaphiris, P., Ioannou, A. (eds.) LCT 2015. LNCS, vol. 9192, pp. 337–346. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20609-7_32

    Chapter  Google Scholar 

  5. Foulonneau, M., Ras, E.: Using educational domain models for automatic item generation beyond factual knowledge assessment. In: Hernández-Leo, D., Ley, T., Klamma, R., Harrer, A. (eds.) EC-TEL 2013. LNCS, vol. 8095, pp. 442–447. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40814-4_36

    Chapter  Google Scholar 

  6. Cablé, B., Guin, N., Lefevre, M.: An authoring tool for semi-automatic generation of self-assessment exercises. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 679–682. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39112-5_87

    Chapter  Google Scholar 

  7. Bühmann, L., Usbeck, R., Ngonga Ngomo, A.-C.: ASSESS — automatic self-assessment using linked data. In: Arenas, M., et al. (eds.) ISWC 2015. LNCS, vol. 9367, pp. 76–89. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-25010-6_5

    Chapter  Google Scholar 

  8. Vega-Gorgojo, G.: Clover Quiz: a trivia game powered by DBpedia. Semantic-Web-Journal.Net

    Google Scholar 

  9. Conejo, R., Guzmán, E., Millán, E., Trella, M., Pérez-de, J.L.: SIETTE : a web based tool for adaptive testing. Int. J. Artif. Intell. Educ. 14(1), 1–33 (2004)

    Google Scholar 

  10. Conejo, R., Guzmán, E., Trella, M.: The SIETTE automatic assessment environment. Int. J. Artif. Intell. Educ. 26(1), 270–292 (2016)

    Article  Google Scholar 

  11. Rios, A., Millán, E., Trella, M., Pérez-de-la Cruz, J.L., Conejo, R.: Internet based evaluation system. In: Artificial Intelligence in Education, vol. 64 pp. 1896–1898 (1999)

    Google Scholar 

  12. Bongir, A., Attar, V., Janardhanan, R.: Automated quiz generator. Adv. Intell. Syst. Comput. 683, 174–188 (2018)

    Article  Google Scholar 

  13. Alsubait, T., Parsia, B., Sattler, U.: Generating multiple choice questions from ontologies: how far can we go? In: Lambrix, P., et al. (eds.) EKAW 2014. LNCS (LNAI), vol. 8982, pp. 66–79. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17966-7_7

    Chapter  Google Scholar 

  14. Jouault, C., Seta, K.: Content-dependent question generation for history learning in semantic open learning space. In: Trausan-Matu, S., Boyer, K.E., Crosby, M., Panourgia, K. (eds.) ITS 2014. LNCS, vol. 8474, pp. 300–305. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07221-0_37

    Chapter  Google Scholar 

  15. Olney, A., Graesser, A., Person, N.: Question generation from concept maps. Dialogue Discourse 3(2), 75–99 (2012)

    Article  Google Scholar 

  16. Wainer, H., Thissen, D.: How is reliability related to the quality of test scores? what is the effect of local dependence on reliability? Educ. Measure. Issues Pract. 15(1), 22–29 (1996)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Malaga, Malaga, Spain

    Ricardo Conejo, Beatriz Barros & Manuel F. Bertoa

Authors
  1. Ricardo Conejo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Beatriz Barros
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manuel F. Bertoa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ricardo Conejo .

Editor information

Editors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, PA, USA

    Carolyn Penstein Rosé

  2. University of Technology, Sydney, NSW, Australia

    Roberto Martínez-Maldonado

  3. University of Duisburg-Essen, Duisburg, Germany

    H. Ulrich Hoppe

  4. UCL Institute of Education, London, UK

    Rose Luckin

  5. UCL Institute of Education, London, UK

    Manolis Mavrikis

  6. UCL Institute of Education, London, UK

    Kaska Porayska-Pomsta

  7. Carnegie Mellon University, Pittsburgh, PA, USA

    Bruce McLaren

  8. University of Sussex, Brighton, UK

    Benedict du Boulay

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Conejo, R., Barros, B., Bertoa, M.F. (2018). Measuring the Quality of Assessment Using Questions Generated from the Semantic Web. In: Penstein Rosé, C., et al. Artificial Intelligence in Education. AIED 2018. Lecture Notes in Computer Science(), vol 10947. Springer, Cham. https://doi.org/10.1007/978-3-319-93843-1_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-93843-1_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-93842-4

  • Online ISBN: 978-3-319-93843-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation