Skip to main content
SpringerLink
Log in

Metrics-based evaluation of learning object reusability

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

This paper aims to help in the selection of reusable educational materials from repositories on the web, developing an indicator of the reusability of learning objects. For this purpose, our research will be carried out in three stages. The first, based on previous studies in this area, will determine those aspects that influence reusability. The second will define a set of metrics that measure those aspects using metadata. The third will propose different methods of aggregation in order to obtain a single resulting value and evaluate the efficiency of the model by analyzing a significant set of learning objects obtained from the eLera and Merlot repositories. The results obtained suggest that the proposed indicator could provide useful information when searching for learning objects in repositories. This reusability measurement could constitute an indicator of quality, which would allow search results to be ordered, with those with the greatest possibility of being reused taking priority. Furthermore, the proposed reusability indicator could be calculated automatically or in an assisted way if metadata elements satisfy the minimum quality requisites identified.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Barnard, J. (1998). A new reusability metric for object-oriented software. Software Quality Journal, 7, 35–50.

    Article  Google Scholar 

  • Barzilai, J. (1997). Deriving weights from pairwise comparison matrices. The Journal of the Operational Research Society, 48(12), 1226–1232.

    MATH  Google Scholar 

  • Benneker, F. (2006). A quick scan on possibilities for automatic metadata generation. Technical report, Utrecht, The Netherlands: Stichting Digitale Universiteit. http://hdl.handle.net/1820/802. Accessed 14 May 2010.

  • Boyle, T. (2003). Design principles for authoring dynamic, reusable learning objects. Australian Journal of Educational Technology, 19(1), 46–58.

    Google Scholar 

  • Cafolla, R. (2006). Project Merlot: Bringing Peer Review to Web-based Educational Resources. Journal of Technology and Teacher Education, 14(2), 313–323.

    Google Scholar 

  • Campbell, L. (2003). Engaging with the learning object economy. In A. Littlejohn (Ed.), Reusing online resources: A sustainable approach to e-learning (pp. 35–45). London: Kogan Page.

    Google Scholar 

  • Cervera, J. F., Lopez, M. G., Fernandez, C., & Sanchez-Alonso, S. (2009). Quality metrics in learning objects. In F. Sartori, M. A. Sicilia, & N. Manouselis (Eds.), Metadata and semantics research (pp. 135–141). Berlin, Heidelberg: Springer.

    Chapter  Google Scholar 

  • Chang, K. (2006). Learning objects: Draft quality criteria and quality assurance approach for learnalberta.ca and the society of advancement of excellence in education, futured consulting education futurists. Inc. Technical report, http://www.futured.com/QualityStandardsforLearningObjects.pdf.pdf. Accessed 14 May 2010.

  • Conte, S. D., Dunsmore, H. E., & Shen, V. Y. (1989). Software engineering metrics and models. San Francisco: Benjamin Cummings.

    Google Scholar 

  • Cuadrado-Gallego, J. J., & Sicilia, M. A. (2005). Learning objects reusability metrics: Some ideas from software engineering, In Proceedings of the international conference on internet technologies and applications. Wreham, UK: North East Wales Institute.

  • Currier, S., & Campbell, L. (2002). Evaluating learning resources for reusability: The dner and learning objects study. In Proceeding of the Australasian society for computers in learning in tertiary education, Auckland, New Zeland.

  • Daniel, B., & Mohan, P. (2004). A model for evaluating learning objects. In Proceedings of the IEEE international conference on advanced learning technologies (pp. 50–60).

  • Downes, S. (2001). Learning objects: Resources for distance education worldwide. International Review of Research in Open and Distance Learning, 2(1).

  • Duval, E. (2004). Learning technology standardization: Making sense of it all. International Journal on Computer Science and Information Systems, 1(1), 33–43.

    Article  Google Scholar 

  • Duval, E., Warkentyne, K., Haenni, F., Forte, E., Cardinaels, K., Verhoeven, B., et al. (2001). The ariadne knowledge pool system. Communications of the ACM, 44(5), 72–78.

    Article  Google Scholar 

  • Etzkorn, L. H., Hughes, W. E., & Davis, C. G. (2001). Automated reusability quality analysis. Information and Software Technology, 43(5), 295–308.

    Article  Google Scholar 

  • Fenton, N. E., & Pfleeger, S. L. (1997). Software metrics (3rd ed.). Boston, Massachusetts: International Thompson Publishing.

    Google Scholar 

  • Friesen, N. (2009). Open educational resources: New possibilities for change and sustainability. International Review of Research in Open and Distance Learning, 10(5).

  • Garcia, E., & Sicilia, M. A. (2009). Preliminary explorations on the statistical profiles of highly-rated learning objects. In F. Sartori, M. A. Sicilia, & N. Manouselis (Eds.), Metadata and semantics research (pp. 108–117). Berlin, Heidelberg: Springer.

    Chapter  Google Scholar 

  • Glass, R. L. (1995). A structure-based critique of contemporary computing research. Journal of Systems and Software, 28(1), 3–7.

    Article  Google Scholar 

  • Huddlestone, J., & Pike, J. (2005). Learning object reuse—A four tier model. In the IEE and MOD HFI DTC symposium on people and systems—who are we designing for, London.

  • IEEE. (1993). Std. 610.12–1990. Standard glossary of software engineering terminology. Los Alamitos,CA, USA: IEEE Computer Society Press.

    Google Scholar 

  • IEEE learning technology standards committee. (2002). Standard for learning object metadata, http://ltsc.ieee.org/wg12. Accessed 1 October 2009.

  • Kay, R., & Knaack, L. (2007). Evaluating the learning in learning objects, open learning. The Journal of Open and Distance Learning, 22(1), 5–28.

    Article  Google Scholar 

  • Kelty, C. M., Burrus, C. S., & Baraniuk, R. G. (2008). Peer review anew: Three principles and a case study in postpublication quality assurance. Proceedings of the IEEE Special Issue on Educational Technology, 96(6), 1000–1011.

    Google Scholar 

  • Koper, R. (2003). Combining reusable learning resources and services to pedagogical purposeful units of learning. In A. Littlejohn (Ed.), Reusing online resources: A sustainable approach to elearning (pp. 46–59). London: Kogan Page.

    Google Scholar 

  • Kumar, V., Nesbit, J. C., & Han, K. (2005). Rating learning object quality with distributed bayesian belief networks: The why and the how. In Fifth IEEE international conference on advanced learning technologies (pp. 685–687).

  • Margaritopoulos, T., Margaritopoulos, M., Mavridis, I., & Manitsaris, A. (2009). A fine grained metric system for the completeness of metadata. In Proceedings of third international conference metadata and semantic research, MTSR (pp. 83–94). Milan, Italy.

  • Marichal, J. L. (2000). An axiomatic approach of the discrete choquet integral as a tool to aggregate interacting criteria. IEEE Transactions on Fuzzy Systems, 8(6), 800–807.

    Article  MathSciNet  Google Scholar 

  • Massart, D. (2009). Adopting standards and specifications for educational content. Technical report, European Commission’s eContentplus programme, http://aspect-project.org/.

  • Nesbit, J. C., & Belfer, K. (2004). Collaborative evaluation of learning objects. In R. McGreal (Ed.), Online education using learning objects. London: Routledge/Falmer.

    Google Scholar 

  • Nesbit, J. C., Li, J., & Leacock, T. (2006). Web-based tools for collaborative evaluation of learning resources. Journal on Systemics, Cybernetics and Informatics, 3(5), 102–112.

    Google Scholar 

  • Ochoa, X., & Duval, E. (2007). Relevance ranking metrics for learning objects. In Proceedings of the second European conference on technology enhanced learning (Vol. 4753, pp. 262–276). Springer.

  • Ochoa, X., & Duval, E. (2008). Measuring learning object reuse, In Proceedings of the 3rd European conference on technology enhanced learning: Times of convergence, Springer-Verlag, Vol. 5192, Lecture Notes in Computer Science, pp. 322–325.

  • Ochoa, X., & Duval, E. (2009). Quantitative analysis of learning object repositories. IEEE Transactions on Learning Technologies, 2(3), 226–238.

    Article  Google Scholar 

  • Omosule, S., Shoniregun, C., & Preston, D. (2008).A framework for culture influence virtual learning environments trust. Third international conference on digital information management, ICDIM (pp. 411–416). London, UK.

  • Palmer, K., & Richardson, P. (2004). Learning object reusability—motivation, production and use. In 11th international conference of the association for learning technology. Devon, England:University of Exeter.

  • Pitkanen, S. H., & Silander, P. (2004). Criteria for pedagogical reusability of learning objects enabling adaptation and individualised learning processes. In Proceedings of the IEEE international conference on advanced learning technologies (pp. 246–250). Joensuu, Finland.

  • Polsani, P. R. (2003). Use and abuse of reusable learning objects. Journal of Digital Information, 3(4).

  • Poulin, J. (1996). Measuring software reuse: Principles, practices, and economic models. Boston, Massachusetts: Addison-Wesley Longman Publishing.

    Google Scholar 

  • Riaz, M., Mendes, E., & Tempero, E. D. (2009). A systematic review of software maintainability prediction and metrics. In Proceedings of the third international symposium on empirical software engineering and measurement, ESEM (pp. 367–377). Florida, USA.

  • Richards, G. (2007). Writing to be read: Readability indices for open educational resources. In First international workshop on learning object discovery and exchange. http://fire.eun.org/lode2007/lode12.pdf. Accessed 14 May 2010.

  • Rodríguez-Ardura, I., Jiménez-Zarco, A. I., Ammetller-Montes, G., & Pacheco-Berna, M. C. (2009). Improving hypermedia teaching resources–new designs for e-learning environments. International Journal of Technology Enhanced Learning, 1(4), 286–296.

    Article  Google Scholar 

  • Sicilia, M. A. (2004). Reusability and reuse of learning objects, myths, realities and possibilities. In Proceedings of the first pluri-disciplinary symposium on design, evaluation and description of reusable learning contents.

  • Sicilia, M.A., & Garcia, E. (2003). On the concepts of usability and reusability of learning objects. International Review of Research in Open and Distance Learning, 4(2).

  • Sosteric, M., & Hesemeier, S. (2002). When is a learning object not an object. The International Review of Research in Open and Distance Learning, 3(2).

  • Tabachnick, B. G., & Fidell, L. S. (1996). Using multivariate statistics (3rd ed.). New York: HarperCollins.

    Google Scholar 

  • Tzikopoulos, A., Manouselis, N., & Vuorikari, R. (2007). An overview of learning object repositories, in idea group publishing. Learning Objects for Instruction: Design and Evaluation, pp. 44–64.

  • Van Assche, F., Ayre, J., Baumgartner, P., Duval, E., Hartinger, S., Mesdom, F., et al. (2009). Melt final report. Technical report, eContentplus, http://info.melt-project.eu/shared/data/melt/MELT_1_3_Final_Project_Report.pdf. Accessed 14 May 2010.

  • Vargo, J., Nesbit J. C., Belfer, K., & Archambault, A. (2003). Learning object evaluation: Computer-mediated collaboration and inter-rater reliability. International Journal of Computers and Applications, 25(3).

  • Vinoski, S. (2005). Old measures for new services. IEEE Internet Computing, 9(6), 72–74.

    Article  Google Scholar 

  • Wiley, D. A. (2002). Connecting learning objects to instructional design theory: A definition, a metaphor and a taxonomy. In D. A. Wiley (Ed.), The instructional use of learning objects (pp. 3–24). Bloomington, Indiana: Agency for Instructional Technology and Association for Educational Communications and Technology.

    Google Scholar 

  • Yang, D., & Yang, Q. (2005). Customizable distance learning: Criteria for developing learning objects and learning model templates. In Proceedings of the 7th international conference on electronic commerce, Xi’an (China) (pp. 765–770). ACM.

  • Zimmermann, B., Meyer, M., Rensing, C., & Steinmetz, R. (2007). Improving retrieval of reusable learning resources by estimating adaptation effort. In Proceedings of the first international workshop on learning object discovery and exchange, Vol. 311, pp. 46–53.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University Carlos III of Madrid, Av. Universidad 30, 28911, Leganes, Madrid, Spain

    Javier Sanz-Rodriguez

  2. University of Cadiz, C/Chile, s/n, 1003, Cadiz, Spain

    Juan Manuel Dodero

  3. University of Alcala de Henares, Ctra. Barcelona km. 33600, 28871, Alcala de Henares, Madrid, Spain

    Salvador Sanchez-Alonso

Authors
  1. Javier Sanz-Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juan Manuel Dodero
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Salvador Sanchez-Alonso
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Javier Sanz-Rodriguez.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sanz-Rodriguez, J., Dodero, J.M. & Sanchez-Alonso, S. Metrics-based evaluation of learning object reusability. Software Qual J 19, 121–140 (2011). https://doi.org/10.1007/s11219-010-9108-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-010-9108-5

Keywords

Search

Navigation