Skip to main content
SpringerLink
Log in

Comparing degree programs from students’ assessments: A LCRA-based adjusted composite indicator

  • Published:
Statistical Methods & Applications Aims and scope Submit manuscript

Abstract

Taking into account the students’ evaluation of the quality of degree programs this paper presents a proposal for building up an adjusted performance indicator based on Latent Class Regression Analysis. The method enables us (i) to summarize in a single indicator statement multiple facets evaluated by students through a survey questionnaire and (ii) to control the variability in the evaluations that is mainly attributable to the characteristics (often referred as the Potential Confounding Factors) of the evaluators (students) rather than to real differences in the performances of the degree programs under evaluation. A simulation study is implemented in order to test the method and assess its potential when the composition of the degree programs as regards to students’ characteristics is sensibly different between one another. Results suggest that when the evaluations are strongly affected by the students’ covariates, the assessment based on the value of an unadjusted indicator can lead to bias and unreliable conclusions about the differences in performance. An application to real data is also provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Agresti A (2002) Categorical data analysis. Wiley-Interscience, Hoboken

    Book  MATH  Google Scholar 

  • Bartholomew DJ (1998) Scaling unobservable constructs in social science. Appl Stat 47: 1–13

    MATH  Google Scholar 

  • Bartholomew DJ, Steele F, Moustaki I, Galbraith JI (2002) The analysis and interpretation of multivariate analysis for social scientists. Chapman & Hall, Boca Raton

    MATH  Google Scholar 

  • Bayol MP, de la Foye A, Tellier C, Tenenhaus M (2000) Use of PLS path modelling to estimate the European consumer satisfaction index (ECSI) model. Stat Appl 12(3): 361–375

    Google Scholar 

  • Bird SM, Cox D, Goldstein H, Holt T, Smith PC (2005) Performance indicators: good, bad and ugly. J R Stat Soc A 168(1): 1–27

    Article  MathSciNet  MATH  Google Scholar 

  • Capursi V, Porcu M (2001) La didattica universitaria valutata dagli studenti: un indicatore basato su misure di distanza fra distribuzioni di giudizi. In: Atti Convegno Intermedio della Società Italiana di Statistica ‘Processi e Metodi Statistici di Valutazione’, Roma 4–6 giugno 2001. Società Italiana di Statistica

  • Cox DR, Fitzpatrick R, Fletcher AE, Gore SM, Spiegelhalter DJ, Jones DR (1992) Quality of life assessment: can we keep it simple?. J R Stat Soc A 155: 353–393

    Article  Google Scholar 

  • Draper D, Gittoes M (2004) Statistical analysis of performance indicators in UK higher education. J R Stat Soc A 167(3): 449–474

    Article  MathSciNet  Google Scholar 

  • Goldstein H, Spiegelhalter DJ (1996) League tables and their limitations: statistical issues in comparisons of istitutional performance. J R Stat Soc A 159: 385–443

    Article  Google Scholar 

  • Grilli L, Rampichini C (2007) Multilevel factor models for ordinal variables. Struct Equ Model A Multidiscip J 14(1): 1–25

    MathSciNet  Google Scholar 

  • Hagenaars JA, McCutcheon AL (2002) Applied latent class analysis. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  • Lanza ST, Collins LM, Lemmon DR, Schafer JL (2007) PROC LCA: a SAS procedure for latent class analysis. Struct Equ Model A Multidiscip J 14(4): 671–694

    Article  MathSciNet  Google Scholar 

  • Leckie G, Goldstein H (2009) The limitation of using school league tables to inform school choice. J R Stat Soc A 172(4): 835–851

    Article  MathSciNet  Google Scholar 

  • Leti G (1983) Statistica Descrittiva. Il Mulino, Bologna

    Google Scholar 

  • Linzer DA, Lewis J (2010) poLCA: polytomous variable latent class analysis. R package version 1.2. http://userwww.service.emory.edu/~dlinzer/poLCA/

  • Moustaki I, Knott D (2000) Generalized latent trait models. Psychometrika 65: 391–411

    Article  MathSciNet  Google Scholar 

  • Quatto P (2011) Descriptive analysis of student ratings. Book of short abstracts IES 2011. (http://ies2011.ds.unifi.it/BookOfAbstractsIES2011.pdf)

  • Rampichini C, Grilli L, Petrucci A (2004) Analysis of university course evaluations: from descriptive measures to multilevel models. Stat Methods Appl 13(3): 357–371

    MathSciNet  MATH  Google Scholar 

  • Skrondal A, Rabe-Hesketh S (2004) Generalized latent variables modeling. Chapman & Hall, Boca Raton

    Book  Google Scholar 

  • Sulis I, Porcu M (2011) Assessing the quality of the management of degree programs by latent class analysis. In: Attanasio M, Capursi V (eds) Statistical methods for the evaluation of university systems, series contributions to statistics. Springer, Berlin, pp 161–172

    Chapter  Google Scholar 

  • Vermunt JK (2003) Multilevel latent class models. Sociol Methodol 33: 213–234

    Article  Google Scholar 

  • Vermunt JK (2008) Latent class and finite mixture models for multilevel data sets. Stat Methods Med Res 17: 33–51

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dipartimento Scienze Sociali e delle Istituzioni, Università di Cagliari, Cagliari, Italy

    Isabella Sulis & Mariano Porcu

Authors
  1. Isabella Sulis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mariano Porcu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Isabella Sulis.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sulis, I., Porcu, M. Comparing degree programs from students’ assessments: A LCRA-based adjusted composite indicator. Stat Methods Appl 21, 193–209 (2012). https://doi.org/10.1007/s10260-011-0185-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10260-011-0185-9

Keywords

Search

Navigation