Skip to main content
SpringerLink
Log in

ROC curve and covariates: extending induced methodology to the non-parametric framework

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

Continuous diagnostic tests are often used to discriminate between diseased and healthy populations. The receiver operating characteristic (ROC) curve is a widely used tool that provides a graphical visualisation of the effectiveness of such tests. The potential performance of the tests in terms of distinguishing diseased from healthy people may be strongly influenced by covariates, and a variety of regression methods for adjusting ROC curves has been developed. Until now, these methodologies have assumed that covariate effects have parametric forms, but in this paper we extend the induced methodology by allowing for arbitrary non-parametric effects of a continuous covariate. To this end, local polynomial kernel smoothers are used in the estimation procedure. Our method allows for covariate effect not only on the mean, but also on the variance of the diagnostic test. We also present a bootstrap-based method for testing for a significant covariate effect on the ROC curve. To illustrate the method, endocrine data were analysed with the aim of assessing the performance of anthropometry for predicting clusters of cardiovascular risk factors in an adult population in Galicia (NW Spain), duly adjusted for age. The proposed methodology has proved useful for providing age-specific thresholds for anthropometric measures in the Galician community.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alonzo, T.A., Pepe, M.S.: Distribution-free ROC analysis using binary regression techniques. Biostatistics 3, 421–432 (2002)

    Article  MATH  Google Scholar 

  • Cai, T.: Semi-parametric ROC regression analysis with placement values. Biostatistics 5, 45–60 (2004)

    Article  MATH  Google Scholar 

  • Cai, T., Pepe, M.S.: Semiparametric receiver operating characteristic analysis to evaluate biomarkers for disease. J. Am. Stat. Assoc. 97, 1099–1107 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  • Carey, V.J., Walters, E.E., Colditz, G.A., Solomon, C.G., Willet, W.C., Rosner, B.A., Speizer, F.E., Manson, J.E.: Body fat distribution and risk of noninsulin-dependent diabetes in women: the Nurses’ Health Study. Am. J. Epidemiol. 145, 614–619 (1997)

    Google Scholar 

  • Chumlea, W.C., Baumgartner, R.N., Garry, P.J., Rhyne, R.L., Nicholson, C., Wayne, S.: Fat distribution and blood lipids in a sample of healthy elderly people. Int. J. Obes. 16, 125–133 (1992)

    Google Scholar 

  • de Boor, C.A.: A Practical Guide to Splines. Springer, New York (2001). Revised Edition

    MATH  Google Scholar 

  • Efron, B., Tibshirani, R.J.: An Introduction to the Bootstrap. Chapman & Hall, New York (1993)

    MATH  Google Scholar 

  • Eilers, P.H.C., Marx, B.D.: Flexible smoothing with B-splines and penalties. Stat. Sci. 11, 89–121 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  • Fan, J., Gijbels, I.: Local Polynomial Modelling and Its Applications. Chapman & Hall, CRC (1996)

  • Fan, J., Marron, J.S.: Fast implementation of non-parametric curve estimators. J. Comput. Graph. Stat. 3, 35–56 (1994)

    Article  Google Scholar 

  • Fan, J., Yao, Q.: Efficient estimation of conditional variance functions in stochastic regression. Biometrika 85, 645–660 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  • Faraggi, D.: Adjusting receiver operating characteristic curves and related indices for covariates. Statistician 52, 179–192 (2003)

    MathSciNet  Google Scholar 

  • González-Manteiga, W., Pardo Fernández, J.C., Van Keilegom, I.: ROC curves in nonparametric location-scale regression models. Scand. J. Stat. (2010). doi:10.1111/j.1467-9469.2010.00693x

    Google Scholar 

  • Haslam, D.W., James, W.P.T.: Obesity. Lancet 366, 1197–1209 (2005)

    Article  Google Scholar 

  • Hastie, T.J., Tibshirani, R.J.: Generalized Additive Models. Chapman & Hall, London (1990)

    MATH  Google Scholar 

  • Hsieh, F., Turnbull, B.W.: Nonparametric and semiparametric estimation of the receiver operating characteristic curve. Ann. Stat. 24, 24–40 (1996)

    MathSciNet  Google Scholar 

  • Hu, G., Qiao, Q., Tuomilehto, J., Eliasson, M., Feskens, E.J., Pyörälä, K.: Plasma insulin and cardiovascular mortality in non-diabetic European men and women: a meta-analysis of data from eleven prospective studies. The DECODE Insulin Study Group. Diabetologia 47, 1245–1256 (2004)

    Google Scholar 

  • International Diabetes Federation: The IDF consensus worldwide definition of the metabolic syndrome. http://www.idf.org/webdata/docs/IDF_Meta_def_final.pdf (2008). Accessed 7 September 2009

  • Janes, H., Pepe, M.S.: Adjusting for covariate effects on classification accuracy using the covariate-adjusted ROC curve. Biometrika 96, 371–382 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  • Karelis, A.D., St-Pierre, D.H., Conus, F., Rabasa-Lhoret, R., Poehlman, E.T.: Metabolic and body composition factors in subgroups of obesity: What do we know? J. Clin. Endocrinol. Metab. 89, 2569–2575 (2004)

    Article  Google Scholar 

  • Lloyd, C.J.: Using smooth receiver operating characteristic curves to summarize and compare diagnostic systems. J. Am. Stat. Assoc. 93, 1356–1364 (1998)

    Article  Google Scholar 

  • López-de-Ullibarri, I., Cao, R., Cadarso-Suárez, C., Lado, M.J.: Nonparametric estimation of conditional ROC curves: application to discrimination tasks in computerized detection of early breast cancer. Comput. Stat. Data Anal. 52, 2623–2631 (2008)

    Article  MATH  Google Scholar 

  • MacIntosh, M.W., Pepe, M.S.: Combining several screening test: optimality of the risk score. Biometrics 58, 657–664 (2002)

    Article  MathSciNet  Google Scholar 

  • Metz, C.E.: Basic principles of ROC analysis. Semin. Nucl. Med. 8, 183–298 (1978)

    Article  Google Scholar 

  • Mirmiran, P., Esmaillzadeh, A., Azizi, F.: Detection of cardiovascular risk factors by anthropometric measures in Tehranian adults: receiver operating characteristic (ROC) curve analysis. Eur. J. Clin. Nutr. 58, 1110–1118 (2004)

    Article  Google Scholar 

  • Nadaraya, E.A.: On estimating regression. Theory Probab. Appl. 9, 141–142 (1964)

    Article  Google Scholar 

  • Peden, M., McGee, K., Krug, E. (eds.): Injury: A Leading Cause of the Global Burden of Disease, 2000. World Health Organization, Geneva (2006)

    Google Scholar 

  • Peng, L., Zhou, X.H.: Local linear smoothing of receiver operating characteristic (ROC) curves. J. Stat. Plan. Inference 118, 129–143 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  • Pepe, M.S.: Three approaches to regression analysis of receiver operating characteristic curves for continuous test results. Biometrics 54, 124–135 (1998)

    Article  MATH  Google Scholar 

  • Pepe, M.S.: The Statistical Evaluation of Medical Tests for Classification and Prediction. Oxford University Press, New York (2003)

    MATH  Google Scholar 

  • Ren, H., Zhou, X.H., Liang, H.: A flexible method for estimating the ROC curve. J. Appl. Stat. 31, 773–784 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  • Ruppert, D., Wand, M.P., Holst, U., Hössjer, O.: Local polinomial variance function estimation. Technometrics 39, 262–273 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  • Schisterman, E.F., Faraggi, D., Reiser, B.: Adjusting the generalized ROC curve for covariates. Stat. Med. 23, 3319–3331 (2004)

    Article  Google Scholar 

  • Su, J.Q., Liu, J.S.: Linear combinations of multiple diagnostic markers. J. Am. Stat. Assoc. 88, 1350–1355 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  • Swets, J.A., Pickett, R.M.: Evaluation of Diagnostic Systems: Methods from Signal Detection Theory. Academic Press, New York (1982)

    Google Scholar 

  • Tomé, M.A., Botana, M.A., Cadarso-Suárez, C., Rego-Iraeta, A., Fernández-Mariño, A., Mato, J.A., Solache, I., Perez-Fernandez, R.: Prevalence of metabolic syndrome in Galicia (NW Spain) on four alternative definitions and association with insulin resistance. J. Endocrinol. Invest. 32, 505–511 (2008)

    Google Scholar 

  • Tosteson, A.N., Begg, C.B.: A general regression methodology for ROC curve estimation. Med. Decis. Mak. 8, 204–215 (1988)

    Article  Google Scholar 

  • Wand, M.P, Jones, M.C.: Kernel Smoothing. Chapman & Hall, London (1995)

    MATH  Google Scholar 

  • Watson, G.S.: Smooth regression analysis. Sankhyā, Ser. A 26, 359–372 (1964)

    MATH  Google Scholar 

  • Zheng, Y., Heagerty, P.J.: Semiparametric estimation of time-dependent ROC curves for longitudinal marker data. Biostatistics 4, 615–632 (2004)

    Article  Google Scholar 

  • Zhou, X.H., Harezlak, J.: Comparison of bandwidth selection methods for kernel smoothing of ROC curves. Stat. Med. 21, 2045–2055 (2002)

    Article  Google Scholar 

  • Zhou, X.H., Obuchowski, N.A., McClish, D.K.: Statistical Methods in Diagnostic Medicine. Wiley, New York (2002)

    Book  MATH  Google Scholar 

  • Zimmet, P., Alberti, K.G.M.M., Shaw, J.: Global and societal implications of the diabetes epidemic. Nature. 414, 782–787 (2001)

    Article  Google Scholar 

  • Zou, K.H., Hall, W.J., Shapiro, D.E.: Smooth nonparametric receiver operating characteristic (ROC) curves for continuous diagnostic test. Stat. Med. 16, 2143–2165 (1997)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Unit of Biostatistics, Dept. of Statistics and Operations Research, Faculty of Medicine, University of Santiago de Compostela, 15782, Santiago de Compostela, Spain

    María Xosé Rodríguez-Álvarez & Carmen Cadarso-Suárez

  2. Instituto de Investigación Sanitaria de Santiago (IDIS), Santiago de Compostela, Spain

    María Xosé Rodríguez-Álvarez & Carmen Cadarso-Suárez

  3. Dept. of Statistics and Operational Research, University of Vigo, c/Torrecedeira 86, 36208, Vigo, Spain

    Javier Roca-Pardiñas

Authors
  1. María Xosé Rodríguez-Álvarez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Javier Roca-Pardiñas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Carmen Cadarso-Suárez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to María Xosé Rodríguez-Álvarez.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rodríguez-Álvarez, M.X., Roca-Pardiñas, J. & Cadarso-Suárez, C. ROC curve and covariates: extending induced methodology to the non-parametric framework. Stat Comput 21, 483–499 (2011). https://doi.org/10.1007/s11222-010-9184-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-010-9184-1

Keywords

Search

Navigation