Skip to main content
SpringerLink
Log in

Modeling Threshold Interaction Effects Through the Logistic Classification Trunk

  • Published:
Journal of Classification Aims and scope Submit manuscript

Abstract

We introduce a model dealing with the identification of interaction effects in binary response data, which integrates recursive partitioning and generalized linear models. It derives from an ad-hoc specification and consequent implementation of the Simultaneous Threshold Interaction Modeling Algorithm (STIMA). The model, called Logistic Classification Trunk, allows us to obtain regression parameters by maximum likelihood through the simultaneous estimation of both main effects and threshold interaction effects. The main feature of this model is that it allows the user to evaluate a unique model and simultaneously the importance of both effects obtained by first growing a classification trunk and then by pruning it back to avoid overfitting. We investigate the choice of a suitable pruning parameter through a simulation study and compare the classification accuracy of the Logistic Classification Trunk with that of 13 alternative models/classifiers on 25 binary response datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • ALLEN, J., and LE, H. (2008), “An Additional Measure of Overall Effect Size for Logistic Regression Models”, Journal of Educational and Behavioral Statistics, 33(4), 416–441.

    Article  Google Scholar 

  • ASSMANN, S.F., HOSMER, D.W., LEMESHOW, S., and MUNDT, K.A. (1996), “Confidence Intervals for Measures of Interaction”, Epidemiology, 7(3), 286–290.

    Article  Google Scholar 

  • BACHE, K., and LICHMAN, M. (2013), “UCI Machine Learning Repository”, University of California, Irvine, School of Information and Computer Sciences, http://archive.ics.uci.edu/ml/.

  • BALLI, H.O., and SORENSON, B.E. ( 2013), “Interaction Effects in Econometrics”, Empirical Economics, 45, 583–603.

    Article  Google Scholar 

  • BERRINGTON DE GONZÁLEZ, A., and COX, D.R. (2007), “Interpretation of Interaction: A Review”, The Annals of Applied Statistics, 1(2), 371–385.

    Article  MathSciNet  MATH  Google Scholar 

  • BREIMAN, L. (1996), “Bagging Predictors”, Machine Learning, 24(2), 123–140.

    MATH  Google Scholar 

  • BREIMAN, L. (2001), “Random Forests”, Machine Learning, 45(1), 5–32.

    Article  MATH  Google Scholar 

  • BREIMAN, L., FRIEDMAN, J., OLSHEN, R., and STONE, C. (1984), Classification and Regression Trees, Monterey CA: Wadsworth and Brooks.

    MATH  Google Scholar 

  • CHIPMAN, H.A., GEORGE, E.I., and MCCULLOCH, R.E. (2010), “Bart: Bayesian Additive Regression Trees”, The Annals of Applied Statistics, 4(1), 266–298.

    Article  MathSciNet  MATH  Google Scholar 

  • COHEN, J., COHEN, P., WEST, S., and AIKEN, L. (2003), Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, Mahwah NJ: Lawrence Erlbaum.

    Google Scholar 

  • CULP, M., JOHNSON, K., and MICHAILIDIS, G. (2012), ada: An R Package for Stochastic Boosting, R package version 2.0-3, http://CRAN.R-project.org/package=ada.

  • DAWSON, J. (2014), “Moderation in Management Research: What, Why, When, and How”, Journal of Business and Psychology, 29(1), 1–19.

    Article  MathSciNet  Google Scholar 

  • DEMŠAR, J. (2006), “Statistical Comparisons of Classifiers Over Multiple Data Sets”, Journal of Machine Learning Research, 7, 1–30.

    MathSciNet  MATH  Google Scholar 

  • DUSSELDORP, E., CONVERSANO, C., and VAN OS, B.J. (2010), “Combining an Additive and Tree-Based Regression Model Simultaneously: Stima”, Journal of Computational and Graphical Statistics, 19(3), 514–530.

    Article  MathSciNet  Google Scholar 

  • FLEMING, T., and HARRINGTON, D. (1991), Counting Processes and Survival Analysis, Hoboken NJ: John Wiley and Sons, Inc.

    MATH  Google Scholar 

  • FRIEDMAN, J.H. (1991), “Multivariate Adaptive Regression Splines”, The Annals of Statistics, 19(1), 1–67.

    Article  MathSciNet  MATH  Google Scholar 

  • FRIEDMAN, J.H., HASTIE, T., and TIBSHIRANI, R. (2000), “Additive Logistic Regression: A Statistical View of Boosting”, The Annals of Statistics, 28(2), 337–374.

    Article  MathSciNet  MATH  Google Scholar 

  • GRUBINGER, T., ZEILEIS, A., and PFEIFFER, K.P. (2011), “Evolutionary Learning of Globally Optimal Classification and Regression Trees in R”,Working Paper 2011-20. Working Papers in Economics and Statistics, Research Platform Empirical and Experimental Economics, Universitaet Innsbruck, http://EconPapers.RePEc.org/RePEc:inn:wpaper:2011-20.

  • HALVORSEN, T. (2012), “ElemStatLearn: Data Sets, Functions and Examples”, in The Elements of Statistical Learning, Data Mining, Inference, and Prediction, T. Hastie, R. Tibshirani, and J. Friedman, R package version 2012.04-0, http://CRAN.R-project.org/package=ElemStatLearn.

  • HAND, D.J. (1997), Construction and Assessment of Classification Rules, Chichester: Wiley.

    MATH  Google Scholar 

  • HASTIE, T.J., FRIEDMAN, J.H., and TIBSHIRANI, R.J. (2009), Elements of Statistical Learning, New York: Springer.

    Book  MATH  Google Scholar 

  • HASTIE, T.J., and TIBSHIRANI, R.J. (1990), Generalized AdditiveModels, London: Chapman and Hall.

    MATH  Google Scholar 

  • HASTIE, T. (2013), gam: Generalized Additive Models, R package version 1.09, http://CRAN.R-project.org/package=gam.

  • HASTIE, T., and TIBSHIRANI, R. (2013), mda: Mixture and Flexible Discriminant Analysis, R package version 0.4-4,S original by Hastie and Tishirani, original R port by Leisch, Hornik, and Ripley, http://CRAN.R-project.org/package=mda.

  • HOTHORN, T., HORNIK, K., and ZEILEIS, A. (2006) , “Unbiased Recursive Partitioning: A Conditional Inference Framework”, Journal of Computational and Graphical Statistics, 15(3), 651–674.

    Article  MathSciNet  Google Scholar 

  • HOSMER, D.W., LEMESHOW, S., and STURDIVANT, R.X. (2013), Applied Logistic Regression (3rd. ed.), Hoboken, NJ: John Wiley and Sons, Inc.

  • HOSMER, D.W., and LEMESHOW, S. (2000), Applied Logistic Regression (2nd ed.), Hoboken NJ: John Wiley and Sons, Inc.

  • HOSMER, D.W., and LEMESHOW, S. (1989), Applied Logistic Regression (1st ed.), Hoboken NJ: John Wiley and Sons, Inc.

  • HOTHORN, T. (2014), “TH.Data: TH’s Data Archive”, R package version 1.0-3, http://CRAN.R-project.org/package=TH.data.

  • KALBFLEISCH, J.D., and PRENTICE, R.L.(1980), The Statistical Analysis of Failure Time Data, New York: John Wiley and Sons.

    MATH  Google Scholar 

  • KAPELNER, A., and BLEICH, J. (2013), “bartMachine: A Powerful Tool for Machine Learning”, ArXiv e-prints, http://arxiv.org/abs/1312.2171.

  • KARATZOGLOU, A., MEYER, D., and HORNIK, K. (2006), “Support Vector Machines in R”, Journal of Statistical Software, 15(9), 1–28.

    Article  Google Scholar 

  • KARATZOGLOU, A., SMOLA, A., HORNIK, K., and ZEILEIS, A. (2004), “kernlab - An S4 Package for Kernel Methods in R”, Journal of Statistical Software, 11(9), 1–20.

    Article  Google Scholar 

  • KIM, H., and LOH, W. (2001), ‘Classification TreesWith Unbiased Multiway Splits’, Journal of the American Statistical Association, 96, 589–604.

    Article  MathSciNet  Google Scholar 

  • KUHN, A. [Contributions from J. Wing, S. Weston, A. Williams, C. Keefer, A. Engelhardt, T. Cooper, Z. Mayer and the R Core Team] (2014), “caret: Classification and Regression Training. R package version 6.0-30”, http://CRAN.R-project.org/package=caret.

  • KUHN, M., WESTON, S., and COULTER, N. (2014), “C50: C5.0 Decision Trees and Rule-Based Models”, R package version 0.1.0-19, (C code for C5.0 by R. Quinlan), http://CRAN.R-project.org/package=C50.

  • LANDWEHR, N., HALL, M. and FRANK, E. (2005), “Logistic Model Trees”, Machine Learning, 59(1-2), 161–205.

    Article  MATH  Google Scholar 

  • LEISCH, F., and DIMITRIADU, E. (2010), “mlbench: Machine Learning Benchmark Problems”, R package version 2.1-1, http://CRAN.R-project.org/package=mlbench.

  • LIAW, A., and WIENER, M. (2002), “Classification and Regression by randomForest”, R News, 2(3), 18–22.

    Google Scholar 

  • LOH, W.-Y. (2009), “Improving the Precision of Classification Trees”, The Annals of Applied Statistics, 3(4), 1710–1737.

    Article  MathSciNet  MATH  Google Scholar 

  • LOH, W.-Y., and SHIH, Y.-S. (1997), “Split Selection Methods for Classification Trees”, Statistica Sinica, 7(4), 815–840.

    MathSciNet  MATH  Google Scholar 

  • MCCULLAGH, P., and NELDER, J.A. (1989), Generalized Linear Models, London: Chapman and Hall.

    Book  MATH  Google Scholar 

  • MCFADDEN, D. (1974), “The Measurement of Urban Travel Demand”, Journal of Public Economics, 3(4), 303–328.

    Article  Google Scholar 

  • MENARD, S. (2000), “Coefficients of Determination for Multiple Logistic Regression Analysis”, The American Statistician, 54(1), 17–24.

    Google Scholar 

  • MORGAN, J.N., and SONQUIST, J.A. (1963), “Problems in the Analysis of Survey Data, and a Proposal”, Journal of the American Statistical Association, 58(302), 415–434.

    Article  MATH  Google Scholar 

  • NAGLER, J. (1993), “Scobit: An Alternative Estimator to Logit and Probit”, American Journal of Political Science, 38(1), 230–255.

    Article  Google Scholar 

  • NELDER, J.A., and WEDDERBURN, R.W.M. (1972), “Generalized Linear Models”, Journal of the Royal Statistical Society, Series A, 135, 370–384.

    Article  Google Scholar 

  • PETERS, A., and HOTHORN, T. (2013), “ipred: Improved Predictors”, R package version 0.9-3, http://CRAN.R-project.org/package=ipred.

  • QUINLAN, J.R. (1993), C4.5: Programs for Machine Learning, San Franciso CA: Morgan Kaufmann Publishers Inc.

  • R CORE TEAM (2016), R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria.

    Google Scholar 

  • RIPLEY, B.D. (1996), Pattern Recognition and Neural Networks, New York NY: Cambridge University Press.

  • RUSCH, T., LEE, I., HORNIK, K., JANK, W., and ZEILEIS, A. (2013), “Influencing Elections with Statistics: Targeting Voters with Logistic Regression Trees”, The Annals of Applied Statistics, 7(3), 1612–1639.

    Article  MathSciNet  MATH  Google Scholar 

  • THERNEAU, T., ATKINSON, B., and RIPLEY, B.D. (2014). rpart: Recursive Partitioning and Regression Trees. R package version 4.1-5, http://CRAN.R-project.org/package=rpart.

  • VENABLES, W.N., and RIPLEY, B.D. (2002) Modern Applied Statistics with S, NewYork: Springer.

    Book  MATH  Google Scholar 

  • WITTEN, I.H., and FRANK, E. (2005), Data Mining: Practical Machine Learning Tools and Techniques, San Francisco CA: Morgan Kaufmann.

    MATH  Google Scholar 

  • ZEILEIS, A., HOTHORN, T., and HORNIK, K.(2008), “Model-Based Recursive Partitioning”, Journal of Computational and Graphical Statistics, 17(2), 492–514.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Scienze Economiche e Aziendali, Università di Cagliari, Cagliari, Italy

    Claudio Conversano

  2. Leiden University, Leiden, The Netherlands

    Elise Dusseldorp

  3. Netherlands Organization for Applied Scientific Research (TNO), The Hague, The Netherlands

    Elise Dusseldorp

Authors
  1. Claudio Conversano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elise Dusseldorp
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Claudio Conversano.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Conversano, C., Dusseldorp, E. Modeling Threshold Interaction Effects Through the Logistic Classification Trunk. J Classif 34, 399–426 (2017). https://doi.org/10.1007/s00357-017-9241-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00357-017-9241-y

Keywords

Search

Navigation