Abstract
In recent years, a method for sampling from conditional distributions for categorical data has been presented by Diaconis and Sturmfels. Their algorithm is based on the algebraic theory of toric ideals which are used to create so called “Markov Bases”. The Diaconis-Sturmfels algorithm leads to a non-asymptotic Monte Carlo Markov Chain algorithm for exact inference on some classes of models, such as log-linear models. In this paper we apply the Diaconis-Sturmfels algorithm to a set of models arising from the rater agreement problem with special attention to the multi-rater case. The relevant Markov bases are explicitly computed and some results for simplify the computation are presented. An extended example on a real data set shows the wide applicability of this methodology.
Similar content being viewed by others
References
Agresti A (1992) Modelling patterns of agreement and disagreement. Statistical Methods in Medical Research 1: 201–218
Agresti A (2001) Exact inference for categorical data: Recent advances and continuing controversies. Statist. Med. 20: 2709–2722
Agresti A (2002) Categorical Data Analysis, 2nd edn. Wiley, New York
Becker MP (1990) Quasisymmetric models for the analysis of square contingency tables. J.R. Statist. Soc. 52(2): 369–378
Bigatti A, La Scala R, Robbiano L (1999) Computing toric ideals. J. Symb. Comput. 27: 351–365
Bigatti A, Robbiano L (2001) Toric ideals. Mat. Contemp. 21: 1–25
Bishop YM, Fienberg S, Holland PW (1975) Discrete multivariate analysis: theory and practice. MIT Press, Cambridge
CDER (2000) Developing Medical Imaging Drugs and Biological products. Guidance for industry. U.S. Department of Health and Human Services, Food and Drug Administration
Chib S, Greenberg E (1995) Understanding the Metropolis-Hastings algorithm. Amer. Statist. 49(4): 327–335
CoCoATeam (2004) CoCoA: a system for doing Computations in Commutative Algebra, Available at http://cocoa.dima.unige.it
Cohen J (1960) A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20: 37–46
Darroch JN, McCloud PI (1986) Category distinguishability and observer agreement. Australian Journal of Statistics 28(3): 371–388
Diaconis P, Sturmfels B (1998) Algebraic algorithms for sampling from conditional distributions. Ann. Statist. 26(1): 363–397
Dillon WR, Mulani N (1984) A probabilistic latent class model for assessing inter-judge reliability. Multivariate Behavioral Research 19: 438–458
Dobra A (2003) Markov bases for decomposable graphical models. Bernoulli 9(6): 1–16
Dobra A, Sullivant S (2002) Markov bases for reducible sets of tables. Workshop on Multidimensional tables: Statistics, Combinatorial Optimization, and Gröbner bases, University of California, Davis, CA
Donner A, Klar N (1996) The statistical analysis of kappa statistics in multiple samples. Journal of Clinical Epidemiology 49: 1053–1058
Donner A, Shoukri MM, Klar N, Bartfay E (2000) Testing the equality of two dependent kappa statistics. Statist. Med. 19: 373–387.
Fienberg S (1980) The Analysis of Cross-Classified Categorical Data. MIT Press, Cambridge
Fleiss JL (1981) Statistical Methods for Rates and Proportions, 2nd edn. Wiley, New York.
Haberman SJ (1974) The Analysis of Frequency Data. The University of Chicago Press, Chicago and London
Hanley JA (1998) Receiver Operating Characteristic (ROC) curves. In: Armitage P, Colton T (eds) Encyclopedia of Biostatistics, 3738–3745, Wiley, New York
Klar N, Lipsitz SR, Ibrahim JG (2000) An estimating equation approach for modelling Kappa.Biom. J. 42: 45–58
Kreuzer M, Robbiano L (2000) Computational Commutative Algebra 1. Springer, New York
Landis RJ, Koch GG (1975) A review of statistical methods in the analysis of data arising from observer reliability studies, Parts I and II. Statist. Neerlandica 29: 101–123, 151–161
Pistone G, Riccomagno E, Wynn HP (2001) Algebraic Statistics: Computational Commutative Algebra in Statistics. Chapman&Hall/CRC, Boca Raton
Rapallo F (2003) Algebraic Markov bases and MCMC for two-way contingency tables. Scand. J. Statist. 30(2): 385–397
Rudas T, Bergsma WP (2002) On generalised symmetry. Technical report, Paul Sabbatier University, Toulouse, France, preprint
Smith PWF, Forster JJ, McDonald JW (1996) Monte Carlo exact tests for square contingency tables. J. R. Statist. Soc. 159(2): 309–321
Tanner MA, Young MA (1985) Modelling agreement among raters. J. Am. Statist. Assoc. 80: 175–180
Author information
Authors and Affiliations
Additional information
Partially supported by MIUR Cofin03 (G. Consonni) and by INdAM projectAlgebraic Statistics.
Rights and permissions
About this article
Cite this article
Rapallo, F. Algebraic exact inference for rater agreement models. Statistical Methods & Applications 14, 45–66 (2005). https://doi.org/10.1007/BF02511574
Issue Date:
DOI: https://doi.org/10.1007/BF02511574