Table of Contents Author Guidelines Submit a Manuscript
Journal of Probability and Statistics
Volume 2013, Article ID 325831, 9 pages
http://dx.doi.org/10.1155/2013/325831
Research Article

Weighted Kappas for Tables

Unit of Methodology and Statistics, Institute of Psychology, Leiden University, P.O. Box 9555, 2300 RB Leiden, The Netherlands

Received 10 April 2013; Accepted 26 May 2013

Academic Editor: Ricardas Zitikis

Copyright © 2013 Matthijs J. Warrens. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. M. A. Tanner and M. A. Young, “Modeling ordinal scale disagreement,” Psychological Bulletin, vol. 98, no. 2, pp. 408–415, 1985. View at Publisher · View at Google Scholar · View at Scopus
  2. A. Agresti, “A model for agreement between ratings on an ordinal scale,” Biometrics, vol. 44, no. 2, pp. 539–548, 1988. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at Scopus
  3. A. Agresti, Analysis of Ordinal Categorical Data, John Wiley & Sons, Hoboken, NJ, USA, 2nd edition, 2010. View at MathSciNet
  4. P. Graham and R. Jackson, “The analysis of ordinal agreement data: beyond weighted kappa,” Journal of Clinical Epidemiology, vol. 46, no. 9, pp. 1055–1062, 1993. View at Publisher · View at Google Scholar · View at Scopus
  5. M. Maclure and W. C. Willett, “Misinterpretation and misuse of the Kappa statistic,” American Journal of Epidemiology, vol. 126, no. 2, pp. 161–169, 1987. View at Publisher · View at Google Scholar · View at Scopus
  6. J. de Mast and W. N. van Wieringen, “Measurement system analysis for categorical measurements: agreement and kappa-type indices,” Journal of Quality Technology, vol. 39, no. 3, pp. 191–202, 2007. View at Google Scholar · View at Scopus
  7. M. J. Warrens, “Inequalities between kappa and kappa-like statistics for k×k tables,” Psychometrika, vol. 75, no. 1, pp. 176–185, 2010. View at Publisher · View at Google Scholar · View at MathSciNet
  8. J. Cohen, “Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit,” Psychological Bulletin, vol. 70, no. 4, pp. 213–220, 1968. View at Publisher · View at Google Scholar · View at Scopus
  9. D. V. Cicchetti, “Assessing inter rater reliability for rating scales: resolving some basic issues,” British Journal of Psychiatry, vol. 129, no. 11, pp. 452–456, 1976. View at Publisher · View at Google Scholar · View at Scopus
  10. M. J. Warrens, “Cohen's kappa is a weighted average,” Statistical Methodology, vol. 8, no. 6, pp. 473–484, 2011. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  11. J. Cohen, “A coefficient of agreement for nominal scales,” Educational and Psychological Measurement, vol. 20, pp. 37–46, 1960. View at Publisher · View at Google Scholar
  12. S. Vanbelle and A. Albert, “A note on the linearly weighted kappa coefficient for ordinal scales,” Statistical Methodology, vol. 6, no. 2, pp. 157–163, 2009. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  13. M. J. Warrens, “Cohen's linearly weighted kappa is a weighted average of 2×2 kappas,” Psychometrika, vol. 76, no. 3, pp. 471–486, 2011. View at Publisher · View at Google Scholar · View at MathSciNet
  14. J. L. Fleiss and J. Cohen, “The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability,” Educational and Psychological Measurement, vol. 33, pp. 613–619, 1973. View at Google Scholar
  15. M. J. Warrens, “Some paradoxical results for the quadratically weighted kappa,” Psychometrika, vol. 77, no. 2, pp. 315–323, 2012. View at Publisher · View at Google Scholar · View at MathSciNet
  16. L. M. Hsu and R. Field, “Interrater agreement measures: comments on kappan, Cohen's kappa, Scott's π and Aickin's α,” Understanding Statistics, vol. 2, pp. 205–219, 2003. View at Publisher · View at Google Scholar
  17. E. Bashkansky, T. Gadrich, and D. Knani, “Some metrological aspects of the comparison between two ordinal measuring systems,” Accreditation and Quality Assurance, vol. 16, no. 2, pp. 63–72, 2011. View at Publisher · View at Google Scholar · View at Scopus
  18. J. de Mast, “Agreement and kappa-type indices,” The American Statistician, vol. 61, no. 2, pp. 148–153, 2007. View at Publisher · View at Google Scholar · View at MathSciNet
  19. J. S. Uebersax, “Diversity of decision-making models and the measurement of interrater agreement,” Psychological Bulletin, vol. 101, no. 1, pp. 140–146, 1987. View at Publisher · View at Google Scholar · View at Scopus
  20. W. D. Perreault and L. E. Leigh, “Reliability of nominal data based on qualitative judgments,” Journal of Marketing Research, vol. 26, pp. 135–148, 1989. View at Publisher · View at Google Scholar
  21. M. J. Warrens, “Conditional inequalities between Cohen's kappa and weighted kappas,” Statistical Methodology, vol. 10, pp. 14–22, 2013. View at Publisher · View at Google Scholar · View at MathSciNet
  22. M. J. Warrens, “Weighted kappa is higher than Cohen's kappa for tridiagonal agreement tables,” Statistical Methodology, vol. 8, no. 2, pp. 268–272, 2011. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  23. M. J. Warrens, “Cohen's quadratically weighted kappa is higher than linearly weighted kappa for tridiagonal agreement tables,” Statistical Methodology, vol. 9, no. 3, pp. 440–444, 2012. View at Publisher · View at Google Scholar · View at MathSciNet
  24. S. I. Anderson, A. M. Housley, P. A. Jones, J. Slattery, and J. D. Miller, “Glasgow outcome scale: an inter-rater reliability study,” Brain Injury, vol. 7, no. 4, pp. 309–317, 1993. View at Publisher · View at Google Scholar · View at Scopus
  25. C. S. Martin, N. K. Pollock, O. G. Bukstein, and K. G. Lynch, “Inter-rater reliability of the SCID alcohol and substance use disorders section among adolescents,” Drug and Alcohol Dependence, vol. 59, no. 2, pp. 173–176, 2000. View at Publisher · View at Google Scholar · View at Scopus
  26. R. L. Spitzer, J. Cohen, J. L. Fleiss, and J. Endicott, “Quantification of agreement in psychiatric diagnosis. A new approach,” Archives of General Psychiatry, vol. 17, no. 1, pp. 83–87, 1967. View at Publisher · View at Google Scholar · View at Scopus
  27. J. S. Simonoff, Analyzing Categorical Data, Springer, New York, NY, USA, 2003. View at MathSciNet
  28. P. E. Castle, A. T. Lorincz, I. Mielzynska-Lohnas et al., “Results of human papillomavirus DNA testing with the hybrid capture 2 assay are reproducible,” Journal of Clinical Microbiology, vol. 40, no. 3, pp. 1088–1090, 2002. View at Publisher · View at Google Scholar · View at Scopus
  29. D. Cicchetti and T. Allison, “A new procedure for assessing reliability of scoring EEG sleep recordings,” The American Journal of EEG Technology, vol. 11, pp. 101–110, 1971. View at Google Scholar
  30. D. V. Cicchetti, “A new measure of agreement between rank ordered variables,” in Proceedings of the Annual Convention of the American Psychological Association, vol. 7, pp. 17–18, 1972.
  31. M. J. Warrens, “Cohen's weighted kappa with additive weights,” Advances in Data Analysis and Classification, vol. 7, pp. 41–55, 2013. View at Google Scholar
  32. Y. M. M. Bishop, S. E. Fienberg, and P. W. Holland, Discrete Multivariate Analysis: Theory and Practice, The MIT Press, Cambridge, Mass, USA, 1975. View at MathSciNet
  33. J. L. Fleiss, J. Cohen, and B. S. Everitt, “Large sample standard errors of kappa and weighted kappa,” Psychological Bulletin, vol. 72, no. 5, pp. 323–327, 1969. View at Publisher · View at Google Scholar · View at Scopus
  34. M. J. Warrens, “Cohen's kappa can always be increased and decreased by combining categories,” Statistical Methodology, vol. 7, no. 6, pp. 673–677, 2010. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  35. M. J. Warrens, “Cohen's linearly weighted kappa is a weighted average,” Advances in Data Analysis and Classification, vol. 6, no. 1, pp. 67–79, 2012. View at Publisher · View at Google Scholar · View at MathSciNet
  36. D. V. Cicchetti and S. A. Sparrow, “Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior,” American Journal of Mental Deficiency, vol. 86, no. 2, pp. 127–137, 1981. View at Google Scholar · View at Scopus
  37. P. E. Crewson, “Reader agreement studies,” American Journal of Roentgenology, vol. 184, no. 5, pp. 1391–1397, 2005. View at Publisher · View at Google Scholar · View at Scopus
  38. J. R. Landis and G. G. Koch, “A one-way components of variance model for categorical data,” Biometrics, vol. 33, no. 4, pp. 159–174, 1977. View at Publisher · View at Google Scholar · View at MathSciNet