Table of Contents Author Guidelines Submit a Manuscript
Journal of Mathematics
Volume 2014 (2014), Article ID 203907, 9 pages
http://dx.doi.org/10.1155/2014/203907
Research Article

New Interpretations of Cohen’s Kappa

Institute of Psychology, Unit Methodology and Statistics, Leiden University, P.O. Box 9555, 2300 RB Leiden, The Netherlands

Received 31 May 2014; Accepted 19 August 2014; Published 3 September 2014

Academic Editor: Yuehua Wu

Copyright © 2014 Matthijs J. Warrens. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. L. M. Hsu and R. Field, “Interrater agreement measures: comments on Kappan, Cohen's Kappa, Scott's π, and Aickin's α,” Understanding Statistics, vol. 2, pp. 205–219, 2003. View at Google Scholar
  2. K. Krippendorff, “Reliability in content analysis: some common misconceptions and recommendations,” Human Communication Research, vol. 30, no. 3, pp. 411–433, 2004. View at Publisher · View at Google Scholar · View at Scopus
  3. M. J. Warrens, “Inequalities between kappa and kappa-like statistics for K x K tables,” Psychometrika, vol. 75, no. 1, pp. 176–185, 2010. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  4. J. Cohen, “A coefficient of agreement for nominal scales,” Educational and Psychological Measurement, vol. 20, pp. 37–46, 1960. View at Publisher · View at Google Scholar
  5. J. A. Hanley, “Standard error of the kappa statistic,” Psychological Bulletin, vol. 102, no. 2, pp. 315–321, 1987. View at Publisher · View at Google Scholar · View at Scopus
  6. M. Maclure and W. C. Willett, “Misinterpretation and misuse of the Kappa statistic,” American Journal of Epidemiology, vol. 126, no. 2, pp. 161–169, 1987. View at Google Scholar · View at Scopus
  7. M. J. Warrens, “On the equivalence of Cohen's kappa and the Hubert-Arabie adjusted Rand index,” Journal of Classification, vol. 25, no. 2, pp. 177–183, 2008. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  8. M. J. Warrens, “Cohen's kappa can always be increased and decreased by combining categories,” Statistical Methodology, vol. 7, no. 6, pp. 673–677, 2010. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet · View at Scopus
  9. W. A. Scott, “Reliability of content analysis: the case of nominal scale coding,” Public Opinion Quarterly, vol. 19, no. 3, pp. 321–325, 1955. View at Publisher · View at Google Scholar · View at Scopus
  10. K. Krippendorff, Content Analysis: An Introduction to Its Methodology, Sage, Thousands Oaks, Calif, USA, 2nd edition, 2004.
  11. K. Krippendorff, “Association, agreement, and equity,” Quality and Quantity, vol. 21, no. 2, pp. 109–123, 1987. View at Publisher · View at Google Scholar · View at Scopus
  12. J. L. Fleiss, “Measuring agreement between two judges on the presence or absence of a trait,” Biometrics, vol. 31, no. 3, pp. 651–659, 1975. View at Google Scholar · View at MathSciNet · View at Scopus
  13. J. L. Fleiss, Statistical Methods for Rates and Proportions, Wiley, New York, NY, USA, 1981. View at MathSciNet
  14. J. L. Fleiss, B. Levin, and M. C. Paik, Statistical Methods for Rates and Proportions, John Wiley & Sons, New York, NY, USA, 3rd edition, 2003. View at Publisher · View at Google Scholar · View at MathSciNet
  15. H. C. Kraemer, “Ramifications of a population model for κ as a coefficient of reliability,” Psychometrika, vol. 44, no. 4, pp. 461–472, 1979. View at Publisher · View at Google Scholar · View at Scopus
  16. S. Vanbelle and A. Albert, “Agreement between two independent groups of raters,” Psychometrika, vol. 74, no. 3, pp. 477–491, 2009. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  17. M. J. Warrens, “Cohen's kappa is a weighted average,” Statistical Methodology, vol. 8, no. 6, pp. 473–484, 2011. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  18. H. C. Kraemer, V. S. Periyakoil, and A. Noda, “Kappa coefficients in medical research,” Statistics in Medicine, vol. 21, no. 14, pp. 2109–2129, 2002. View at Publisher · View at Google Scholar · View at Scopus
  19. A. Agresti, “Modelling patterns of agreement and disagreement,” Statistical Methods in Medical Research, vol. 1, no. 2, pp. 201–218, 1992. View at Publisher · View at Google Scholar · View at Scopus
  20. S. Fennig, T. J. Craig, M. Tanenberg-Karant, and E. J. Bromet, “Comparison of facility and research diagnoses in first-admission psychotic patients,” The American Journal of Psychiatry, vol. 151, no. 10, pp. 1423–1429, 1994. View at Google Scholar · View at Scopus
  21. F. B. Baulieu, “A classification of presence/absence based dissimilarity coefficients,” Journal of Classification, vol. 6, no. 2, pp. 233–246, 1989. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  22. M. J. Warrens, “On similarity coefficients for 2×2 tables and correction for chance,” Psychometrika, vol. 73, no. 3, pp. 487–502, 2008. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  23. M. J. Warrens, “On association coefficients for 2×2 tables and properties that do not depend on the marginal distributions,” Psychometrika, vol. 73, no. 4, pp. 777–789, 2008. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet · View at Scopus
  24. M. J. Warrens, “Chance-corrected measures for 2 × 2 tables that coincide with weighted kappa,” British Journal of Mathematical and Statistical Psychology, vol. 64, no. 2, pp. 355–365, 2011. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  25. L. R. Dice, “Measures of the amount of ecologic association between species,” Ecology, vol. 26, pp. 297–302, 1945. View at Publisher · View at Google Scholar
  26. D. A. Bloch and H. C. Kraemer, “2 x 2 kappa coefficients: measures of agreement or association,” Biometrics, vol. 45, no. 1, pp. 269–287, 1989. View at Publisher · View at Google Scholar · View at Scopus
  27. J. Cohen, “Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit,” Psychological Bulletin, vol. 70, no. 4, pp. 213–220, 1968. View at Publisher · View at Google Scholar · View at Scopus
  28. M. J. Warrens, “Some paradoxical results for the quadratically weighted kappa,” Psychometrika, vol. 77, no. 2, pp. 315–323, 2012. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet · View at Scopus
  29. M. J. Warrens, “Equivalences of weighted kappas for multiple raters,” Statistical Methodology, vol. 9, no. 3, pp. 407–422, 2012. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  30. M. J. Warrens, “Conditional inequalities between Cohen's kappa and weighted kappas,” Statistical Methodology, vol. 10, pp. 14–22, 2013. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  31. J. S. Coleman, “Measures of concordance or consensus between members of social groups,” Johns Hopkins University, 1966.
  32. R. J. Light, “Measures of response agreement for qualitative data: some generalizations and alternatives,” Psychological Bulletin, vol. 76, no. 5, pp. 365–377, 1971. View at Publisher · View at Google Scholar · View at Scopus
  33. Y. M. M. Bishop, S. E. Fienberg, and P. W. Holland, Discrete Multivariate Analysis: Theory and Practice, MIT Press, Cambridge, UK, 1975.
  34. A. Agresti, Categorical Data Analysis, Wiley Series in Probability and Statistics, Wiley-Interscience, Hoboken, NJ, USA, 2nd edition, 2002. View at Publisher · View at Google Scholar · View at MathSciNet
  35. A. N. Albatineh, M. Niewiadomska-Bugaj, and D. Mihalko, “On similarity indices and correction for chance agreement,” Journal of Classification, vol. 23, no. 2, pp. 301–313, 2006. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  36. J. L. Fleiss, J. Cohen, and B. S. Everitt, “Large sample standard errors of kappa and weighted kappa,” Psychological Bulletin, vol. 72, no. 5, pp. 323–327, 1969. View at Publisher · View at Google Scholar · View at Scopus