Table of Contents Author Guidelines Submit a Manuscript
Journal of Probability and Statistics
Volume 2015 (2015), Article ID 751803, 8 pages
http://dx.doi.org/10.1155/2015/751803
Research Article

Measurement of Interobserver Disagreement: Correction of Cohen’s Kappa for Negative Values

Departments of Mechanical Engineering and Industrial & Systems Engineering, University of Minnesota, Minneapolis, MN 55455, USA

Received 24 June 2015; Accepted 6 September 2015

Academic Editor: Z. D. Bai

Copyright © 2015 Tarald O. Kvålseth. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. J. Cohen, “A coefficient of agreement for nominal scales,” Educational and Psychological Measurement, vol. 20, no. 1, pp. 37–46, 1960. View at Publisher · View at Google Scholar
  2. J. L. Fleiss, B. Levin, and M. C. Paik, Statistical Methods for Rates and Proportions, Wiley, Hoboken, NJ, USA, 3rd edition, 2003.
  3. K. L. Gwet, Handbook of Inter-Rater Reliability, Advanced Analytics, Gaithersburg, Md, USA, 4th edition, 2014.
  4. M. M. Shoukri, Measures of Interobserver Agreement and Reliability, CRC Press, Boca Raton, Fla, USA, 2nd edition, 2011.
  5. A. Von Eye and E. Y. Mun, Analyzing Rater Agreement: Manifest Variable Methods, Lawrence Erlbaum Associates, Mahwah, NJ, USA, 2005.
  6. M. J. Warrens, “New interpretations of Cohen's kappa,” Journal of Mathematics, vol. 2014, Article ID 203907, 9 pages, 2014. View at Publisher · View at Google Scholar · View at MathSciNet
  7. M. J. Warrens, “Five ways to look at Cohen's kappa,” Journal of Psychology & Psychotherapy, vol. 5, no. 4, pp. 1–4, 2015. View at Publisher · View at Google Scholar
  8. Z. Yang and M. Zhou, “Kappa statistic for clustered matched-pair data,” Statistics in Medicine, vol. 33, no. 15, pp. 2612–2633, 2014. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  9. M. L. McHugh, “Interrater reliability: the kappa statistic,” Biochemia Medica, vol. 22, no. 3, pp. 276–282, 2012. View at Google Scholar · View at Scopus
  10. H. L. Kundel and M. Polansky, “Measurement of observer agreement,” Radiology, vol. 228, no. 2, pp. 303–308, 2003. View at Publisher · View at Google Scholar · View at Scopus
  11. R. L. Spitzer, J. Cohen, J. L. Fleiss, and J. Endicott, “Quantification of agreement in psychiatric diagnosis,” Archives of General Psychiatry, vol. 17, no. 1, pp. 83–87, 1967. View at Publisher · View at Google Scholar · View at Scopus
  12. T. O. Kvålseth, “Note on Cohen's kappa,” Psychological Reports, vol. 65, no. 1, pp. 223–226, 1989. View at Publisher · View at Google Scholar
  13. J. Cohen, “Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit,” Psychological Bulletin, vol. 70, no. 4, pp. 213–220, 1968. View at Publisher · View at Google Scholar · View at Scopus
  14. D. V. Cicchetti and T. Allison, “A new procedure for assessing reliability of scoring EEG sleep recordings,” American Journal of EEG Technology, vol. 11, no. 3, pp. 101–109, 1971. View at Publisher · View at Google Scholar
  15. T. O. Kvålseth, “Weighted specific-category Kappa measure of interobserver agreement,” Psychological Reports, vol. 93, no. 3, pp. 1283–1290, 2003. View at Publisher · View at Google Scholar · View at Scopus
  16. T. O. Kvålseth, “Kappa coefficients of agreement,” in International Encyclopedia of Statistical Science, M. Lovric, Ed., part 11, pp. 710–713, Springer, Berlin, Germany, 2011. View at Google Scholar
  17. H. E. A. Tinsley and D. J. Weiss, “Interrater reliability and agreement,” in Handbook of Applied Multivariate Statistics and Mathematical Modeling, H. E. A. Tinsley and S. D. Brown, Eds., pp. 95–124, Academic Press, San Diego, Calif, USA, 2000. View at Google Scholar
  18. K. F. Hirji and M. H. Rosove, “A note on interrater agreement,” Statistics in Medicine, vol. 9, no. 7, pp. 835–839, 1990. View at Publisher · View at Google Scholar · View at Scopus
  19. A. Agresti, Categorical Data Analysis, John Wiley & Sons, Hoboken, NJ, USA, 2nd edition, 2002. View at Publisher · View at Google Scholar · View at MathSciNet
  20. A. C. Davison and D. V. Hinkley, Bootstrap Methods and Their Applications, Cambridge University Press, Cambridge, UK, 1997. View at Publisher · View at Google Scholar · View at MathSciNet
  21. W. C. Parr and H. D. Tolley, “Jackknifing in categorical data analysis,” The Australian Journal of Statistics, vol. 24, no. 1, pp. 67–79, 1982. View at Publisher · View at Google Scholar · View at MathSciNet
  22. J. L. Fleiss, J. Cohen, and B. S. Everitt, “Large sample standard errors of kappa and weighted kappa,” Psychological Bulletin, vol. 72, no. 5, pp. 323–327, 1969. View at Publisher · View at Google Scholar · View at Scopus