Journal of Healthcare Engineering

Journal of Healthcare Engineering / 2010 / Article

Research Article | Open Access

Volume 1 |Article ID 549082 |

Osnat Tourgeman-Bashkin, David Shinar, Yisrael Parmet, Ehud Zmora, "Inter-Observer Agreement Among Medical Professionals in Critical Care of Neonates and Children", Journal of Healthcare Engineering, vol. 1, Article ID 549082, 10 pages, 2010.

Inter-Observer Agreement Among Medical Professionals in Critical Care of Neonates and Children


Inter-observer agreement is essential to medical staff members and has a major effect on communication. The goal of the study was to examine the way medical professionals evaluate the potential severity of Almost Adverse Events (AAEs) that were observed in two intensive care units (ICUs). One hundred and fourteen AAEs were observed and recorded in both units by engineering students. Each AAE was rated independently by five senior medical staff members from each ICU, chosen by the unit manager, on a three- point severity level scale. Statistical analysis (K statistic and Cohen's Kappa) yielded relatively low levels of agreement among raters in both ICUs (< 0.3), but significantly greater agreement was found among nurses than among physicians in both ICUs. Low levels of agreement are attributed to the nature of work and characteristics of each ICU. Recommendations for improving agreements including forming shared mental models are specified.


  1. Institute of Medicine, Committee on Quality of Health Care in America, In: Kohn L.T., Corrigan J.M., Donaldson M.S., eds. To Err is Human Building a Safer Health System, Washington, DC: National Academy Press, 2000.
  2. M. S. Bogner, Human error in medicine, Lawrence Erlbaum Associates, Hillsdale, NJ, 1994.
  3. A. Theodossi, R. P. Knill-Jones, A. Skene et al., “Inter-observer variation of symptoms and signs in jaundice,” Liver, vol. 1, pp. 21–32, 1981. View at: Google Scholar
  4. K. W. Lindsay, G. M. Teasdale, and R. P. Knill-Jones, “Observer variability in assessing the clinical features of subarachnoid hemorrhage,” Journal of Neurosurgery, vol. 58, pp. 57–62, 1983. View at: Google Scholar
  5. F. Tomasello, F. Mariani, C. Fieschi et al., “Assessment of inter-observer differences in the Italian multicenter study on reversible cerebral ischemia,” Stroke, vol. 13, pp. 32–35, 1982. View at: Google Scholar
  6. J. S. Cowen and M. A. Kelly, “Errors and bias in using predictive scoring systems,” Critical Care Clinics, vol. 10, pp. 53–78, 1994. View at: Google Scholar
  7. D. Teres, “Comment on “The case for using objective scoring systems to predict intensive care outcome”,” Critical Care Clinics, vol. 10, pp. 91–92, 1994. View at: Google Scholar
  8. D. Teres and S. Lemeshow, “Why severity models should be used with caution,” Critical Care Clinics, vol. 10, pp. 93–110, 1994. View at: Google Scholar
  9. Y. Badihi, The Formal Structure of Thinking and Its Implications to Communication and Performance Errors, Research Thesis for the Degree of Doctor of Science. Technion, Israel. 1993.
  10. A. M. Joshua, D. S. Celermajer, and M. R. Stockler, “Beauty is in the eye of the examiner: reaching agreement about physical signs and their value,” Internal Medicine Journal, vol. 35, pp. 178–787, 2005. View at: Google Scholar
  11. A. F. Cook, Disagreement on Medical Error Reporting May Place Patients at Risk, New American Journal of Nursing Study. Journal to Convene Special Symposia to Review and Address Challenges, Advancing Nursing Practice Excellence: State of the Science, 2004.
  12. O. Tourgeman-Bashkin, D. Shinar, and E. Zmora, “causes of near misses in critical care of neonates and children,” Acta Pediatrica, vol. 97, pp. 299–303, 2008. View at: Google Scholar
  13. J. L. Fleiss, “Measuring nominal scale agreement among many raters,” Psychological Bulletin, vol. 76, pp. 378–382, 1971. View at: Google Scholar
  14. E. J. Thomas, J. B. Sexton, and R. L. Helmreich, “Discrepant attitudes about teamwork among critical care nurses and physicians,” Critical Care Medicine, vol. 31, no. 3, pp. 956–959, 2003. View at: Google Scholar
  15. W. D. Croser, “The contemporary nurse-physician relationship: Insights from scholars outside the two professions,” Nurse Outlook, vol. 48, pp. 263–268, 2000. View at: Google Scholar
  16. E. B. Raftery and W. W. Holland, “Examination of the Heart: an investigation into variation,” American Journal of Epidemiology, vol. 85, pp. 438–44, 1967. View at: Google Scholar
  17. F. Tomasello, F. Mariani, C. Fieschi, C. Argentino, G. Bono, L. De Zanche et al., “Assessment of interobserver differences in the Italian multicenter study on reversible cerebral ischemia,” Stroke, vol. 13, pp. 32–5, 1982. View at: Google Scholar
  18. W. B. Rouse and N. M. Morris, “On looking into the black box: prospects and limits in the search for mental models,” Psychological bulletin, vol. 100, pp. 349–363, 1986. View at: Google Scholar
  19. R. J. Stout, J. A. Cannon-Bowers, E. Salas, and D. M. Milanovich, “Planning, Shared Mental Models, and Coordinated Performance: An Empirical Link Is Established,” Human Factors, vol. 41, pp. 61–71, 1999. View at: Google Scholar
  20. D. P. Minionis, S. J. Zaccaro, and R. Perez, “Shared mental models, team coordination, and team performance,” in Paper presented at the 10th Annual Conference of the Society for Industrial/Organizational Psychology, Orlando, FL, 1995. View at: Google Scholar

Copyright © 2010 Hindawi Publishing Corporation. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

More related articles

 PDF Download Citation Citation
 Order printed copiesOrder

Related articles

Article of the Year Award: Outstanding research contributions of 2020, as selected by our Chief Editors. Read the winning articles.