Abstract
Histopathological reports frequently contain phrases describing the degree of uncertainty of the diagnosis. We examined the interpretations of such terms by cellular pathologists, other doctors, and medical students. 203 respondents estimated the degree of certainty they would associate with the following phrases in a cellular pathology report: the features are indicative of; raise the possibility of; are compatible with; are probably those of; are diagnostic of; are in keeping with; and are suggestive of. For all phrases assessed other than “diagnostic of”, all groups showed a wide spread in the interpreted probability. For example, the probability associated with the term “in keeping with” by individual consultant pathologists ranged from 25 to 100%. This study demonstrates that pathologists vary widely in how they interpret the meaning of phrases describing probability that are commonly used in pathology reports. We suggest that this potential risk is highlighted during pathology training.
1. Introduction
Uncertainty is inherent in medicine, and communicating the degree of uncertainty in diagnosis and prognosis is an important part of professional practice. For example, an analysis of recordings of 216 consultants between clinician and patient in a university-affiliated American general medical clinic showed that 71% of consultations included expressions of uncertainty [1].
The importance of doctor-to-doctor communication regarding the extent of uncertainty in a diagnosis is no less important than that between clinician and patient. Misinterpretation of the level of diagnostic certainty may lead to inappropriate treatment, a failure to uncover alternative diagnoses, and provision of incorrect prognostic information. Given the invasive nature of some investigations, misinterpretation of the certainty of a diagnosis could lead to an unnecessary investigation with serious consequences.
There is evidence that children and adults when asked on several occasions are reasonably consistent in their interpretation of the level of interpreted probability for different expressions of uncertainty; however, there is wide variation in such interpretations between observers [2].
We aimed with this study to determine whether the wide range of interpretation of such terms by the general public is also reflected in the interpretations by cellular pathologists and also whether there were differences between consultant and trainee cellular pathologists and medical students.
2. Methods and Materials
A survey was carried out, in which the respondents were asked to estimate (in the context of a cellular pathology report) the “degree of certainty you would associate with the following terms (as a percentage chance that the suggested diagnosis is the disease the patient actually has):
the features are indicative of …
raise the possibility of …
are compatible with …
are probably those of …
are diagnostic of …
are in keeping with …
are suggestive of …”.
The questionnaire was part of an anonymous study relating to postgraduate pathology education, which had National Health Service research ethics committee and Royal Free Hampstead NHS Trust Research and Development Department approval. Four groups were surveyed: consultant cellular pathologists, trainee cellular pathologists, medical doctors (consultants and trainees) who are not cellular pathologists, and medical students (3rd and 4th year medical students at University College London Medical School). All doctors surveyed were practicing in the United Kingdom at the time of the study.
The survey was conducted partly online (with invitations to undertake the survey included within the email newsletter of the Royal College of Pathologists and also sent by email to personal contacts in the UK). Approximately half of responses (predominantly those provided by medical students) were obtained in a paper format, rather than the online version.
A total of 203 responses were received (61 cellular pathology consultants, 28 trainee cellular pathologists, 73 medical students, 41 clinicians). Statistical analysis was by Student’s -test. The study had ethical and local R + D approval.
It is acknowledged that this method of sampling has limitations, including potential sample bias and response bias, and also that it is impossible to reliably assess the response rate to the survey; however according to the Royal College of Pathologists workforce department, in 2007 there were 1448 consultant cellular pathologists and 555 trainees in the United Kingdom (personal communication). Therefore it can be estimated that responses were obtained from approximately 4% of the consultant cellular pathologists and 5% of the trainee pathologists practicing in the United Kingdom.
3. Results and Discussion
For analysis the respondents were divided into the following groups:Group 1: “cellular pathology consultants,”Group 2: “cellular pathology trainees,”Group 3: “medical students” (from years 3 and 4 at UCL),Group 4: “clinicians” (doctors other than cellular pathologists, including other types of pathologists such as chemical pathologists and microbiologists),Group 5: “all participants” (aggregate group consisting of groups ),Group 6: “pathologists” (consisting of groups ),Group 7: “all doctors” (consisting of groups ),Group 8: “nonpathologists” (consisting of group , including pathologists who are not cellular pathologists).
We demonstrate that all groups show marked variability between individuals in the probability they associate with terms frequently used to describe levels of uncertainty in a diagnosis (other than “diagnostic of”). For example, the probabilities associated with the term “in keeping with” by individual consultant pathologists ranged from 25 to 100%.
The variations in the levels of certainty understood by the different groups from the different phrases is shown graphically by boxplots (Figure 1) and is quantified by the standard deviations from the means (Table 1, figures to 2 significant figures). The standard deviations are therefore used as a measure of ambiguity, with a lower value indicating less ambiguity of the corresponding phrase, that is, the phrase is better at meaning the same thing to different people in that group.
The standard deviations for the terms for all doctors group (group 7) were as follows: “raise the possibility of” 18.0, “suggestive of” 17.7, “compatible with” 17.5, “probably those of” 13.2, “in keeping with” 16.1, “indicative of” 12.0, and “diagnostic of” 5.2. This demonstrates a substantial variation in the interpretation by doctors of the terms other than “diagnostic of”.
The median figures for the interpreted probability by pathologists (group 6) were as follows: “diagnostic of” 98.4%, “indicative of” 92.1%, “in keeping with” 82.1%, “probably those of” 75.6%, “compatible with” 74.3%, “suggestive of” 69.8%, and “raises the possibility of” 52.9%. The consultant pathologists, trainee pathologists, and other doctors showed similar median interpreted probabilities for the phrases. When comparing the results from the pathologists (group 6) and the other doctors (group 4), only the term “indicative of” showed a significant difference; however it is acknowledged that the small sample size of clinicians in the study may preclude reliable assessment for this group.
Generally medical students associated most terms with a lower interpreted probability than the other groups (Figure 2). Comparing the medical students with all doctors (group 7), there were statistically significant differences for the terms “compatible with,” “in keeping with,” “indicative of,” “and diagnostic of” at the level, and “suggestive of” and “probably” at the level. The only term not found to be associated with a significant difference in this comparison was “raise the possibility of”.
When comparing the phrases themselves, we see that most are approximately equally ambiguous, with standard deviations falling between 16 and 19. The exception is the phrase “The features are diagnostic of …”, which has a standard deviation of 8.4. This phrase can therefore be said to stand out as better at meaning the same thing to different people. However, we also see that this phrase has a standard deviation of 11 when interpreted by medical students, which is relatively high compared to the standard deviations for the other groups.
It is acknowledged that there is a potential ambiguity in the question. Respondents were asked to estimate “as a percentage chance that the suggested diagnosis is the disease the patient actually has”. Respondents might be providing their estimate of the percentage certainty intended by the pathologist, or the percentage certainty that the patient has the disease, given both the uncertainty of the pathologist and the possibility that the diagnosis of the pathologist might be erroneous.
These results demonstrate that individual pathologists and clinicians vary widely in their interpretation of the implied probability associated with terms used in pathology reports to describe the degree of uncertainty in a diagnosis.
The significant difference in the interpretation of the phrases by medical students compared with other doctors was unexpected. We are not aware of any previous literature documenting this.
Every day clinicians are making decisions about patient treatment on the basis of their interpretation of cellular pathology reports. An understanding of the degree of uncertainty of a diagnosis is critical for safe patient management; for example, if a patient has a 99.9% chance of having a glioblastoma, few would consider it reasonable to reoperate to increase the level of certainty to 100%. In contrast few would be comfortable to undergo chemotherapy if their diagnosis was only 51% certain. This study demonstrates that there is great variation in the interpretation of the degree of certainty implied by terms describing the level of uncertainty in routine use in pathology reports.
Given the potential impact of this ambiguity on patient care, it is perhaps surprising that this topic has not received more attention in the literature. Attanoos and colleagues [3] studied the phraseology used in 300 randomly selected surgical reports from the University Hospital Wales. They found that the terms “diagnostic of,” “that of,” “show,” “characteristic of,” “indicative of,” “represent,” “in keeping with,” “consistent with,” “highly suggestive of,” “favour,” “suggestive of,” “suspicious of,” and “reminiscent of” were the terms most frequently used to describe the level of certainty of a diagnosis.
Attanoos et al. provided a questionnaire to 20 pathologists and 20 surgeons, asking them to score the certainty implied by 13 descriptive phrases, on a scale of 0 (total uncertainty) to 5 (total certainty) [3]. The pathologists were also asked to estimate how often they used these phrases, and the surgeons asked whether they liked or disliked the use of such phrases in reports. They reported that the phrase “diagnostic of” was considered to convey more diagnostic certainty by pathologists than surgeons. They also noted that terms frequently used by pathologists, including “in keeping with,” “consistent with,” “those of,” and “suggestive of” were disliked by many surgeons.
There may be understandable reasons why there are differences in preference for such terms between surgeons and pathologists. Where possible, clinicians are likely to prefer unambiguous diagnoses on which to base management decisions. Pathologists may use descriptors of uncertainty to accurately reflect the level of uncertainty in a diagnosis; however the possibility has been raised that pathologists might also introduce an element of uncertainty into their reporting to minimise their own personal legal risk in relation to misdiagnosis [3]. In Christopher and Hotz’s study of veterinary pathologists, one respondent explicitly stated that because of the legal implications of a diagnosis “I most often say “compatible with …” even if I am 100% sure of the diagnosis” [4]. If the pathologist is certain but does not convey this in the report in a way interpretable by clinicians, this may do a disservice to the patient, who may undergo unnecessary further procedures. This is a potential example of where defensive medicine may be against the interests of patient care.
The discussion in the paper by Attanoos et al. gives the impression that words and phrases to have exact definitions, which can be determined with reference to a dictionary, and their use judged to be correct or incorrect [3]. For example, they state that “six of the 13 descriptive phrases (“diagnostic of,” “that of,” “show,” “characteristic of,” “indicative of,” and “represent”) are semantically definitive terms which should only be used to communicate total certainty”. In contrast, we would argue that language is an evolving process [5], largely defined by its usage, and reference to nonmedical dictionary definitions in isolation does not necessarily determine an objectively correct usage. There are many different dictionaries, and each word typically has several meanings. For example, “indicative of” is a term which we would consider in our own practice to be suggesting at least a very high level of certainty in a diagnosis. One dictionary definition (the popular online dictionary.com) gives five synonyms for “indicative of,” one of which is “suggestive” [6]. As can be seen in both this study and in the study by Attanoos et al. [3], “suggestive of” is associated with a large range of variation in interpreted certainty. If it were to be argued that a dictionary definition alone determines the degree of certainty to be associated with a term, then the same degree of certainty should be associated with its synonyms.
Despite the differences in the theoretical perspective on linguistics, the results found in the study by Attanoos et al. and our study are broadly consistent—there are several phrases routinely used in cellular pathology diagnostic practice, the interpretation of which is subject to large variation between individuals.
This conclusion is also supported by data from a study of veterinary pathologists [4]. 96 veterinary pathologists reported the use of 68 unique terms to describe level of uncertainty in a diagnosis in cytology reports. There was no statistically significant difference in the numerical interpretation of 10 of the 18 terms investigated in the study.
Nakao and Axelrod [7] examined the interpretation of the frequency associated with a term by 103 physicians and 106 nonphysicians (although their study was not specifically examining the use of these terms in pathological reports). They considered the range of the interpretation of these terms by physicians and nonphysicians to be so great as to make their use in medical practice inappropriate and recommend the use of numerical descriptions.
We are not necessarily saying that phrases to describe uncertainty are abandoned in favour of numerical probabilities. There may be problems in the translation of an individual’s impression of the meaning of a description of uncertainty into percentage terms for several reasons. Firstly, a pathologist may have a feeling that their preferred diagnosis is not certain, but they do not directly see a numerical probability under the microscope. If they translate this subjective feeling of uncertainty into a numerical probability, there may be some improvement in mutual understanding between clinician and pathologist; however we do not know how reliable the translation by the pathologist of subjective uncertainty into numerical probability is. We may risk exchanging vague communication for spurious precision.
Secondly, it is important to appreciate that verbal descriptions of uncertainty are not just vague approximations of numerical descriptions. They convey information additional to the numerical. The phrase used may reveal (or be interpreted as revealing) the attitudes of the pathologist towards consequences of a diagnosis. Telgen and Brun have classified verbal descriptions of probability into those that imply a positive or a negative frame of [8]. This is separate to the issue of whether they are associated with a high or low interpreted level of certainty. For example, the term “possible” may refer to a probability below 50%; however it draws to attention the plausibility of the considered diagnosis, so would be considered a positive term. The term “not entirely certain” is likely to be interpreted as a high probability of the considered diagnosis, but the pathologist is drawing attention to the plausibility of the diagnosis being incorrect, so would be considered a negative term. They illustrate this distinction with the suggestion that if a phrase is followed by “because,” whether the arguments that typically follow would be reasons to support or refute the diagnosis demonstrate whether the term is positive or negative.
They demonstrated in studies with university students that even when the participants consider a positive and negative descriptor of uncertainty to be associated with a similar predicted probability, the positive term and negative terms are interpreted as having significantly different meanings [8]. For example, positive terms used to describe the likelihood of a person carrying out an action are interpreted as signifying greater likelihood of the action occurring than numerically equivalent negative terms.
Positive and negative terms may also be seen as carrying different meanings regarding which choice is most appropriate. For example a patient is unlikely to interpret “there is a chance that the treatment will cure your cancer” as being equivalent to “it is unlikely that the treatment will cure your cancer”, even though they may be mathematically equivalent. The former would probably be interpreted as a recommendation to take the treatment, and the latter a discouragement.
It is unlikely (although not directly investigated in this study) that similar framing effects occur in cellular pathology reports, whether intended or not by the pathologist. To take an example, one of the authors of this paper is a diagnostic neuropathologist (MG). Imagine a brain biopsy which I believe has a 99.9% chance of being a glioblastoma but to have a 0.1% chance of being an alternative diagnosis (such as a nonneoplastic mimic of neoplasia). The following statements would both be logically consistent.(1)The features very strongly favour glioblastoma.(2)The diagnosis of glioblastoma is not entirely certain.
Adding because at the end of the first phrase could result in a sentence such as “the features very strongly favour glioblastoma because there is a diffuse infiltrate of mitotically active atypical astrocytes, vascular endothelial hyperplasia, and necrosis” (i.e., the arguments following “because” are all supportive of the diagnosis). A similar process with the second phrase could generate a sentence such as “the diagnosis of glioblastoma is not entirely certain because non-neoplastic lesions may be associated with necrosis, reactive astrocytes may mimic neoplastic astrocytes, mitotic activity may be seen in inflammatory conditions, and vascular endothelial hyperplasia is not exclusively seen in glioblastoma”. Even if the numerical data were to be included in the sentences, it still would not make sense to say “the diagnosis of glioblastoma (99% probability) is not entirely certain because there is a diffuse infiltrate of mitotically active atypical astrocytes, vascular endothelial hyperplasia, and necrosis,” because the term describing the probability of the diagnosis is a negative term. I may intend, or the clinician may interpret statement 1 (“the features very strongly favour glioblastoma”) as implying that no further diagnostic investigation is required. Statement 2 is more likely to lead the surgeon to at least consider rebiopsy.
It is also important to be aware that the interpretation of the meaning of probabilistic terms varies depending on the context, in particular in relation to the likely consequences of the outcome. Mazur and Merz, for example, have described how the probability patients associated with a “rare” complication depends not only on the severity of the complication but also on the age of the patient and previous experience of the complication [9].
When recommending changing of standard practices, the risk of unintended consequences should always be considered. Nakao and Axelrod’s suggestion that numerical descriptions of certainty should be used rather than verbal descriptions may lead to different problems in communication. In many situations the level of probability cannot be ascertained from the literature, as study populations often do not match those encountered in routine practice. It is unlikely that many pathologists or clinicians categorise the situation for individual patients in terms of numerical percentage of likelihood of differential diagnoses but instead rely on a “gut feeling” of the degree of certainty. Christopher and Hotz found in their study that 96.8% of the veterinary pathologists they studied preferred to use words rather than numbers to express probability in cytology reports, and there is evidence that people are prone to cognitive errors in the expression of their own confidence [4]. Providing a number may also give an impression of spurious precision (i.e., if the level of uncertainty is uncertain, it could be argued that a vague term may be a more accurate reflection of this).
4. Conclusions
This study demonstrates that there is wide variation in the interpretation by individual pathologists, medical students, and other doctors of phrases routinely used in pathology practice to describe the degree of uncertainty of a diagnosis.
We would argue that users and providers of pathology services should be aware that the interpretation of the degree of certainty of a diagnosis by the clinician may bear little relation to the degree of certainty intended to be conveyed by the pathologist. Where the pathologist considers the diagnosis to be as certain as is reasonable for a diagnostic test to be, it is recommended that terms that are known to be interpreted by most pathologists and clinicians as conveying a high degree of certainty with relatively low spread of interpretations are used. In our study only the phrase “diagnostic of” met these criteria.
Attanoos et al. suggested that national guidelines on phraseology in pathology reporting may be required. There is an inevitable tension between individuality in pathology reporting and standardisation [10]. There are both logistical and theoretical reasons why ascribing a specific probability to phrases describing uncertainty may be difficult or potentially counterproductive; however if there were to be sufficient support from the pathological community, it could be plausible to develop guidelines that define the meaning of such terms when used in the context of a pathology report. If such terms were then used consistently, and the clinicians receiving pathology reports were aware of the specific defined meaning of such terms in the context of the report, then this could contribute to a reduction in the risk of misunderstanding. Further research using larger sample sizes and a wider variety of phrases would be helpful if considering such a process. Although it may be difficult to impose guidelines on the interpretation of the meaning of phrases used to describe uncertainty in pathology reports, we suggest at least that when verbal descriptions of uncertainty are used in reports, the potential for miscommunication is considered by both pathologists and clinicians.
Acknowledgments
The authors would like to thank Tarek Hasan for his statistical advice. Preliminary data from this study has been presented at the British Neuropathological Society winter meeting 2010.