Abstract
Background. We sought to develop diagnostic test guidance definitions for pediatric enteric infections to facilitate the interpretation of positive test results in the era of multianalyte molecular diagnostic test platforms. Methods. We employed a systematic, two-phase, modified Delphi consensus process consisting of three web-based surveys and an expert panel face-to-face meeting. In phase 1, we surveyed an advisory panel of North American experts to select pathogens requiring diagnostic test guidance definition development. In phase 2, we convened a 14-member expert panel to develop, refine, and select the final definitions through two web-based questionnaires interspersed with a face-to-face meeting. Both questionnaires asked panelists to rate the degree to which they agreed that if the definition is met the pathogen is likely to be causative of clinical illness. Results. The advisory panel survey identified 19 pathogens requiring definitions. In the expert panel premeeting survey, 13 of the 19 definitions evaluated were rated as being highly likely (“agree” or “strongly agree”) to be responsible for acute gastroenteritis symptoms by ≥67% of respondent panel members. The definitions for the remaining six pathogens (Aeromonas, Clostridium difficile, Edwardsiella, nonenteric adenovirus, astrovirus, and Entamoeba histolytica) were indeterminate. After the expert panel meeting, only two of the modified definitions, C. difficile and E. histolytica/dispar, failed to achieve the a priori specified threshold of ≥67% agreement. Conclusions. We developed diagnostic test guidance definitions to assist healthcare providers for 17 enteric pathogens. We identified two pathogens that require further research and definition development.
1. Introduction
Diarrheal disease is the second leading cause of child mortality worldwide [1, 2] and is among the most common causes of illness globally, with over 4 billion episodes occurring annually [3–5]. In the United States, there are 178.8 million episodes of diarrhea contributing to 473,832 hospitalizations each year [6]. Infection by a variety of gastrointestinal pathogens, including parasites, bacteria, and viruses, can cause vomiting and diarrhea that typifies acute gastroenteritis. Multiple diagnostic testing methods are available to identify enteric pathogens including traditional culture techniques for enteric bacteria, direct microscopy for parasites, enzyme immunoassay (EIA) for select enteric viruses, bacterial toxins and some parasites, and molecular methods for specific targets. Despite the availability of multiple modalities to identify enteric pathogens, the etiology often remains unidentified. However, recent practice changes including the collection of rectal swabs rather than stool specimens [7–9] to increase specimen acquisition rates, and the increasing use of molecular methods, especially syndromic panels which detect multiple pathogens simultaneously [10–12] have significantly increased the detection rates of enteric pathogens in symptomatic individuals [13].
Existing guidelines on the diagnosis [14] and management [15] of enteric infectious diseases do not adequately consider the implications of recent advances in sample collection or molecular diagnostic techniques. For example, recently published adult guidelines by the American College of Gastroenterology suggest that the best applicability for molecular diagnostic tests is for the clinician in practice, but do not provide guidance on how clinicians should interpret the results of such testing [16]. The implementation of commercial multiplex gastrointestinal pathogen panels presents practical challenges to healthcare providers in assessing the clinical relevance of positive results and to public health officials who are responsible for surveillance and follow-up of reportable infectious agents. Positive results generated by molecular assays cannot distinguish between asymptomatic carriage, subclinical infection, active disease, resolved disease, or the presence of target nucleic acids liberated by nonpathogens [16]. High rates of enteric pathogen detection have been reported in asymptomatic children [17]. Commercially available multianalyte panels also identify organisms that were not previously routinely tested for such as non-O157 Shiga toxin-producing Escherichia coli (STEC). These challenges highlight the need for the use of standardized definitions to guide the interpretation of multiplex polymerase chain reaction (PCR) results by clinicians who often have limited knowledge of laboratory testing procedures. While definitions do exist for some enteric pathogens in the context of public health and outbreak management, no specific definitions exist to guide interpretation and management by healthcare providers. The latter are important because accurate diagnosis by front-line clinicians is crucial to identify those patients who would benefit from antimicrobial therapy and those where unnecessary antibiotics may cause harm [15].
Given the existing knowledge gap, and the paucity of high-quality evidence supporting the clinical interpretation of multianalyte molecular diagnostic test platforms, we conducted a modified Delphi consensus process to develop provisional diagnostic test guidance definitions for pediatric enteric infections to guide the interpretation, by clinicians, of positive diagnostic test results. The goal was to provide guidance for clinicians providing care to children up to 18 years of age with symptoms of acute gastroenteritis defined as a decrease in stool consistency and/or by the presence of three or more episodes of stool or vomiting in a 24-hour period with duration of symptoms of less than seven days [18, 19].
2. Materials and Methods
The approach consisted of a systematic, two-phase, modified Delphi consensus methodology with three web-based surveys and an expert panel face-to-face meeting [20]. Ethics approval for this study was obtained by the Conjoint Health Research Ethics Board of the University of Calgary.
2.1. Phase 1: Selection of Pathogens Requiring Diagnostic Test Guidance Definition Development
An advisory panel was convened to (1) identify pathogens that require further definition development and (2) select diagnostic tests including culture, electron microscopy (EM), PCR and EIA, and specimen types (i.e., stool, oral, and rectal swabs [21]), which, if positive, would fulfill the definition for each agent. The advisory panel consisted of pediatricians, pediatric emergency medicine and infectious diseases physicians, gastroenterologists, medical microbiologists, and experts in virology and laboratory medicine. Selection criteria for panel members included interest or expertise in pediatric enteric infections as demonstrated by publications in the field, experience in infectious diseases, geographic and practice diversity, and balance between individuals with clinical and laboratory expertise. Candidate panel members received a written invitation to participate that contained a description of study goals, responsibilities, and timelines.
Candidate advisory panel members received a web-based survey managed in REDCap [22] in which they were asked to (1) rate the degree to which they agreed, on a scale of 1-disagree strongly to 5-agree strongly, that a positive result for each of pathogen/diagnostic test/sample collection methods is causative of the child’s symptoms. It was determined a priori that any pathogen which was unanimously rated as “agree strongly” for all diagnostic test/sample collection modalities would be removed from further consideration as unanimous agreement would indicate that further definition is not required. All other pathogens were retained for further definition development. Free text comments were permitted.
We assessed a list of 19 agents (11 bacteria, 5 viruses, and 3 parasites) that were included in the diagnostic testing panels being evaluated as part of the APPETITE (Alberta Provincial Pediatric EnTeric Infection TEam) project [7]. APPETITE is a provincial initiative designed to comprehensively understand the epidemiology of acute pediatric enteric infections employing rectal swabs and stool samples and a comprehensive diagnostic testing approach.
2.2. Phase 2: Development of Diagnostic Test Guidance Definitions
The study team identified existing definitions for each pathogen retained from phase 1 based on a review of Canada’s provincial public health case definitions. For pathogens with no existing syndromic-based definitions, the study team developed candidate definitions based on the results of the advisory panel survey in phase 1.
2.3. Expert Panel
We convened an expert panel to refine and select the final definitions. The expert panel consisted of individuals trained in pediatrics, infectious diseases, public health, gastroenterology, medical microbiology, virology, and laboratory medicine. Panelists were selected by study investigators from the initial advisory panel and based on recommendations from APPETITE’s International Scientific Advisory Committees [7]. Selection criteria included interest or expertise in pediatric enteric infections, clinical decision making experience, geographic and practice diversity, and publications.
A description of research project goals, a summary of the advisory panel survey results, and the existing and newly developed evidence-based definitions were sent to expert panel members. A modified Delphi technique consisting of two rounds of anonymous questionnaires, and a face-to-face meeting was used to generate final study definitions.
2.3.1. Expert Panel Survey 1
The first web-based questionnaire asked panelists to rate the degree to which they agreed that if the definition is met, in a child with vomiting and/or diarrhea, the pathogen is likely to be the cause of the illness. All definitions were specified to be in the context of (1) clinical illness with acute gastroenteritis defined as a decrease in stool consistency and/or by the presence of three or more episodes of loose stool or vomiting in a 24-hour period with a duration of symptoms less than seven days and (2) a positive laboratory diagnostic test for a single pathogen (i.e., no codetection). Specimens considered included stool as well as oral [23] and rectal swabs. Panelists were asked to rate each definition employing a Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). An option was provided to suggest additional or alternate definitions.
2.3.2. Face-to-Face Meeting
After completion of the first survey, an expert panel face-to-face meeting was convened in Edmonton, Alberta, on March 4, 2016. At the meeting, panelists reviewed anonymized ratings of all the definitions. It was determined a priori that unanimously highly rated definitions (i.e., those rated “agree” or “agree strongly” by all respondents) would be retained, and unanimously low rated definitions (i.e., rated “disagree strongly” or “disagree” by all respondents) would be discarded and not discussed further. Panelists were asked to refine highly scored definitions (i.e., ≥67% of panelists rated “agree” or “strongly agree”) and to discuss the remaining definitions.
2.3.3. Expert Panel Survey 2
After the face-to-face meeting, expert panel members received a final survey requesting that they independently re-rate each of the refined definition using the same rating scale. Definitions rated “agree” or “strongly agree” by ≥67% of panelists were retained in the final list of definitions. The threshold for retention was determined a priori by the expert panel at the outset of the expert panel meeting and has been previously used with such methods [24]. Pathogens for which a consensus definition could not be reached (i.e., further research required) were identified.
3. Results
3.1. Phase 1
Modified Delphi consensus process used to develop diagnostic test guidance definitions for pediatric enteric diseases is shown in Figure 1.
The advisory panel web-based survey was initially distributed to 88 individuals on March 26, 2015, with two follow-up e-mail reminders. Thirty-seven of 88 invited experts (42%) completed the advisory panel survey. The majority (89%; 33/37) of respondents were ≥40 years of age and practiced in Canada (76%; 28/37); the remaining 24% (9/37) were from the United States. The majority of the Canadian participants were from Alberta (52%; 14/27). Other provinces and territories represented included British Columbia, Northwest Territories, Nova Scotia, Ontario, and Quebec. Advisory panel participants included individuals with expertise in medical microbiology, pediatric infectious disease, pediatric emergency medicine, community and public health, and laboratory medicine (see Supplemental Table 1). Respondents were similar to nonrespondents with respect to specialty and geographic location.
No pathogen-test method had unanimous agreement (i.e., rated “strongly agree” by all) in terms of assigning causality of symptoms in a child with acute gastroenteritis (Table 1). Consequently, all 19 pathogens were considered to require definition development. For bacterial pathogens, the median proportion agreeing with causality (i.e., rated “agree” or “strongly agree”) was higher for stool culture (90%) compared with stool PCR (83%) and oral swab PCR (30%). The bacterial pathogen with the lowest percent endorsement by both culture (40%; 95% CI: 23, 59) and PCR (30%; 95% CI: 15, 50) was Edwardsiella. Aeromonas and Clostridium difficile also generated <50% agreement as to likelihood of being etiologic when identified. Viral pathogens identified by stool PCR had high levels of agreement as likely being the cause of the symptoms (range: 75–97%) with the exception of general adenovirus which was associated as etiologic by only 51% of respondents (95% CI: 34%, 68%). All positive results on stool evaluated for parasites were endorsed by over 50% of study participants.
3.2. Phase 2
3.2.1. Expert Panel: Premeeting Survey
Fourteen of 17 (82%) individuals invited to participate in the in-person expert panel process completed a premeeting definition survey (Supplemental Table 2). The survey was distributed on January 22, 2016, with an e-mail reminder one week later (Table 2). Thirteen of the 19 definitions evaluated were rated as being highly likely (“agree” or “strongly agree”) to be responsible for acute gastroenteritis symptoms by ≥67% of respondent panel members. The definitions for the remaining six agents (Aeromonas, C. difficile, Edwardsiella, nonenteric adenovirus, astrovirus, and Entamoeba histolytica) were indeterminate. The lowest rated bacterial pathogen was C. difficile (36%; 95% CI: 14, 64) while Escherichia coli O157 : H7 was the only pathogen classified as etiologic by 100% of respondents. Adenovirus was the lowest rated (36%; 95% CI: 14, 64) viral pathogen while norovirus was the only pathogen classified as etiologic by 100% of respondents.
3.2.2. Expert Panel: Meeting
Twelve of 14 (86%) expert panel members convened at a meeting on March 5, 2016, in Edmonton, Alberta. The discussion focused on achieving definitions which could obtain broad endorsement by experts. The results of the advisory panel survey (Table 1) and the first expert panel survey (Table 2) were presented and used to modify the definitions. The definitions for all 19 pathogens were discussed and revised with the meeting focused primarily on reviewing the six agents with indeterminate ratings. Revisions included the specific wording of both the definition and the agent. For example, the generic term “nucleic acid test” (NAT) was adopted to replace more specific terminology such as PCR. Yersinia pseudotuberculosis was removed from the definition for Yersinia enterocolitica because the expert panel felt it is unclear if Yersinia pseudotuberculosis causes GI illness. For viruses, adenovirus (any serotype) was reworded to adenovirus (nontyped). For parasites, Entamoeba histolytica was updated to Entamoeba histolytica/dispar because, although dispar is not pathogenic, the two agents cannot be differentiated using direct microscopy or by most antigen-detection tests [25–27]. The definitions that emerged from this process were then recirculated via a final survey to all attendees.
3.2.3. Expert Panel: Postmeeting Survey
All twelve experts who attended the panel meeting completed the final survey which was distributed on March 21, 2016 (100% response rate; Table 3). Eight of the final definitions had a proportional agreement of 100%. C. difficile and E. histolytica/dispar did not achieve the a priori specified threshold of ≥67% agreement. Sixteen of the 19 (84%) modified definitions had a higher proportion agreement than the definitions presented in the expert panel premeeting survey.
4. Discussion
This rigorous and systematic process provides expert consensus-based diagnostic test guidance definitions for 17 organisms contained in gastroenteritis nonculture detection panels. These definitions provide guidance to healthcare providers on the interpretation of positive diagnostic test results in the era of multianalyte molecular diagnostics. We identified two pathogens (C. difficile and E. histolytica/dispar) for which further research and definition development is required.
Our definition work is important as multianalyte enteric pathogen detection approaches are rapidly being adopted across North America. Many healthcare providers may have a limited understanding that these molecular techniques detect nucleic acids of targeted organisms but do not differentiate nonviable organisms or free DNA/RNA from viable pathogens. Positive results generated by molecular assays do not definitively distinguish between asymptomatic carriage, subclinical infection, active disease, resolved disease, or the presence of nucleic acids originating in nonpathogens. They also do not provide definitive information about virulence potential, and asymptomatic children have been found to have high rates of enteric pathogen identification [17, 28].
A positive C. difficile test represents the classic challenge clinicians face in multianalyte arrays as its presence may represent colonization and not pathogenic infection, particularly in infancy [11, 13, 28, 29]. Many multianalyte panels also identify organisms that were not previously routinely sought, such as enteroaggregative E. coli (EAEC), and enteropathogenic E. coli (EPEC). The latter two bacteria have been identified in 84% of mixed infections, raising questions about the clinical significance of their identification and the validity of the definitions employed [13]. As such, the definitions developed in our study can serve as a starting point for clinicians interpreting the results of multianalyte arrays.
Our study also reflects current understanding of the use of oral swab NAT for the diagnosis of bacterial and viral enteric organisms. The paucity of evidence regarding the utility of oral swabs is reflected in the poor ratings for the use of oral swabs in our advisory panel survey and has subsequently been supported by a recent study [21]. As a result, oral swabs are not included in any of the definitions with the exception of norovirus for which there is some evidence of clinical utility [23, 30].
We were unable to develop consensus-based diagnostic test guidance definitions for two pathogens, C. difficile and E. histolytica/dispar. This is not surprising given the challenges surrounding the accurate diagnosis of C. difficile. Among healthy adults, asymptomatic C. difficile colonization prevalence varies between 0 [31] and 7% [32], and it is even higher in healthy newborns and infants [33]. The prevalence of C. difficile colonization decreases from 35% to 40% during the first month of life to approximately 15% by one year of age [34]. However, 10–20% of older infants continue to asymptomatically carry toxigenic organisms and the associated toxin genes, and in many asymptomatic children, the toxins are well expressed [28, 35]. In addition, there is limited evidence on the duration of the colonized state, the risk factors related to transmission, and the mechanisms underlying progression from colonization to disease [33]. The lack of a standardized definition for asymptomatic colonization, or carriage, makes the interpretation of test findings, and the development of a diagnostic test guidance definition, difficult. Further research is required to develop an evidence based, standardized definition for C. difficile disease, and equally importantly, colonization.
In the developing world, E. histolytica is a common cause of protozoan morbidity and mortality [36]. However, infection is uncommon in North America, and occurs primarily in immigrants and travelers from developing countries [37]. Nonetheless, asymptomatic colonization is common, occurring in up to 4% of individuals in high-risk areas [38], such as California, and it occurs with both E. histolytica and the nonpathogenic E. dispar. Colonization with the morphologically identical parasite E. dispar is three times more common in developing countries and at least ten times more common in developed nations [37]. Routine diagnostic tests such as direct microscopy cannot differentiate between E. histolytica and E. dispar based on morphologic criteria, so most laboratory reports indicate E. histolytica/dispar [14]. The relatively low prevalence in North America, combined with the high colonization rates with both the pathogenic, and nonpathogenic form, and the difficulty in differentiating the two, likely account for the lack of consensus for a diagnostic test guidance definition. It should also be noted that the panel consisted of individuals who have largely practiced in North America, where amebiasis is exceptionally rare in childhood. However, recently published work on the most common causes of diarrhea using molecular methods indicated that even in Asia and Africa, the most common causes of diarrhea were Shigella, rotavirus, adenovirus 40/41, heat-stable enterotoxin-producing E. coli, Cryptosporidium, and Campylobacter, which supports the broad-based utility of our clinical definitions [2].
A strength of this project is the use of a rigorous consensus methodology consisting of multiple survey rounds and a face-to-face meeting, which enabled dialogue [20]. Similar methods have been used to combine best evidence and expert opinion in a number of clinical and research settings [39, 40] including antimicrobial stewardship [41]. Both our advisory and expert panels had geographic representation from throughout North America and including a diversity of disciplines, thereby contributing to the utility, credibility, and generalizability of our definitions. We had representation from a broad spectrum of stakeholders involved in the diagnosis and management of pediatric infectious enteric diseases including front-line clinicians, medical microbiologists, infectious disease specialists, and public health experts. The engagement of our expert panel, with 86% of our expert panel participating in the face-to-face meeting, and a 100% response rate on our final survey, is evidence of the relevance and importance of the topic and the validity of the results. Nonetheless, our definitions can be considered as a derivation set; future work should include a validation with a distinct expert panel.
It should be noted that the definitions provided are only applicable in the setting of a test that is positive for a single pathogen. Clinicians should also be cognizant that in most cases where the definitions provided are met, therapy remains predominantly supportive and few of the enteropathogens included in Table 3 require antimicrobial treatment, which can at times even be detrimental [42]. Although multiple pathogens are detected in ∼15% of stool specimens in children with diarrhea [10], determining guidelines for the interpretation of a positive test for more than one pathogen was beyond the scope of this project. The guidance provided is also very general and does not incorporate the myriad of clinical scenarios that can occur. The definitions are not intended to replace expert opinion (e.g., infectious disease specialist, medical microbiologist) but rather are intended to assist clinicians in the context of otherwise healthy children with typical clinical features and <7 days of symptoms. We also identified two agents, C. difficile and E. histolytica/dispar, for which further research is required to develop accurate diagnostic test guidance definitions. Prospective research currently underway [7] is attempting to validate these definitions in the clinical setting of pediatric enteric illness.
We also did not assess the confidence in the individual items included in each definition, nor the potential role of quantitative molecular techniques [2]. Thus, although the definitions may include multiple diagnostic testing methods, the test characteristics of each modality vary. While these definitions may lead clinicians to conclude that all diagnostic tests are created equal, as diagnostic testing evolves and progresses, less accurate tests are phased out and replaced by superior diagnostic tests. It should also be noted that our process did not include some of the diarrheagenic E. coli strains (e.g., ETEC, EAEC, and EPEC) that are included in some multianalyte tests while it did include some pathogens not routinely identified by all laboratories (e.g., Edwardsiella). This likely stems from the fact that few clinicians and microbiologists have any significant experience with the detection of these enteric pathogens outside of research settings. Currently, very few commercially available assays include all these pathogen targets [43]. As experience is accumulated with the detection of these pathogens in clinical laboratories, it would certainly be useful to include them in a subsequent evaluation. Finally, as we had to limit the number of study participants, and their expert opinions may not reflect the entire spectrum of expertise, our findings should not be interpreted as the only perspective on this topic.
In conclusion, we report a rigorous, systematic, expert consensus process that includes diagnostic test guidance definitions for 17 enteric organisms. These definitions can be employed by healthcare providers to interpret positive diagnostic test results in the era of multianalyte molecular diagnostics as we await further clinical data, particularly with respect to consideration of clinical settings and interpretation of detection of more than one pathogen. We also identified pathogens for which further definition development is required.
Abbreviations
| EAEC: | Enteroaggregative E. coli |
| EIA: | Enzyme immunoassay |
| EPEC: | Enteropathogenic E. coli |
| NAT: | Nucleic acid test |
| PCR: | Polymerase chain reaction |
| STEC: | Shiga toxin-producing Escherichia coli. |
Data Availability
All data generated or analysed during this study are included in this published article and the supplemental material.
Ethical Approval
Ethics approval for this study was obtained from the Conjoint Health Research Ethics Board of the University of Calgary.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This paper is on behalf of the Alberta Provincial Pediatric EnTeric Infection TEam (APPETITE). This research was funded in part by The Alberta Provincial Pediatric EnTeric Infection TEam (APPETITE), which was supported by an Alberta Innovates-Health Solutions Collaborative Research Innovation Opportunity (CRIO) Team Grant. The Alberta Provincial Pediatric EnTeric Infection TEam (APPETITE) received in-kind support from Luminex Corporation and COPAN. All involved researchers would like to thank the APPETITE for its support. Additional support for APPETITE was provided by the Alberta Children’s Hospital Research Institute (Calgary, Alberta) and the Women and Children’s Health Research Institute (Edmonton, Alberta). Dr. Stephen Freedman was supported by the Alberta Children’s Hospital Foundation Professorship in Child Health and Wellness.
Supplementary Materials
Supplemental Table 1: demographic information for participants of the initial survey. Supplemental Table 2: expert panel members. (Supplementary Materials)