Abstract
Diagnostic codes within electronic health record systems can vary widely in accuracy. It has been noted that the number of instances of a particular diagnostic code monotonically increases with the accuracy of disease phenotype classification. As a growing number of health system databases become linked with genomic data, it is critically important to understand the effect of this misclassification on the power of genetic association studies. Here, I investigate the impact of this diagnostic code misclassification on the power of genetic association studies with the aim to better inform experimental designs using health informatics data. The trade-off between (i) reduced misclassification rates from utilizing additional instances of a diagnostic code per individual and (ii) the resulting smaller sample size is explored, and general rules are presented to improve experimental designs.
1. Introduction
Clearly, a wealth of important clinical information is contained within large electronic health record (EHR) systems. Such information can be an invaluable resource for measuring disease prevalence [1] and disease comorbidity [2], the association between birth month and disease susceptibility [3], the prediction of outcomes [4], the measurement of economic impact of health care [5], and the discovery of etiological factors [6]. A key feature of these data is in the diagnostic codes given by medical professionals to patient records. However, the accuracy of inferring disease phenotypes from electronic diagnostic codes can vary widely across diseases and is often subject to high degrees of error [7–10]. These studies have noted the substantial misclassification effects from the use of electronic diagnostic code data, sufficient to undermine experiments utilizing cases and controls defined by the International Classification of Diseases (ICD) codes alone. The ICD coding system is instituted by the World Health Organization and has been adopted in the United States by the National Center for Health Statistics. More sophisticated approaches to disease classification, such as those using a variety of EHR data and machine learning methods, are difficult to generalize across all diseases and implement in a high-throughput manner. That said, I anticipate that machine learning methods applied to problems of phenotype prediction using EHR variables as features in the predictive modeling will eventually supplant the sole use of ICD code data. Until that time, the use of ICD data may still have utility in initial screens, to be subsequently validated through methods with higher positive and negative predictive values.
2. Related Work
In a general setting, the effect of phenotypic misclassification on statistical power of genetic association studies has been previously explored [11–14]. Edwards and colleagues characterized the noncentrality parameter in asymptotic power distributions given the presence of phenotypic misclassification [11]. The authors use cost functions to capture the effect of misclassification and show that the cost of misclassifying a control as a case becomes large and the cost of misclassifying a case individual as a control becomes small as the disease prevalence becomes small. Similarly, Ji et al. also investigated the calculation of a noncentrality parameter capturing phenotype errors for subsequent use in a likelihood ratio test for genetic association studies [12]. Later, Gordon and colleagues showed how to incorporate misclassification error rates into a trend test for genetic association in case/control studies [13]. More recently, Manchia and colleagues investigated the impact of heterogeneity within a clinical phenotype on genetic association [14].
Considering ICD data with misclassification, the type I and type II error rates for genomic association studies were recently thoroughly explored by Duan et al. [15]. The Duan et al. study found little inflation in false-positive rates, but not in considerable false-negative rates under certain allele frequency, effect size, and disease prevalence parameters. In the context of initial screens of ICD codes in EHR systems, several studies have investigated the relationship between the number of instances of particular ICD codes and the measures of diagnostic utility [1, 16–18]. In general, the accuracy of diagnoses improves with the number of instances of the code; however, this is at the expense of smaller sample sizes/increasing false negatives. Hence, there is a trade-off between type I and type II error rates with the number of ICD code instances used to define a disease. In this work, I investigate this trade-off and provide a framework for determining highly powered EHR-based experimental designs using diseases defined by different numbers of instances of ICD codes.
3. Materials and Methods
For a large genetic association scan of using ICD data, define a simple disease classification scheme such that cases are those individuals with instances of a particular ICD code. Consider a design where individuals with ambiguous numbers of instances () of the code (i.e., ) are excluded from the analysis. Further consider a comparison of well-defined cases (i.e., those with at least instances) against a large, fixed set of controls. With regard to the genetics, restrict the methods to biallelic markers with minor alleles segregating in the population at a frequency of at least 1% single-nucleotide polymorphisms (SNPs). Define the alleles at a SNP contributing to the susceptibility of the disease as and . Let the relative risk of the minor allele, , be , such that . Let the frequency of in the general population be . Accordingly, is the frequency of . Define as the number of cases obtained from the definition of having at least instances of the ICD code being evaluated. Set the number of controls as , such that . Assume that the frequency in controls is approximately . Model the decrease in the misclassification proportion within cases as increases with a monotonic function , such that the expected number of truly positive cases is . The form of may vary considerably for different ICD codes. Lastly, let be the statistical threshold for determining a positive finding in analyses where value < . The statistical test of genetic association considered is the binomial test of proportions which evaluates the null hypothesis of no correlation between the frequency of and the disease status.
Statistical power will be used to evaluate the impact of increasing and the resulting experimental design. Under the model specified above, the power to detect association at an autosomal SNP, , is calculated by the approximation as follows: where is the standard Gaussian cumulative distribution function, is the inverse standard Gaussian score, , and and are the frequencies in controls and cases, respectively. Using Bayes’ theorem, the expected frequency of within cases under the misclassification model is given by
To model the decrease in the misclassification rate with increasing numbers of ICD code instances, consider the simple decay function for : where is the parameter that can be estimated for each ICD code. Similarly consider the following form for as a function of to model the reduction in the number of cases defined by using increasing numbers of instances of an ICD code: where is the parameter that captures the rate of decline in case numbers as the definition for case status becomes more stringent with the use of larger numbers of ICD code instances and can also be estimated for each ICD code. The machinery is now in place for the calculation of statistical power to detect disease association at a genetic marker using data from linked ICD coding systems.
4. Results and Discussion
The above model is used to conduct an exploration of the impact of ICD code definitions on power. To obtain a value of which maximizes power to detect genetic association, one can numerically solve the following differential equation for :
The solution to (5) can be solved through standard numerical methods applied to solving where
The closest integer value to the value of that solves this continuous equation can be used to optimize the power for a given set of parameters. To exemplify the use of this approach, let , , , , , and . Call this set of parameters the baseline model. solves the differential equation. Therefore, using seven instances of an ICD code will yield the optimal design weighing the trade-off between the case sample size and the misclassification. For that set of parameters, Figure 1 shows the power curve for this set of parameters.
To investigate the power curves, varying the baseline number of cases (), the calculations were performed as the varied from 100 to 800. Visual inspection shows the peak of power at approximately 7 instances. Figure 2 shows the results.
Next, to determine the role of the and parameters on the power curves, the calculations were performed fixing the other parameters. Figures 3 and 4 display these results.
5. Conclusions
Genetic data linked to longitudinal electronic health records can serve as a very useful tool in modern disease genetics. However, misclassification present in ICD coding systems can severely hamper large-scale screens using those codes for the purpose of genetic association studies. This work has described a simple approach to better understand the impact of misclassification present in EHR systems for the purpose of optimizing experimental designs that screen numerous ICD codes in genetic association studies. Under the mathematical models considered, the methods offer an approach to select the number of instances of an ICD code for the purpose of defining cases and obtaining an optimal experimental design for the identification of genetic markers. Additional work is needed in this area to improve disease classification schemes for genetic association studies as well as for other investigations.
Disclosure
The content is solely the responsibility of the author and does not necessarily represent the official views of the National Institutes of Health.
Conflicts of Interest
The author declares that he has no conflicts of interest.
Acknowledgments
The author would like to thank his colleagues Mehdi Maadooliat, Zhan Ye, Scott Hebbring, Ahmad Pahlavan Tafti, and Peggy Peissig for the very useful conversations pertinent to this investigation. The research reported in this publication was supported by generous donors to the Marshfield Clinic, NIMH of the National Institutes of Health Award (4RO1MH097464-04), and the Institute for Clinical and Translational Research supported by the Clinical and Translational Science Award (CTSA) program and the National Center for Advancing Translational Sciences (NCATS) (Grant UL1TR000427).