Table of Contents Author Guidelines Submit a Manuscript
Computational and Mathematical Methods in Medicine
Volume 2013, Article ID 170120, 8 pages
Research Article

Validation Study Methods for Estimating Odds Ratio in Tables When Exposure is Misclassified

1Department of Biostatistics, Faculty of Medicine, Shiraz University of Medical Sciences, Shiraz 7134845794, Iran
2Department of Biostatistics, Infertility Research Center, Shiraz University of Medical Sciences, Shiraz 7134845794, Iran

Received 27 September 2013; Revised 27 November 2013; Accepted 30 November 2013

Academic Editor: Edelmira Valero

Copyright © 2013 Bijan Nouri et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


Background. Misclassification of exposure variables in epidemiologic studies may lead to biased estimation of parameters and loss of power in statistical inferences. In this paper, the inverse matrix method, as an efficient method of the correction of odds ratio for the misclassification of a binary exposure, was generalized to nondifferential misclassification and tables. Methods. Simple estimates for predictive values when misclassification is nondifferential are presented. Using them, we estimated the corrected log odds ratio and its variance for tables, using the inverse matrix method. A two-step weighted likelihood method was also developed. Moreover, we compared the matrix and inverse matrix methods to the maximum likelihood (MLE) method using a simulation study. Results. In all situations, the inverse matrix method proved to be more efficient than the matrix method. Matrix and inverse matrix methods for nondifferential situations are more efficient than differential misclassification. Conclusions. Although MLE is optimal among all of the methods, it is computationally difficult and requires programming. On the other hand, the inverse matrix method with a simple closed-form presents acceptable efficiency.

1. Introduction

In epidemiology studies, where the assessment of the relationship between exposure and outcome variables is the main goal, misclassification of exposure variable leads to biased estimate of odds ratio. In multicenter clinical trials with an increasing number of centers, the possibility of misclassification of the exposure variable and the biases induced by it will arise. Methods to correct a possibly misclassified exposure so that the strength of the association between exposure and outcome variables can be precisely assessed have been a focus of statistical and epidemiological research for over 30 years. Beginning with classic papers, the issue of misclassification on tabular data has long been recognized and their adjustment has been considered [15]. In 1977, the matrix method was presented by Barron in order to correct nondifferential misclassification in tables [6]. Greenland and Kleinbaum generalized it to differential misclassification and match paired data [7]. In addition, Greenland proposed a variance estimate for the matrix method under the assumptions of differential and nondifferential misclassifications [8]. Selen, in a simulation study, showed that both the matrix and the maximum likelihood methods performed equally well [9]. In 1990, Marshall presented a more direct inverse matrix approach for correcting differential misclassifications [10]. Morrissey and Spiegelman compared the matrix and inverse matrix methods to the maximum-likelihood estimator using a grid search for tables [11]. He found that, under the assumption of differential misclassification, the inverse matrix method was always more efficient than the matrix method. Examples of validation studies in the misclassification context in linear models are prevalent in the statistical and epidemiologic literature [1215]. More recent works are included in the reference section [1620].

The main impediment to the use of the inverse matrix method, despite its superiority compared to the matrix method, is that it is restricted to differential misclassification and tables, while, in practice, it is very likely that misclassification is nondifferential or that we have a table, in which determines the number of centers in a multicenter clinical trial or is the levels of a confounder [21]. For example, when misclassification is due to recall bias, it is expected that the rates of misclassification in case and control groups are the same, leading to nondifferential misclassification. Here, we aim to further extend the focus within the inverse matrix method when misclassification is nondifferential and data is stratified on strata.

Section 2.1 provides definition and notation for a table with a binary error-prone exposure. For the first time, we propose estimates for negative and positive predictive values as misclassification parameters under the assumption of nondifferential misclassification, which can be used to generalize an inverse matrix method to nondifferential cases (Section 2.2). Then we will generalize an inverse matrix approach to a nondifferential assumption and situations where the data is stratified on strata (Section 3.1). Intuitive closed-form formulae for misclassification-adjusted effect and its variance are presented. The matrix method and the likelihood method will be briefly reviewed later. In addition, we will present a new two-step method in Section 3.4, which uses corrected cell count by the matrix method to construct a weighted maximum likelihood method. Finally, the study will continue with a simulation study that compares the mean square error (MSE) of each of the presented methods to the MLE under the assumptions of differential and nondifferential misclassifications. The main text focuses on simple formulae for conditions involving a binary misclassified exposure in a case-control study, with stratification on confounders. Several assumptions were made in this work. First, the disease status and level of strata were measured accurately. Second, the methods required an error-free criterion for exposure to validate the misclassified exposure in the validation study. Finally, we assumed that the main and validation studies are independent. The formulae are illustrated with data from an often-cited case-control study of sudden infant death syndrome (SIDS) and maternal use of antibiotics during pregnancy [22].

2. Method and Material

2.1. Definitions and Notations

To begin, consider a case-control study sample; binary exposure and outcome variables were measured by two error-prone and correct methods, respectively, as well as a level variable ( can be a confounder or a combination of some confounders). So, the data could be classified in a table, for which its th stratum contains cases and controls. Because the exposure status is an error-prone variable, the cross-table will be misclassified. The th stratum of a misclassified table and the notations that are employed in this paper are displayed in Table 1.

Table 1: Notation for th stratum of a misclassified table.

is supposed to be an error-free variable that shows the actual status of the exposure. The sensitivity and specificity of the error-prone diagnosis of exposure are known as misclassification parameters or misclassification rates and defined by and for and , respectively. Also one can offer misclassification rates by another set of parameters, known as positive and negative predictive values, which can be defined as and for and , respectively. Among different approaches developed to deal with the misclassification problem, the inverse matrix method (direct method) uses positive and negative predictive values as misclassification parameters; on the other hand, the matrix method (indirect method) and the likelihood method both apply sensitivity and specificity values instead. Let represents the prevalence of error-prone exposure for and , and also .

Both of the sets of misclassifications are unknown, and it is required to estimate them through a validation study. Thus, in addition to the sample described above known as the main study, another random sample of size is drawn separately. This sample makes up the external validation study. In order to estimate the misclassification parameters, as well as the error-prone status of exposure , the correct exposure status should be measured precisely for each subject of the validation sample by a dichotomous error-free variable .

2.2. Estimation of Misclassification Parameters

Misclassification parameters are allowed to vary by levels of in this approach and separate misclassification parameters for each stratum can be estimated, but if we notice that the misclassification rates are similar in strata, regardless of stratification, validation samples of strata should be combined. Due to the simplicity of notation, we will describe the case in which misclassification rates are the same across strata; otherwise, estimation of the parameters is similar. Table 2 displays the available data for the validation study, where subscripts , , and indicate the outcome, error-free, and error-prone exposure status, respectively. Misclassification parameters and their variances are estimated in the validation study by the following formulae: In this case, a point, “.”, in the subscripts stands for summation over that subscript. Misclassification is nondifferential when the misclassification rates are independent of the outcome status. In other words, when and = ; otherwise, misclassification is differential. We can easily adapt estimates of sensitivity and specificity to the following nondifferential assumption: But once we assume that misclassification is nondifferential, the sensitivity and specificity are equal given the disease status. In this case, nondifferential sensitivity and specificity do not mean that PPV and NPV are equal across strata of the outcome because PPV and NPV are functions of exposure prevalence and sensitivity and specificity. Due to this problem, the methods that apply predictive values as misclassification parameters (such as the inverse matrix method) will be restricted to the differential assumption. In order to overcome this difficulty, Morrissey and spiegelman recommended the use of the same predictive values as those estimated by the differential misclassification assumption, which is not a reasonable approach [11]. Instead, in the case of nondifferential misclassification, we present the following estimates for predictive values: where . If sensitivity and specificity are constant and known, using the probability roles, we can estimate predictive values as follows: When equality of the misclassification parameters in strata cannot be assumed, using distinct validation sample for each stratum and the above relations, misclassification parameters can be estimated separately.

Table 2: Data layout for the validation study.

Example 1. Table 3 exhibits the main study data from an often-cited case-control study on sudden infant death syndrome (SIDS), which has examined the relationship between maternal use of antibiotics during pregnancy and the odds of SIDS [22]. The table is classified on two strata () according to the sex of infants, a common risk factor for SIDS. Drug use error-prone measurement was an interview response , and it was validated by medical records . A separate sample from a validation study with dual exposure measurement (both error-prone and error-free) is presented in Table 2. Using (1)–(3) and the validation data, separate estimates with regard to both differential and nondifferential misclassification assumptions for sensitivity, specificity, and predictive values are presented in Table 4. Two cells in the last column of Table 4 are blank because, for nondifferential misclassification, sensitivity and specificity are the same for case and control groups.

Table 3: Uncorrected data from SIDS study stratified on the sex of infant.
Table 4: Estimates of misclassification parameters and their variance.

3. Estimate of the Actual Association

3.1. Inverse Matrix Method

As mentioned in the introduction, the inverse matrix method was proposed by Marshall to correct misclassified tables [10]. He restricted the use of his method to the assumption of differential misclassification. According to the estimates we present for predictive values in (3), the inverse matrix method can be developed to the differential assumption. Now, we generalize this approach to tables with misclassified exposure and either differential or nondifferential assumptions. Estimates for correct proportions of exposed and unexposed subjects in the th stratum of the main study are and , respectively. These equations can be written in the matrix form as The biased log odds ratio estimate of the th stratum is , with asymptotic variance estimate where . Corrected values in (5) are used to build the inverse matrix corrected log-odds-ratio where representing the estimate of actual prevalence of exposure for and , and also . Considering binomial distribution for , we derived the asymptotic variance for using the delta method as where the letter stands for an abbreviation for variance and . If nondifferential misclassification is assumed, instead of the estimated positive and negative predictive values and their variance estimate in (1), corresponding values in (3) must be considered substitutes in expression (6) and also the corrected proportions in the disease group are no longer independent of the corrected proportions in the control group. Consequently, twice the covariance between log odds (7) must be subtracted from expression (6). Consider If in the validation study for each stratum a separate misclassification-rate is estimated, the log odds ratios are independent; otherwise, using the same misclassification parameters for all the strata leads to covariance (8) between and . Consider If the same misclassification parameters for all the strata are estimated by assuming nondifferential misclassification, the covariance estimate for and is given by

Let be the symmetric by variance-covariance matrix of the estimated odds-ratio, with th element . Then is the minimum variance weighted average of the estimated log-odds ratios, where , and also and are -vector of ones and estimated odds ratios, respectively. For large strata sample, is a consistent uniformly asymptotic normal estimator for the actual log odds ratio, with consistent variance estimator . Hence, construction of a corrected Wald test of no association will be possible using the statistic , with a corresponding corrected percent confidence interval for the actual log odds ratio as , where is the percentile of a standard normal distribution.

In order to evaluate the homogeneity of , we can utilize an approximate degree of freedom Chi-square statistics . Large values of this statistic indicate loss of homogeneity and so will be an inadequate summary of the association.

3.2. Matrix Method

Now we briefly outline the matrix method. In this method sensitivity and specificity are used as misclassification parameters to estimate the log-odds ratio. The matrix method uses the equations and to estimate correct cell counts in the th stratum; in matrix form these equations can be written as The matrix corrected log-odds-ratio is , and . Considering binomial distribution for , Greenland provided the asymptotic variance for , given by where . The other components of the variance-covariance matrix are similar to what we derived for the inverse matrix method.

This method has two limitations: when the sum of sensitivity and specificity is equal to one, the corrected count estimates are undefined because the misclassification matrix is singular in (10); if the sum is less than one, negative estimates for corrected cell counts occur.

3.2.1. Direct Likelihood Method

Some authors use the maximum likelihood estimate (MLE) to estimate the actual odds ratio and test the exposure-disease association. Although this method can be the most efficient method of correction, it does not have a close form and requires an iterative solution to a nonlinear set of equations under some constraints. If the error-free exposure variable was available, we would be able to estimate the covariate-adjusted odds ratio by the following multiple logistic regression model:

However, we just have access to the error-prone binary variable . We assume that the covariates are dummy variables that determine to which stratum a subject belongs. If the subject belongs to the th stratum , as mentioned before, indicates overall log odds ratio. To obtain the ML estimate for unknown parameter , we should numerically maximize the following log likelihood function with respect to . The approximate standard error for this estimate is available by inversing the observed information matrix. Consider where was presented in Table 1; note that in expression (13) is the observed data likelihood contribution in the likelihood function which can be obtained as follows: where the first term in (14) can be obtained using the estimated sensitivity and specificity in (1) or (2) according to whether misclassification is differential or nondifferential, the second term displays the logistic model (12), and finally the last term is a nuisance parameter that can be modeled via a second logistic regression model of on .

3.3. A Weighted Likelihood Method

Now, we briefly present a two-step method which is a combination of the inverse matrix method and the likelihood method. In the first step, we use the inverse matrix method to correct misclassified tables, and in the second step, we utilize corrected cell count in the first step that is assigned to log-likelihood contribution as weights. Formally, the weighted log-likelihood is given by where the represents the corrected data cell count by the inverse matrix method and subscripts , , and indicate error-free exposure, outcome, and strata number, respectively. represents the corrected data cell count by the inverse matrix method and is its log-likelihood contribution. This weighted method is simpler than the previous likelihood method and directly uses log-likelihood contribution instead of . The variance of corrected log-OR can be obtained by inversing the Fisher information matrix.

3.4. Combining the Results

Now, we need an approach to combine the results from the main study and the internal validation study. We emphasize that, for subjects in the validation study, a conventional analysis using an error-free exposure can be done to obtain a log-OR estimate with variance estimate . The estimate of the effect for the main study is biased and must be adjusted using the earlier formulae in order to obtain a corrected log-OR estimate with variance estimate . A weighted method to combine the log-OR from the two samples can be constructed as follows: has variance estimate . The weighted method proceeds well if and estimate a common value, and this is achieved when samples are randomly selected from the population, and the misclassification parameter estimates are unbiased.

Example 2. For the data in Table 3, we have misclassified and , in which the term in the bracket represents the estimated variance. Suppose that the investigator assumes that misclassification rates are equal in both the strata of male and female infants. Under the nondifferential assumption and using the parameters estimated in Table 4, the matrix method yields and , and the inverse matrix method yields and . In contrast, we supposedly have differential estimates for misclassification rates in Table 2; thus, the results from the matrix and inverse matrix method will be and , respectively.
Applying the direct ML (13) to the data in Table 3 yields and under the differential and nondifferential assumptions, respectively. Note the dramatic shift in the implication of assuming a differential misclassification. The application of the two-step weighted likelihood approach produces and given the differential and nondifferential cases, respectively.

4. Simulation Studies

We conducted a simulation study to compare corrected log-odds-ratios obtained from both matrix and inverse matrix methods and also to assess the appropriateness of the maximum likelihood and weighted maximum likelihood for tables. In order to mimic the SIDS example, a total of 1000 sets of data were generated with a binary outcome () and also two other binary variables as error-free exposure status and stratification indicator ( and ) and a total sample size of 1144. Correct cell counts for each table were generated with regard to the following simulation conditions: prevalence of and conditional prevalence of exposure are and . In order to generate the response variable, the binary distribution with parameter was used, where we set the effect parameters for all the simulation scenarios as , and . The misclassified exposure () was generated assuming numerous nondifferential conditions. In order to estimate the misclassification parameters for every one of the simulated data sets, three scenarios of 20%, 30%, and 40% of each of their strata were taken as internal validation samples separately. The programming code could be provided via request from the authors.

The last four columns of Table 5 represent different scenarios related to cases in which nondifferential misclassifications vary across strata, while the four columns before them display various situations in which nondifferential misclassifications are constant across strata; analysis of each column is in accordance with the manner in which that column was generated. For the purpose of comparing the different methods, the mean square error (MSE) measure was used in order to take into account the bias and variance of estimators simultaneously. As you can see, the MLE estimator is more efficient than other estimator methods in all scenarios. Our simulation study showed that for nondifferential misclassification the performance of these inverse matrix methods and likelihood method is very close but is not equivalent. The inverse matrix method is significantly more precise than the matrix method. Performance of the likelihood and weighted likelihood methods is relatively the same under both assumptions. The efficiency of all estimators would be improved by increasing the validation study sample size, but this improvement would be more prominent in the weighted likelihood method, as the efficiency of the estimator will be approximately doubled by increasing the validation study sample size.

Table 5: Simulation results for a table when binary exposure is misclassified.

5. Discussion and Conclusions

The misclassification of exposure is an issue of broad concern in epidemiological research studies, and considerable amount of pieces of literature discuss adjusting inferences on exposure-disease relationships regarding such misclassification. Matrix method, inverse matrix method, likelihood method, Bayesian method, and SIMEX method are some of the general tools for performing such adjustments. Our main focus was on the development of the inverse matrix method to nondifferential misclassification and tables. At least in the context of misclassified binary exposure, this paper has illustrated several positive attributes of the inverse matrix method. First, to assess the diagnostic tests, the importance of predictive values (rather than sensitivity and specificity) is well known. The inverse matrix method, as a direct method, uses the predictive values. We presented simple closed-form estimates for positive and negative predictive values when misclassification is nondifferential. Second, the simulation study showed that the inverse matrix method estimator is practically more precise than the matrix method in all scenarios.

We also examined the effect of the parameters controlled by the researcher on the MSEs of these approaches. We found that the optimal estimator depends on the size of the validation study. This was due to the fact that, in some cases where the validation sample is large, a good part of the information about can come from the validation study. Morrissey and spiegelman noted this phenomenon previously from the comparison of these methods for tables. These researchers showed that the efficiency of estimators depends more on the relative size of the validation study than on the case-control ratio [11]. In another study, Greenland noted that the samples from which the misclassification parameters are estimated must be large enough to assure the approximate normality of SE, SP, PPV, and NPV [8]. Lyles noted clearly that under the differential misclassification setting, the inverse matrix method is equivalent to the maximum likelihood approach [19]. There are closed-form solutions for both, which are exactly the same. For this reason, both approaches should be equally more efficient than the matrix method.

The methods described in this study handle the misclassification of exposure, but they are easily adapted for disease misclassification. The study depends on some assumptions stated at the end of the introduction. Further studies seem to be necessary to compare and extend these methods, as those assumptions are not met. This would make sense when the outcome status and exposure status are simultaneously misclassified and when there is not a gold standard to validate the exposure.

In conclusion, when trying to make a decision about which method to be used, it can be helpful to consider the following concept. Although the MLE method has minimum MSE in all the scenarios, it can be computationally difficult and does not have a simple closed-form expression like the matrix and inverse matrix methods. When misclassification is non-differential, the inverse matrix method performs very well. Even when the size of the validation study is large enough, it can perform equal to the likelihood method. The two-step likelihood method has a simpler form than the likelihood method and can perform for nondifferential misclassification like the matrix and inverse matrix methods.

Conflict of Interests

The authors declare that they have no conflict of interests.

Authors’ Contribution

Bijan Nouri and Najaf Zare were responsible for the design, simulation, and interpretation. Seyyed Mohammad Taghi Ayatollahi supervised the study and interpreted the results. All the authors read and approved the final paper.


This work was a part of the Ph.D. thesis of Bijan Nouri and was supported by Shiraz University of Medical Sciences, Shiraz, Iran. The authors are also thankful to the reviewers for their invaluable comments.


  1. I. Bross, “Misclassification in 2×2 tables,” Biometrics, vol. 10, no. 4, pp. 478–486, 1954. View at Google Scholar
  2. I. D. J. Bross, “Spurious effects from an extraneous variable,” Journal of Chronic Diseases, vol. 19, no. 6, pp. 637–647, 1966. View at Google Scholar · View at Scopus
  3. J. Cornfield, W. Haenszel, E. C. Hammond, A. M. Lillienfeld, M. B. Shimkin, and E. L. Wynder, “Smoking and lung cancer: recent evidence and a discussion of some questions,” Journal of the National Cancer Institute, vol. 22, pp. 173–203, 1959. View at Google Scholar
  4. G. G. Koch, “The effect of non-sampling errors on measures of association in 2×2 contingency tables,” Journal of the American Statistical Association, vol. 64, no. 327, pp. 852–863, 1969. View at Google Scholar
  5. A. Tenenbein, “A double sampling scheme for estimating from binomial data with misclassifications,” Journal of the American Statistical Association, vol. 65, no. 331, pp. 1350–1361, 1970. View at Google Scholar
  6. B. A. Barron, “The effects of misclassification on the estimation of relative risk,” Biometrics, vol. 33, no. 2, pp. 414–418, 1977. View at Google Scholar · View at Scopus
  7. S. Greenland and D. Kleinbaum, “Correcting for misclassification in two-way tables and matched-pair studies,” International Journal of Epidemiology, vol. 12, no. 1, pp. 93–97, 1983. View at Google Scholar · View at Scopus
  8. S. Greenland, “Variance estimation for epidemiologic effect estimates under misclassification,” Statistics in Medicine, vol. 7, no. 7, pp. 745–757, 1988. View at Google Scholar · View at Scopus
  9. J. Selen, “Adjusting for errors in classification and measurement in the analysis of partly and purely categorical data,” Journal of the American Statistical Association, vol. 81, no. 393, pp. 75–81, 1986. View at Google Scholar
  10. R. J. Marshall, “Validation study methods for estimating exposure proportions and odds ratios with misclassified data,” Journal of Clinical Epidemiology, vol. 43, no. 9, pp. 941–947, 1990. View at Publisher · View at Google Scholar · View at Scopus
  11. M. J. Morrissey and D. Spiegelman, “Matrix methods for estimating odds ratios with misclassified exposure data: extensions and comparisons,” Biometrics, vol. 55, no. 2, pp. 338–344, 1999. View at Google Scholar · View at Scopus
  12. O. Davidov, D. Faraggi, and B. Reiser, “Misclassification in logistic regression with discrete covariates,” Biometrical Journal, vol. 45, no. 5, pp. 541–553, 2003. View at Publisher · View at Google Scholar · View at Scopus
  13. X. Liu and K.-Y. Liang, “Adjustment for non-differential misclassification error in the generalized linear model,” Statistics in Medicine, vol. 10, no. 8, pp. 1197–1211, 1991. View at Google Scholar · View at Scopus
  14. X. Luan, W. Pan, S. G. Gerberich, and B. P. Carlin, “Does it always help to adjust for misclassification of a binary outcome in logistics regression?” Statistics in Medicine, vol. 24, no. 14, pp. 2221–2234, 2005. View at Publisher · View at Google Scholar · View at Scopus
  15. L. S. Magder and J. P. Hughes, “Logistic regression when the outcome is measured with uncertainty,” The American Journal of Epidemiology, vol. 146, no. 2, pp. 195–203, 1997. View at Google Scholar · View at Scopus
  16. R. J. Carroll, D. Ruppert, L. A. Stefanski, and C. M. Crainiceanu, Measurement Error in Nonlinear Models: A Modern Perspective, Chapman & Hall, New York, NY, USA, 2010.
  17. H. Chu, Z. Wang, S. R. Cole, and S. Greenland, “Sensitivity analysis of misclassification: a graphical and a Bayesian approach,” Annals of Epidemiology, vol. 16, no. 11, pp. 834–841, 2006. View at Publisher · View at Google Scholar · View at Scopus
  18. S. Greenland, “Maximum-likelihood and closed-form estimators of epidemiologic measures under misclassification,” Journal of Statistical Planning and Inference, vol. 138, no. 2, pp. 528–538, 2008. View at Publisher · View at Google Scholar · View at Scopus
  19. R. H. Lyles, “A note on estimating crude odds ratios in case-control studies with differentially misclassified exposure,” Biometrics, vol. 58, no. 4, pp. 1034–1037, 2002. View at Google Scholar · View at Scopus
  20. R. H. Lyles and J. Lin, “Sensitivity analysis for misclassification in logistic regression via likelihood methods and predictive value weighting,” Statistics in Medicine, vol. 29, no. 22, pp. 2297–2309, 2010. View at Publisher · View at Google Scholar · View at Scopus
  21. A. M. Jurek, S. Greenland, and G. Maldonado, “Brief report: how far from non-differential does exposure or disease misclassification have to be to bias measures of association away from the null?” International Journal of Epidemiology, vol. 37, no. 2, pp. 382–385, 2008. View at Publisher · View at Google Scholar · View at Scopus
  22. J. F. Kraus, S. Greenland, and M. Bulterys, “Risk factors for sudden infant death syndrome in the US collaborative perinatal project,” International Journal of Epidemiology, vol. 18, no. 1, pp. 113–120, 1989. View at Google Scholar · View at Scopus