Journal of Probability and Statistics

Volume 2015, Article ID 934362, 8 pages

http://dx.doi.org/10.1155/2015/934362

## Confidence Interval Estimation of an ROC Curve: An Application of Generalized Half Normal and Weibull Distributions

Department of Statistics, Pondicherry University, Pondicherry 605014, India

Received 25 June 2015; Revised 22 September 2015; Accepted 7 October 2015

Academic Editor: Shesh N. Rai

Copyright © 2015 S. Balaswamy and R. Vishnu Vardhan. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

In the recent past, the work in the area of ROC analysis gained attention in explaining the accuracy of a test and identification of the optimal threshold. Such types of ROC models are referred to as bidistributional ROC models, for example Binormal, Bi-Exponential, Bi-Logistic and so forth. However, in practical situations, we come across data which are skewed in nature with extended tails. Then to address this issue, the accuracy of a test is to be explained by involving the scale and shape parameters. Hence, the present paper focuses on proposing an ROC model which takes into account two generalized distributions which helps in explaining the accuracy of a test. Further, confidence intervals are constructed for the proposed curve; that is, coordinates of the curve (FPR, TPR) and accuracy measure, Area Under the Curve (AUC), which helps in explaining the variability of the curve and provides the sensitivity at a particular value of specificity and vice versa. The proposed methodology is supported by a real data set and simulation studies.

#### 1. Introduction

In classification analysis, the Receiver Operating Characteristic (ROC) curve is a widely used tool to evaluate the performance of a test. Further, the intrinsic measures such as sensitivity, specificity, and accuracy are essential to describe a diagnostic test’s ability to classify an individual into one of the two groups/populations. Sensitivity provides an estimate of how good the test is at predicting a disease. Specificity estimates how likely patients without disease can be correctly identified. ROC curve is a graphical representation of 1 − specificity and sensitivity. That is, the points of the curve are obtained by moving the classification threshold from the most positive classification value to the most negative. For a random classification, the ROC curve is a straight line connecting the origin to top right corner of the graph . Further, the accuracy measure is defined as the area under the ROC curve. Therefore, the criterion widely used to measure the accuracy of a test in ROC context is the area under an ROC curve (AUC).

In classification, the main aim is to discriminate between normal and abnormal populations with better accuracy. In the literature so far many ROC models exist based on bidistributional assumptions such as binormal (Egan [1]), bilogistic and bilognormal (Dorfman and Alf Jr. [2, 3]), bibeta and biexponential (Zou et al. [4]; Tang et al. [5]; Tang and Balakrishnan [6]), and bigamma etcetera (Hussain [7]). If the test scores of normal and abnormal populations follow different distributions, then these ROC forms will not produce reliable outputs. For instance, consider that a marker, namely, APACHE (Acute Physiology and Chronic Health Evaluation) II, is used to predict the mortality status of patients who gets admitted into ICU. The pattern of APACHE scores for live and dead patient’s does not possess the normality and explains skewed nature of the data. Here, the conventional binormal ROC model will fail to produce reliable outputs in terms of AUC, threshold, sensitivity, and specificity. However, the distribution of scores may follow any skewed distributions. Hence, the main concentration of the paper lies in handling the situations when distributions of two populations are different and the data skewed nature of the data. We propose an ROC model that takes into account Generalized Half Normal (normal population) and Weibull (abnormal population) distribution with shape and scale parameters. In medical, engineering, and life studies, data tend to have extended tails; in this situation, the conventional binormal ROC curve fails to explain the hidden accuracy of the test considered. Recently, Balaswamy et al. [8] addressed this issue and developed a Hybrid ROC (HROC) curve which is based on Half Normal and Exponential distributions. However, this model is restricted by considering only scale parameters to illustrate the accuracy. But there are other statistical measures which accounts the information about the tail property of the data. In this paper, an extended version of the HROC curve is proposed by considering the Generalized Half Normal and Weibull distributions with both scale and shape parameters corresponding to normal as well as abnormal populations. A bootstrap study is used to construct the 95% confidence intervals and other measures of the proposed ROC curve. Further, the proposed methodology is demonstrated using simulation studies as well as a real data set.

The present paper is organized as follows. The ROC curve is developed based on Generalized Half Normal (GHN) and Weibull distributions with scale () and shape () parameters of both functions and GHROC curve accuracy measure, Area Under the Curve, is derived. Further, the confidence intervals for AUC and proposed ROC curve are estimated through bootstrap method. Finally, the results obtained using proposed methodology are illustrated in Results and Discussion.

#### 2. Methodology

Let be the test scores, which are observed in normal () and abnormal () populations, respectively. Here, it is assumed that and populations follow Generalized Half Normal (GHN) and Weibull distributions with shape and scale parameters as and , respectively. The probability density function and cumulative distribution function of GHN (Cooray and Ananda [9]) and Weibull distributions are given as follows:where is the c.d.f. of the standard normal distribution:In classification, ROC curve is a graphical plot that illustrates the performance of a binary classifier as its discrimination threshold varies (Green and Swets [10]). The curve is obtained by plotting the false positive rate (FPR) against the true positive rate (TPR).

The expression for FPR is derived by using its probabilistic definition ason further simplification, the expression for can be obtained by the formulawhere is the inverse cumulative standard normal distribution function.

Similarly, the expression for TPR is derived by using its probabilistic definition from Weibull distribution ason substituting (4) into (6), the expression for TPR can be written ashere , , and (7) is the expression of ROC Curve based on Generalized Half Normal and Weibull distributions. This expression (7) can be referred to as the* Generalized Hybrid ROC (GHROC) curve*, since the ROC curve is developed based on two generalized distributions.

In ROC methodology, the statistical measure which helps in explaining the overlapping area and the accuracy of a classifier is the Area Under the Curve (AUC). It can be interpreted as the probability that a subject randomly selected from the group with the condition will have a discriminating score indicating greater likelihood than that of a randomly selected subject from the group without condition (Bamber [11]). The AUC can take values between 0 and 1 with practical lower bound value of 0.5 (chance diagonal). The expression for the accuracy measure AUC can be obtained by integrating the ROC expression (7) over the range with respect to the false positive rate asThe above expression has no closed form; hence it has to be solved using numerical integration. In the next subsection, the variance and confidence intervals for AUC are estimated through bootstrapping method.

##### 2.1. Confidence Intervals for AUC

The confidence interval for AUC can be defined aswhere is the standard normal percentile and is the estimated variance of , which is obtained using bootstrapping. Let “” be the number of bootstraps obtained from the data with the sample sizes and , respectively, from normal and abnormal populations. Then the bootstrapped AUC estimate and its variance are where is the th bootstrap estimate of AUC. The next subsection deals with the construction of confidence intervals for the proposed ROC curve to explain the variability of the curve at each and every threshold value.

##### 2.2. Confidence Intervals for GHROC Curve

The confidence intervals for the GHROC curve are estimated using delta method. This confidence interval for the ROC Curve represents the range at each point of false positive rate and its corresponding true positive rate. Therefore, the confidence intervals for FPR and TPR are as follows:where and are the estimated FPR and TPR, respectively, and their variances areFurther, the confidence intervals for FPR and TPR can be obtained using the following expression: (for complete proof, refer to appendix). These confidence interval lines show the variability of the proposed ROC curve at each and every point on the ROC curve.

In the next section, the results are carried out using simulation studies and a real data set to explain the proposed methodology. Further, the confidence intervals are evaluated for the summary measure AUC and the intrinsic measures FPR and TPR.

#### 3. Results and Discussion

The proposed methodology is demonstrated using simulation studies and real data set (SAPS III).

##### 3.1. Simulation Studies

Simulation studies are conducted with different combinations of scale and shape parameters of both normal and abnormal populations and the entire simulations are done at various sample sizes with bootstraps. At every parameter combination and sample size, the AUC and its confidence intervals are obtained. The main purpose of conducting simulations is to show how the AUC of GHROC curve possesses different values as the scale and shape parameters of the normal and abnormal distributions change. The variations in the parameter values of both populations are used to explain the overlapping area in terms of AUC; this mean that the higher the AUC, the lesser the overlapping area and vice versa. Further, to demonstrate the behavior of AUC, the entire simulation work is carried out with three different experiments. In the first experiment, the shape parameter of abnormal population is varied by fixing the other parameters as constant; in second experiment, the scale parameter of abnormal population is varied by fixing the other parameters as constant and, in the third experiment, the shape parameters of both populations are considered to be equal with varying scale in abnormal population. The results so obtained from these experiments are reported in Table 1.