Symmetry Properties of Bi-Normal and Bi-Gamma Receiver Operating Characteristic Curves are Described by Kullback-Leibler Divergences

Abstract

Receiver operating characteristic (ROC) curves have application in analysis of the performance of diagnostic indicators used in the assessment of disease risk in clinical and veterinary medicine and in crop protection. For a binary indicator, an ROC curve summarizes the two distributions of risk scores obtained by retrospectively categorizing subjects as cases or controls using a gold standard. An ROC curve may be symmetric about the negative diagonal of the graphical plot, or skewed towards the left-hand axis or the upper axis of the plot. ROC curves with different symmetry properties may have the same area under the curve. Here, we characterize the symmetry properties of bi-Normal and bi-gamma ROC curves in terms of the Kullback-Leibler divergences (KLDs) between the case and control distributions of risk scores. The KLDs describe the known symmetry properties of bi-Normal ROC curves, and newly characterize the symmetry properties of constant-shape and constant-scale bi-gamma ROC curves. It is also of interest to note an application of KLDs where their asymmetry—often an inconvenience—has a useful interpretation.

1. Introduction

Receiver operating characteristic (ROC) curve analysis provides a basis for describing the performance of a diagnostic indicator when deployed in a binary diagnostic test. ROC curve analysis has found application in clinical medicine, veterinary medicine and crop protection (e.g., [1,2,3]). For a comprehensive overview of the methodology, see [4,5].

For the purpose of the present work, an outline description of the process by which an ROC curve may be derived allows us to introduce our terminology and notation. We refer generically to data provided by the diagnostic indicator as “risk scores”. During the process of characterizing a diagnostic indicator, a risk score is recorded for each of a number of experimental subjects. Each subject is also classified definitively as either a “case” (e.g., subject is diseased) or a “control” (e.g., subject is healthy) by a gold standard assessment (independent of the putative indicator). The ultimate goal of the experimental procedure as described is to provide a basis for decision-making in practice that does not require reference to the gold standard. When the decision in question is binary, an ROC curve is a useful summary of the performance of the diagnostic indicator [6].

We now have a number of subjects, and two values for each: a risk score provided by means of the diagnostic indicator and the true status (case or control) provided by the gold standard. We can present the results graphically as frequency distributions of risk scores plotted separately for cases and controls. It is normal practice to calibrate the output of the diagnostic indicator so that higher risk scores tend to be associated with case status, and lower risk scores tend to be associated with control status. Typically, then, the mean of the distribution of risk scores for cases is larger than the mean of the distribution of risk scores for controls.

An ROC curve is, in essence, a summary of the (normalized) frequency distributions of risk scores for cases and controls. In this article, we are concerned with the properties of ROC curves based on continuous parametric models for the distributions of risk scores (e.g., [5,7]). In practice, then, model parameters must be estimated from the experimental data; we do not describe this part of the analysis. For a continuous indicator variable X we refer to the resulting probability density functions (pdfs) as f₁(x) (for cases) and f₂(x) (for controls). The corresponding cumulative distribution functions (cdfs) are F₁(x) and F₂(x), respectively.

Now, consider the graphical plot of the pdfs of risk scores plotted separately for cases and controls. A diagnostic indicator and a threshold risk score together constitute a diagnostic test. In the process of developing a diagnostic test, our task is to characterize a threshold on the risk score scale such that subjects with risk scores above the threshold will be treated, and subjects with risk scores at or below the threshold will not be treated. The problem is that, typically, the distributions of risk scores for cases and controls overlap, so that there is no unequivocal “best” threshold risk score. Consider a particular choice of threshold risk score, and recall that we are working with the pdfs of risk scores for cases and controls. The proportion of cases correctly classified is the true positive proportion (TPP) and the proportion of controls correctly classified is the true negative proportion (TNP). The false negative proportion is FNP = 1 − TPP and the false positive proportion is FPP = 1 − TNP. The values of these proportions change with the choice of threshold risk score.

An ROC curve is a graphical plot of TPP [=1 − F₁(x)] against FPP [=1 − F₂(x)], with pairs of TPP and FPP values obtained by allowing a single threshold risk score to vary over the range of the indicator variable. Thus, points along the curve represent potential thresholds on the scale of the indicator variable, from each of which a binary test may be characterized. An ROC curve can therefore provide a useful summary of the characteristics of an indicator variable used as the basis for a binary test. Depending on the choice of model for risk scores for cases and controls, it may be possible to write down an analytical equation for the ROC curve, but this is immaterial in the present context. ROC curves that are monotone increasing above the main diagonal of the plot over the whole domain are sometimes referred to as “proper” ROC curves (see, e.g., Section 4.6 in [1]). Some continuous parametric ROC curves are proper, some are not; for example, it is well-known that the bi-Normal ROC curve is not in general proper, while the bi-gamma ROC curve is proper [8].

While on the one hand the ROC curve represents a summary of the distributions of risk scores for cases and controls, on the other there are methods by which a summary of the ROC curve itself is sought [7,9]. By far the most common single-figure ROC curve summary measure in use is the area under the curve (AUC) as an index of diagnostic accuracy (e.g., [10]). Briefly, the idea is that diagnostic indicators with ROC curves which pass close to the top left-hand corner of the graphical plot of TPP against FPP (high AUC) provide tests for which TPP and TNP are both high, offering good discrimination between cases and controls. Diagnostic indicators with ROC curves close to the main diagonal of the plot of TPP against FPP (low AUC) have little to offer in terms of discrimination between cases and controls. However, in the present context, the AUC is unsuitable for use in the description of the symmetry properties of ROC curves. It is not difficult to see that ROC curves with the same AUC may have different symmetry properties (e.g., Figure 2A in [11]; Figure 2 in [12]).

This article describes the symmetry properties of some parametric ROC curves based on continuous distributions. The article is set out as follows. The generic symmetry properties of ROC curves are described graphically. The application of the Kullback-Leibler divergence is outlined within the context of the present work. Some useful properties of the Pareto distribution are illustrated. The symmetry properties of bi-Normal, bi-exponential and bi-gamma ROC curves are analyzed. A general discussion is provided.

2. Analytical Background

2.1. Geometric Symmetry of ROC Curves

Geometric symmetry of ROC curves refers to an axis of symmetry that is the negative diagonal of the ROC plot (i.e., the line TPP = TNP). Green and Swets [13], Killeen and Taylor [14] and Hughes [15] have discussed the conditions for symmetry of ROC curves. However, ROC curves may be asymmetrical (skewed)—for example, the curve may “cling to the left edge of the ROC space longer than it does to the top” [16]. We refer to this kind of skew as TPP-asymmetry, and to the kind of skew where the curve clings to the top edge of the ROC space longer than it does to the left as TNP-asymmetry [17]. Figure 1 provides graphical definitions of these symmetry and asymmetry properties.

Figure 1. Graphical description of symmetric and asymmetric ROC curves. The dotted lines show, for reference, TPP = 1 − FPP (the negative diagonal) and the lines FPP = a (vertical) and TPP = 1 − a (horizontal). The FPP coordinate of point A = a, and the FPP coordinate of point C = a*, such that a < a*. The solid line is a symmetric ROC curve passing through the points A (a, b) and B (a₁, b₁) (such that a₁ = 1 − b, b₁ = 1 − a). Point C (a*, 1 − a*) also lies on the symmetric ROC curve. Asymmetries are defined by reference to the symmetric curve passing through point A, as follows. The dashed line is a TPP-asymmetric ROC curve passing through the points A (a, b) and D (a₂, b₂) (such that a₂ > 1 − b, b₂ = 1 − a). The dot-dashed line is a TNP-asymmetric ROC curve passing through the points A (a, b) and E (a₃, b₃) (such that a₃ < 1 − b, b₃ = 1 − a).

Figure 1. Graphical description of symmetric and asymmetric ROC curves. The dotted lines show, for reference, TPP = 1 − FPP (the negative diagonal) and the lines FPP = a (vertical) and TPP = 1 − a (horizontal). The FPP coordinate of point A = a, and the FPP coordinate of point C = a*, such that a < a*. The solid line is a symmetric ROC curve passing through the points A (a, b) and B (a₁, b₁) (such that a₁ = 1 − b, b₁ = 1 − a). Point C (a*, 1 − a*) also lies on the symmetric ROC curve. Asymmetries are defined by reference to the symmetric curve passing through point A, as follows. The dashed line is a TPP-asymmetric ROC curve passing through the points A (a, b) and D (a₂, b₂) (such that a₂ > 1 − b, b₂ = 1 − a). The dot-dashed line is a TNP-asymmetric ROC curve passing through the points A (a, b) and E (a₃, b₃) (such that a₃ < 1 − b, b₃ = 1 − a).

2.2. Kullback-Leibler Divergences

For a continuous indicator variable X we denote pdfs f₁(x) (for cases) and f₂(x) (for controls). Then the Kullback-Leibler divergences (KLDs) [18] are I(f₁,f₂) (with cases as the comparison distribution and controls as the reference distribution):

$$I\left(f_1,f_2\right)={\displaystyle {\int }_D{f}_1}\left(x\right)\ln \left[\frac{f_1\left(x\right)}{f_2\left(x\right)}\right]\mathrm{d}x$$

(1)

and I(f₂,f₁) (with controls as the comparison distribution and cases as the reference distribution):

$$I\left(f_2,f_1\right)={\displaystyle {\int }_D{f}_2}\left(x\right)\ln \left[\frac{f_2\left(x\right)}{f_1\left(x\right)}\right]\mathrm{d}x$$

(2)

where D is the common support of f₁ and f₂.

From Cover and Thomas [19] (who refer to continuous KLDs as differential relative entropies) we note that I(f₁,f₂) and I(f₂,f₁) ≥ 0, with equality only if f₁(x) and f₂(x) are identical. Typically, I(f₁,f₂) ≠ I(f₂,f₁) [10] although for an ROC curve based on f₁(x) (for cases) and f₂(x) (for controls) that is symmetric about the negative diagonal, I(f₁,f₂) = I(f₂,f₁) [17]. A KLD can be interpreted as a kind of distance between probability distributions [19], although the asymmetry in its arguments (apart from some special cases) clearly indicates it is not a distance in the Euclidian sense. We will work in natural logarithms, so the KLDs are denominated in nits [20]. For a discussion of measures of distance between distributions as used in summarizing ROC curves, see Section 4.3.4 in [1].

2.3. The Pareto Distribution

Now, without for the moment invoking any ROC-related context, consider the Pareto densities:

$$f_1\left(x\right)=\frac{1}{{\lambda }_1}{\left(x+1\right)}^{-\left(\left(1/{\lambda }_1\right)+1\right)}$$

(3)

$$f_2\left(x\right)=\frac{1}{{\lambda }_2}{\left(x+1\right)}^{-\left(\left(1/{\lambda }_2\right)+1\right)}$$

(4)

$x>0;{\lambda }_i>0,i=1,2$. Following Ullah (Equation (3) in [21]) we obtain KLDs for two Pareto distributions, as follows:

$$I\left(f_1,f_2\right)=\frac{{\lambda }_1}{{\lambda }_2}-1-\ln \left(\frac{{\lambda }_1}{{\lambda }_2}\right)$$

(5)

$$I\left(f_2,f_1\right)=\frac{{\lambda }_2}{{\lambda }_1}-1-\ln \left(\frac{{\lambda }_2}{{\lambda }_1}\right).$$

(6)

Let $z={\lambda }_2/{\lambda }_1$; since $z-1\ge \ln \left(z\right)$ with equality only if z = 1 (lemma 6.1 in [21]) we have both I(f₁,f₂) and I(f₂,f₁) ≥ 0, with equality only if f₁(x) and f₂(x) are identical (i.e., if ${\lambda }_1={\lambda }_2$), as required. Figure 2A shows the graphical plots of the two Pareto KLDs $I\left(f_1,f_2\right)=g_1\left(z\right)=\frac{1}{z}-1-\ln \left(\frac{1}{z}\right)$ and $I\left(f_2,f_1\right)=g_2\left(z\right)=z-1-\ln \left(z\right)$, from which it appears that (for z > 0):

• $I\left(f_1,f_2\right)>I\left(f_2,f_1\right)$ when z < 1,
• $I\left(f_1,f_2\right)=I\left(f_2,f_1\right)$ when z = 1,
• $I\left(f_1,f_2\right)<I\left(f_2,f_1\right)$ when z > 1.

It turns out to be easier to characterize the inequality portrayed in Figure 2A if we calculate the derivatives $g_1^{/}\left(z\right)=\frac{z-1}{z^2}$, $g_2^{/}\left(z\right)=\frac{z-1}{z}$. Now it is not difficult to see that for z > 0, $g_2^{/}\left(z\right)\ge g_1^{/}\left(z\right)$ with equality only if z = 1 (see also Figure 2B), and that this inequality describes the relationship between I(f₁,f₂) and I(f₂,f₁) shown in Figure 2. We will use these results on the Pareto distribution in the following sections.

Figure 2. (A). The figure shows graphical plots of Kullback-Leibler divergences for two Pareto densities: $g_1\left(z\right)=I\left(f_1,f_2\right)=\left(1/z\right)-1-\ln \left(1/z\right)$ (the solid line), and $g_2\left(z\right)=I\left(f_2,f_1\right)=z-1-\ln \left(z\right)$ (the dashed line), with $z={\lambda }_2/{\lambda }_1$. (B). The derivatives $g_1^{/}\left(z\right)=\left(z-1\right)/z^2$ (the solid line) and $g_2^{/}\left(z\right)=\left(z-1\right)/z$ (the dashed line).

Figure 2. (A). The figure shows graphical plots of Kullback-Leibler divergences for two Pareto densities: $g_1\left(z\right)=I\left(f_1,f_2\right)=\left(1/z\right)-1-\ln \left(1/z\right)$ (the solid line), and $g_2\left(z\right)=I\left(f_2,f_1\right)=z-1-\ln \left(z\right)$ (the dashed line), with $z={\lambda }_2/{\lambda }_1$. (B). The derivatives $g_1^{/}\left(z\right)=\left(z-1\right)/z^2$ (the solid line) and $g_2^{/}\left(z\right)=\left(z-1\right)/z$ (the dashed line).

3. The Bi-Normal ROC Curve

For a continuous indicator variable X we have $X~\text{N}\left({\mu }_1,{\sigma }_1^2\right)$ for cases and $X~\text{N}\left({\mu }_2,{\sigma }_2^2\right)$ for controls, in which ${\mu }_1\hspace{0.17em}\text{and}\hspace{0.17em}{\sigma }_1^2$ denote the mean and variance, respectively, of f₁(x); and ${\mu }_2\hspace{0.17em}\text{and}\hspace{0.17em}{\sigma }_2^2$ denote the mean and variance, respectively, of f₂(x). The indicator variable is calibrated so that μ₁ > μ₂.

First we consider the symmetric bi-Normal ROC curve. For such curves, the standard deviations of the case and control distributions are equal, ${\sigma }_1={\sigma }_2$ [1]. Also, for symmetric ROC curves in general, the KLDs are equal $I\left(f_1,f_2\right)=I\left(f_2,f_1\right)$ [17], and for the symmetric bi-Normal ROC curve in particular, $I\left(f_1,f_2\right)=I\left(f_2,f_1\right)=\frac{{\left({\mu }_1-{\mu }_2\right)}^2}{2{\sigma }^2}\left(>0\right);{\sigma }^2={\sigma }_1^2={\sigma }_2^2$ [23]. For a numerical example, consider Killeen and Taylor’s Figure 1 (top) in [14]. In this example, the distribution of risk scores for cases f₁(x) is Normal with mean μ₁ = 3.4 and standard deviation σ₁ = 1 and the distribution of risk scores for controls f₂(x) is Normal with mean μ₂ = 2 and standard deviation σ₂ = 1. The resulting ROC curve is geometrically symmetric [14] and I(f₁,f₂) = I(f₂,f₁) = 0.980 nits.

Asymmetric bi-Normal ROC curves are discussed by Green and Swets [13], Pepe [1] and Marzban [24]. In the terminology of the present article, bi-Normal ROC curves are TPP-asymmetric when ${\sigma }_2/{\sigma }_1<1$ and TNP-asymmetric when ${\sigma }_2/{\sigma }_1>1$. Writing in the context of applications of bi-Normal indicators in clinical epidemiology, Pepe [1] notes that the distribution of risk scores for controls is typically less dispersed than the distribution of risk scores for cases, in which case a typical bi-Normal ROC curve would be TPP-asymmetric (e.g., Figure 4.1 in [1], where ${\sigma }_2/{\sigma }_1=0.85$).

The KLDs are now (e.g., [15]):

$$I\left(f_1,f_2\right)=\frac{1}{2}\cdot \left[\frac{{\sigma }_1^2}{{\sigma }_2^2}-1-\ln \left(\frac{{\sigma }_1^2}{{\sigma }_2^2}\right)+\frac{{\left({\mu }_1-{\mu }_2\right)}^2}{{\sigma }_2^2}\right]$$

(7)

$$I\left(f_2,f_1\right)=\frac{1}{2}\cdot \left[\frac{{\sigma }_2^2}{{\sigma }_1^2}-1-\ln \left(\frac{{\sigma }_2^2}{{\sigma }_1^2}\right)+\frac{{\left({\mu }_1-{\mu }_2\right)}^2}{{\sigma }_1^2}\right].$$

(8)

Let $z=\frac{{\sigma }_2^2}{{\sigma }_1^2}$, and we can write these as:

$$I\left(f_1,f_2\right)=\frac{{\left({\mu }_1-{\mu }_2\right)}^2}{2{\sigma }_2^2}+\frac{1}{2}\cdot \left[\frac{1}{z}-1-\ln \left(\frac{1}{z}\right)\right]$$

(9)

$$I\left(f_2,f_1\right)=\frac{{\left({\mu }_1-{\mu }_2\right)}^2}{2{\sigma }_1^2}+\frac{1}{2}\cdot \left[z-1-\ln \left(z\right)\right].$$

(10)

We compare this with the situation when ${\sigma }_1={\sigma }_2$, $I\left(f_1,f_2\right)=I\left(f_2,f_1\right)$, and the ROC curve is symmetric. For TPP-asymmetry, we have ${\sigma }_2/{\sigma }_1<1$; then $\frac{{\left({\mu }_1-{\mu }_2\right)}^2}{2{\sigma }_2^2}>\frac{{\left({\mu }_1-{\mu }_2\right)}^2}{2{\sigma }_1^2}$ and (referring to Figure 2A) we can see that $\frac{1}{2}\cdot \left[\frac{1}{z}-1-\ln \left(\frac{1}{z}\right)\right]>\frac{1}{2}\cdot \left[z-1-\ln \left(z\right)\right]$. For TNP-asymmetry ${\sigma }_2/{\sigma }_1>1$; then $\frac{{\left({\mu }_1-{\mu }_2\right)}^2}{2{\sigma }_2^2}<\frac{{\left({\mu }_1-{\mu }_2\right)}^2}{2{\sigma }_1^2}$ and (referring again to Figure 2A) $\frac{1}{2}\cdot \left[\frac{1}{z}-1-\ln \left(\frac{1}{z}\right)\right]<\frac{1}{2}\cdot \left[z-1-\ln \left(z\right)\right]$. The inclusion of the factor ½ does not affect the inequality portrayed. Thus, for TPP-asymmetry, we have $I\left(f_1,f_2\right)>I\left(f_2,f_1\right)$ and for TNP-asymmetry, we have $I\left(f_1,f_2\right)<I\left(f_2,f_1\right)$. This is illustrated in Figure 3, using values of μ₁ and μ₂ from Killeen and Taylor (Figure 1 in [14]). We note also from Figure 3 that the point where the two curves intersect characterizes the symmetric ROC curve with I(f₁,f₂) = I(f₂,f₁) = 0.980 nits.

Figure 3. Analysis of a bi-Normal ROC curve. The graph shows the Kullback-Leibler divergences I(f₁,f₂) (the solid line) and I(f₂,f₁) (the dashed line) for two Normal densities; f₁(x) for cases has μ₁ = 3.4 and σ₁ is varied over a range that includes σ₁ = 1, and f₂(x) for controls has μ₂ = 2.0 and σ₂ = 1. When σ₂/σ₁ = 1, I(f₁,f₂) = I(f₂,f₁) and the corresponding ROC curve is symmetric about the negative diagonal. When σ₂/σ₁ < 1, I(f₁,f₂) > I(f₂,f₁) and the corresponding ROC curve is TPP-asymmetric; when σ₂/σ₁ > 1, I(f₂,f₁) > I(f₁,f₂) and the corresponding ROC curve is TNP-asymmetric.

Figure 3. Analysis of a bi-Normal ROC curve. The graph shows the Kullback-Leibler divergences I(f₁,f₂) (the solid line) and I(f₂,f₁) (the dashed line) for two Normal densities; f₁(x) for cases has μ₁ = 3.4 and σ₁ is varied over a range that includes σ₁ = 1, and f₂(x) for controls has μ₂ = 2.0 and σ₂ = 1. When σ₂/σ₁ = 1, I(f₁,f₂) = I(f₂,f₁) and the corresponding ROC curve is symmetric about the negative diagonal. When σ₂/σ₁ < 1, I(f₁,f₂) > I(f₂,f₁) and the corresponding ROC curve is TPP-asymmetric; when σ₂/σ₁ > 1, I(f₂,f₁) > I(f₁,f₂) and the corresponding ROC curve is TNP-asymmetric.

4. The Bi-Exponential ROC Curve

We deal with the bi-exponential ROC curve in passing, since it turns out to be a special case of the constant-shape bi-gamma ROC curve, below. Here, we have exponential densities for cases and controls (e.g., [25]), respectively:

$$f_1\left(x\right)=\frac{1}{{\lambda }_1}\exp \left(-\frac{x}{{\lambda }_1}\right)$$

(11)

$$f_2\left(x\right)=\frac{1}{{\lambda }_2}\exp \left(-\frac{x}{{\lambda }_2}\right)$$

(12)

$x>0;{\lambda }_i>0,i=1,2$. The indicator variable is calibrated so that the mean of the case distribution is larger than the mean of the control distribution, which requires ${\lambda }_1>{\lambda }_2$. A graphical plot of 1−F₁(x) against 1−F₂(x) then provides the ROC curve. Such ROC curves are TPP-asymmetric (as described in Figure 1) (see, e.g., Figure 1 in [25]).

Asadi et al. (Table 13.1 in [26]) provide a table of distributions related to the Pareto distribution by fixed transformation. Note that in the notation of Asadi et al. [26], our Pareto parameterization $\frac{1}{{\lambda }_i}={\beta }_i,i=1,2$. Then, following Asadi et al. [26], we obtain KLDs for two exponential distributions as follows:

$$I\left(f_1,f_2\right)=\frac{{\lambda }_1}{{\lambda }_2}-1-\ln \left(\frac{{\lambda }_1}{{\lambda }_2}\right)$$

(13)

$$I\left(f_2,f_1\right)=\frac{{\lambda }_2}{{\lambda }_1}-1-\ln \left(\frac{{\lambda }_2}{{\lambda }_1}\right).$$

(14)

Let $z={\lambda }_2/{\lambda }_1$ and refer to Figure 2A. For a useful ROC curve we require ${\lambda }_1>{\lambda }_2$, so we are only interested in the part of Figure 2A where z <1, and here we have $I\left(f_1,f_2\right)>I\left(f_2,f_1\right)$. For ${\lambda }_1={\lambda }_2$, the case and control distributions are identical, $I\left(f_1,f_2\right)=I\left(f_2,f_1\right)=0$, and the corresponding ROC curve follows the main diagonal of the plot; such diagnostic indicators offer no discrimination between cases and controls.

5. The Bi-Gamma ROC Curve

We start by writing a general gamma density:

$$f\left(x\right)=\frac{x^{r-1}}{{\lambda }^r\cdot \Gamma \left(r\right)}\exp \left(-\frac{x}{\lambda }\right)$$

(15)

$x>0;r,\lambda >0$. We refer to r as a shape parameter and λ as a scale parameter. Mathiassen et al. [27], Faraggi and Reiser [28], Faraggi et al. [29], and Hussain [30] use the same format. For two such gamma densities, respectively f₁(x) and f₂(x), we have X~gamma(x, r₁, λ₁) and X~gamma(x, r₂, λ₂) and the corresponding KLDs (e.g., [27]) are:

$$I\left(f_1,f_2\right)=\ln \left(\frac{\Gamma \left(r_2\right)\cdot {\lambda }_2^{r_2}}{\Gamma \left(r_1\right)\cdot {\lambda }_1^{r_1}}\right)+\left(r_1-r_2\right)\cdot \left(\mathsf{\Psi }\left(r_1\right)+\ln \left({\lambda }_1\right)\right)+r_1\cdot \left(\frac{{\lambda }_1}{{\lambda }_2}-1\right)$$

(16)

$$I\left(f_2,f_1\right)=\ln \left(\frac{\Gamma \left(r_1\right)\cdot {\lambda }_1^{r_1}}{\Gamma \left(r_2\right)\cdot {\lambda }_2^{r_2}}\right)+\left(r_2-r_1\right)\cdot \left(\mathsf{\Psi }\left(r_2\right)+\ln \left({\lambda }_2\right)\right)+r_2\cdot \left(\frac{{\lambda }_2}{{\lambda }_1}-1\right)$$

(17)

in which Γ(∙) is the gamma function and Ψ(∙) is the digamma function (the derivative of the logarithm of the gamma function, [31]). Here, we describe separately a constant-shape ROC curve and a constant-scale ROC curve.

5.1. The Constant-Shape Bi-Gamma ROC Curve

Here, $r_1=r_2=r,0<r<\infty ;{\lambda }_1\ge {\lambda }_2>0$. For f₁(x) and f₂(x) respectively, X~gamma(x, r, λ₁) and X~gamma(x, r, λ₂). The indicator variable is calibrated so that the mean of the case distribution is larger than the mean of the control distribution, which requires $r{\lambda }_1>r{\lambda }_2$. A graphical plot of 1−F₁(x) against 1−F₂(x) then provides the ROC curve. Such curves are TPP-asymmetric (as described in Figure 1). For example, see Dorfman et al. [8]. If r = 1, f₁(x) and f₂(x) are the same as for the bi-exponential ROC curve (above), and the symmetry properties then follow. Otherwise, the general gamma KLDs above simplify to:

$$I\left(f_1,f_2\right)=r\cdot \left[\frac{{\lambda }_1}{{\lambda }_2}-1-\ln \left(\frac{{\lambda }_1}{{\lambda }_2}\right)\right]$$

(18)

$$I\left(f_2,f_1\right)=r\cdot \left[\frac{{\lambda }_2}{{\lambda }_1}-1-\ln \left(\frac{{\lambda }_2}{{\lambda }_1}\right)\right]$$

(19)

and again, we can refer to Figure 2A (the inclusion of the (constant) factor r does not affect the inequality portrayed). For a useful ROC curve we require ${\lambda }_1>{\lambda }_2$, so we are only interested in the part of Figure 2A where z <1, and here we have $I\left(f_1,f_2\right)>I\left(f_2,f_1\right)$. For ${\lambda }_1={\lambda }_2$, the case and control distributions are identical, $I\left(f_1,f_2\right)=I\left(f_2,f_1\right)=0$, and the corresponding ROC curve follows the main diagonal of the plot; such diagnostic indicators offer no discrimination between cases and controls.

5.2. The Constant-Scale Bi-Gamma ROC Curve

Here, $r_1\ge r_2>0;\hspace{0.17em}\hspace{0.17em}{\lambda }_1={\lambda }_2=\lambda >0$. For simplicity, we follow Hanley [32] and Tang et al. [33] who have λ = 1. For f₁(x) and f₂(x) respectively, X~gamma(x, r₁, λ) and X~gamma(x, r₂, λ). The indicator variable is calibrated so that the mean of the case distribution is larger than the mean of the control distribution, which requires $r_1\lambda >r_2\lambda$. A graphical plot of 1−F₁(x) against 1−F₂(x) then provides the ROC curve. Such curves are TNP-asymmetric (as described in Figure 1). The general gamma KLDs above simplify to:

$$I\left(f_1,f_2\right)=\ln \left(\frac{\Gamma \left(r_2\right)}{\Gamma \left(r_1\right)}\right)+\left(r_1-r_2\right)\cdot \left(\mathsf{\Psi }\left(r_1\right)\right)$$

(20)

$$I\left(f_2,f_1\right)=\ln \left(\frac{\Gamma \left(r_1\right)}{\Gamma \left(r_2\right)}\right)+\left(r_2-r_1\right)\cdot \left(\mathsf{\Psi }\left(r_2\right)\right)$$

(21)

Now, we set r₂ = 1 and r₁ = ζ, ζ > 0. Figure 4A shows the graphical plots of:

• $I\left(f_1,f_2\right)=g_1\left(\zeta \right)=-\ln \left(\Gamma \left(\zeta \right)\right)+(\zeta -1)\cdot \mathsf{\Psi }\left(\zeta \right)$
• $I\left(f_2,f_1\right)=g_2\left(\zeta \right)=\ln \left(\Gamma \left(\zeta \right)\right)+\left(1-\zeta \right)\cdot \mathsf{\Psi }\left(1\right)$

from which it appears that (for ζ > 0):

• $I\left(f_1,f_2\right)>I\left(f_2,f_1\right)$ when ζ < 1,
• $I\left(f_1,f_2\right)=I\left(f_2,f_1\right)$ when ζ = 1,
• $I\left(f_1,f_2\right)<I\left(f_2,f_1\right)$ when ζ > 1.

On calculating the derivatives, $g_1^{/}\left(\zeta \right)=\left(\zeta -1\right)\cdot {\mathsf{\Psi }}^{\left(1\right)}\left(\zeta \right)$ in which ${\mathsf{\Psi }}^{\left(1\right)}(\cdot )$ is the trigamma function (the first derivative of the digamma function, [31]) and $g_2^{/}\left(\zeta \right)=\mathsf{\Psi }\left(\zeta \right)+\gamma$ in which γ is Euler’s constant (= 0.5772…) (see also Figure 4B). Recall that $\mathsf{\Psi }\left(1\right)=-\gamma$, then we have $g_1^{/}\left(1\right)=g_2^{/}\left(1\right)=0$, and the inequality portrayed in Figure 4 appears to have the same characteristics as that shown in Figure 2. Let $g_3\left(\zeta \right)=g_2^{/}\left(\zeta \right)-g_1^{/}\left(\zeta \right)=\mathsf{\Psi }\left(\zeta \right)+\gamma -\left(\zeta -1\right)\cdot {\mathsf{\Psi }}^{\left(1\right)}\left(\zeta \right)$. Then the derivative $g_3^{/}\left(\zeta \right)=-\left(\zeta -1\right)\cdot {\mathsf{\Psi }}^{\left(2\right)}\left(\zeta \right)$ in which ${\mathsf{\Psi }}^{\left(2\right)}(\cdot )$ is the tetragamma function (the second derivative of the digamma function, [31]). For $\zeta >0,{\mathsf{\Psi }}^{\left(2\right)}\left(\zeta \right)$ is negative [31], so (for ζ > 0):

• when ζ < 1, $g_3^{/}\left(\zeta \right)$ is negative and $g_3\left(\zeta \right)=g_2^{/}\left(\zeta \right)-g_1^{/}\left(\zeta \right)$ is decreasing,
• when ζ = 1, $g_3^{/}\left(1\right)$ is zero and $g_3\left(\zeta \right)$ is stationary,
• when ζ > 1, $g_3^{/}\left(\zeta \right)$ is positive and $g_3\left(\zeta \right)$ is increasing,

which shows that for $\zeta >0$, $g_3\left(\zeta \right)\ge 0$; i.e., $g_2^{/}\left(\zeta \right)\ge g_1^{/}\left(\zeta \right)$ with equality only if ζ = 1. This inequality describes the relationship between I(f₁,f₂) and I(f₂,f₁) shown in Figure 4.

Figure 4. Analysis of a constant scale bi-gamma ROC curve. (A). Graphical plots of Kullback-Leibler divergences: $g_1\left(\zeta \right)=I\left(f_1,f_2\right)=-\ln \left(\Gamma \left(\zeta \right)\right)+(\zeta -1)\cdot \mathsf{\Psi }\left(\zeta \right)$ (the solid line), and $g_2\left(\zeta \right)=I\left(f_2,f_1\right)=\ln \left(\Gamma \left(\zeta \right)\right)+\left(1-\zeta \right)\cdot \mathsf{\Psi }\left(1\right)$ (the dashed line), with r₂ = 1 and r₁ = ζ, ζ > 0. (B). The derivatives $g_1^{/}\left(\zeta \right)=\left(\zeta -1\right)\cdot {\mathsf{\Psi }}^{\left(1\right)}\left(\zeta \right)$ (the solid line) and $g_2^{/}\left(\zeta \right)=\mathsf{\Psi }\left(\zeta \right)+\gamma$ (the dashed line).

Figure 4. Analysis of a constant scale bi-gamma ROC curve. (A). Graphical plots of Kullback-Leibler divergences: $g_1\left(\zeta \right)=I\left(f_1,f_2\right)=-\ln \left(\Gamma \left(\zeta \right)\right)+(\zeta -1)\cdot \mathsf{\Psi }\left(\zeta \right)$ (the solid line), and $g_2\left(\zeta \right)=I\left(f_2,f_1\right)=\ln \left(\Gamma \left(\zeta \right)\right)+\left(1-\zeta \right)\cdot \mathsf{\Psi }\left(1\right)$ (the dashed line), with r₂ = 1 and r₁ = ζ, ζ > 0. (B). The derivatives $g_1^{/}\left(\zeta \right)=\left(\zeta -1\right)\cdot {\mathsf{\Psi }}^{\left(1\right)}\left(\zeta \right)$ (the solid line) and $g_2^{/}\left(\zeta \right)=\mathsf{\Psi }\left(\zeta \right)+\gamma$ (the dashed line).

For a useful ROC curve we require $r_1>r_2$, so we are only interested in the part of Figure 4A where ζ > 1, and here we have $I\left(f_1,f_2\right)<I\left(f_2,f_1\right)$. For $r_1=r_2$, the case and control distributions are identical, $I\left(f_1,f_2\right)=I\left(f_2,f_1\right)=0$, and the corresponding ROC curve follows the main diagonal of the plot; such diagnostic indicators offer no discrimination between cases and controls.

6. Discussion

For continuous parametric ROC curves, we can define symmetry conditions. Notwithstanding, it is sometimes rather difficult to tell from a graphical plot whether an empirical ROC curve is actually symmetrical or only approximately so (e.g., Figure 2 in [24]). It is harder to define asymmetry conditions for continuous parametric ROC curves, although often relatively easy to tell from a graphical plot when an empirical ROC curve is asymmetric (e.g., Figure 2 in [10]). Marzban [24] asks if asymmetry can be explained in terms of the underlying case and control distributions, and concludes that asymmetry in an ROC curve “can be attributed to unequal widths of the underlying distributions”. What is lacking is an independent assessment of asymmetry for comparison with the statistical assessment based on the relative dispersion of the case and control distributions. Here, we bring together a graphical definition of asymmetry (Figure 1) with an analysis of the KLDs for the case and control distributions for some examples of continuous parametric ROC curves.

The main findings are as follows. Bi-Normal ROC curves may be symmetric, TPP-asymmetric or TNP-asymmetric. For symmetric bi-Normal curves, we have $I\left(f_1,f_2\right)=I\left(f_2,f_1\right)$; for TPP-asymmetric curves, $I\left(f_1,f_2\right)>I\left(f_2,f_1\right)$; and for TNP-asymmetric curves, $I\left(f_1,f_2\right)<I\left(f_2,f_1\right)$. Of particular interest is the point of intersection of the two curves in Figure 3. The fact that at this point we have $I\left(f_1,f_2\right)=I\left(f_2,f_1\right)>0$ indicates the existence of symmetric curves that lie above the main diagonal of the bi-Normal ROC plot. This in itself is not surprising, of course, but it is noted here for reference below.

Bi-exponential ROC curves may only be TPP-asymmetric. For these TPP-asymmetric curves, $I\left(f_1,f_2\right)>I\left(f_2,f_1\right)$. In this case the KLDs are equal only when $I\left(f_1,f_2\right)=I\left(f_2,f_1\right)=0$ (referring to Figure 2), indicating that (unlike the bi-Normal case) there is no symmetric curve that lies above the main diagonal of the bi-exponential ROC plot.

Bi-gamma ROC curves may be TPP-asymmetric or TNP-asymmetric. A constant-shape bi-gamma ROC curve is always TPP-asymmetric and $I\left(f_1,f_2\right)>I\left(f_2,f_1\right)$. For the constant-scale bi-gamma ROC curve we considered the case of λ = 1. This is always TNP-asymmetric and $I\left(f_1,f_2\right)<I\left(f_2,f_1\right)$. In both cases (i.e., constant-shape and constant-scale) the KLDs are equal only when $I\left(f_1,f_2\right)=I\left(f_2,f_1\right)=0$ (referring to Figure 2A and Figure 4A), indicating that (unlike the bi-Normal case) there is no symmetric curve that lies above the main diagonal of the bi-gamma ROC plot.

The choice of operational threshold on an ROC curve amounts to specification of the error rates (FPP and FNP=1 − TPP) of the resulting diagnostic test. Recalling Figure 1 (for example), we can see that the symmetry properties of an ROC curve influence the trade-off between these error rates that is of interest in the process of choosing a threshold. ROC curve symmetry and both kinds of asymmetry are observed empirically in the study of disease diagnostics. This is beyond the scope of summaries based on area under curve calculations. As noted by Marzban [24], the ROC curve is a two-dimensional representation of a diagnostic indicator, so a single-figure summary measure cannot characterize all its properties. Further difficulties with the area under the ROC curve as a summary measure of performance of a diagnostic indicator are discussed in [34]. Our work so far, relating to continuous parametric ROC curves, indicates the following. First, although the KLD is usually not a symmetric quantity [35], it is noteworthy that for an ROC curve based on f₁(x) (for cases) and f₂(x) (for controls) that is symmetric about the negative diagonal, I(f₁,f₂) = I(f₂,f₁) [17]. Second, although the lack of symmetry of the KLD has been referred to as a nuisance in applications [36], in this particular study we find that the asymmetry of the KLD usefully characterizes the asymmetry of bi-Normal and bi-gamma ROC curves.