Abstract
This paper presents a Bayesian analysis of bivariate ordered probit regression model with excess of zeros. Specifically, in the context of joint modeling of two ordered outcomes, we develop zero-inflated bivariate ordered probit model and carry out estimation using Markov Chain Monte Carlo techniques. Using household tobacco survey data with substantial proportion of zeros, we analyze the socioeconomic determinants of individual problem of smoking and chewing tobacco. In our illustration, we find strong evidence that accounting for excess zeros provides good fit to the data. The example shows that the use of a model that ignores zero-inflation masks differential effects of covariates on nonusers and users.
1. Introduction
This paper is concerned with joint modeling of two ordered data outcomes allowing for excess zeros. Economic, biological, and social science studies often yield data on two ordered categorical variables that are jointly dependent. Examples include the relationship between desired and excess fertility [1, 2], helmet use and motorcycle injuries [3], ownership of dogs and televisions [4], severity of diabetic retinopathy of the left and right eyes [5], and self-assessed health status and wealth [6]. The underlying response variables could be measured on an ordinal scale. It is also common in the literature to generate a categorical or grouped variable from an underlying quantitative variable and then use ordinal response regression model (e.g., [4, 5, 7]). The ensuing model is usually analyzed using the bivariate ordered probit model.
Many ordered discrete data sets are characterized by excess of zeros, both in terms of the proportion of nonusers and relative to the basic ordered probit or logit model. The zeros may be attributed to either corner solution to consumer optimization problem or errors in recording. In the case of individual smoking behavior, for example, the zeros may be recorded for individuals who never smoke cigarettes or for those who either used tobacco in the past or are potential smokers. In the context of individual patents applied for by scientists during a period of five years, zero patents may be recorded for scientists who either never made patent applications or for those who do but not during the reporting period [8]. Ignoring the two types of zeros for nonusers or nonparticipants leads to model misspecification.
The univariate as well as bivariate zero-inflated count data models are well established in the literature for example, Lambert [9], Gurmu and Trivedi [10], Mullahy [11], and Gurmu and Elder [12]. The recent literature presents a Bayesian treatment of zero-inflated Poisson models in both cross-sectional and panel data settings (see [13, 14], and references there in). By contrast, little attention has been given to the problem of excess zeros in the ordered discrete choice models. Recently, an important paper by Harris and Zhao [15] developed a zero-inflated univariate ordered probit model. However, the problem of excess zeros in ordered probit models has not been analyzed in the Bayesian framework. Despite recent applications and advances in estimation of bivariate ordered probit models [1–6], we know of no studies that model excess zeros in bivariate ordered probit models.
This paper presents a Bayesian analysis of bivariate ordered probit model with excess of zeros. Specifically, we develop a zero-inflated ordered probit model and carry out the analysis using the Bayesian approach. The Bayesian analysis is carried out using Markov Chain Monte Carlo (MCMC) techniques to approximate the posterior distribution of the parameters. Bayesian analysis of the univariate zero-inflated ordered probit will be treated as a special case of the zero-inflated bivariate order probit model. The proposed models are illustrated by analyzing the socioeconomic determinants of individual choice problem of bivariate ordered outcomes on smoking and chewing tobacco. We use household tobacco prevalence survey data from Bangladesh. The observed proportion of zeros (those identifying themselves as nonusers of tobacco) is about 76% for smoking and 87% for chewing tobacco.
The proposed approach is useful for the analysis of ordinal data with natural zeros. The empirical analysis clearly shows the importance of accounting for excess zeros in ordinal qualitative response models. Accounting for excess zeros provides good fit to the data. In terms of both the signs and magnitudes of marginal effects, various covariates have differential impacts on the probabilities associated with the two types of zeros, nonparticipants and zero-consumption. The usual analysis that ignores excess of zeros masks these differential effects, by just focusing on observed zeros. The empirical results also show the importance of taking into account the uncertainty in the parameter estimates. Another advantage of the Bayesian approach to modeling excess zeros is the flexibility, particularly computational, of generalizing to multivariate ordered response models.
The rest of the paper is organized as follows. Section 2 describes the proposed zero-inflated bivariate probit model. Section 3 presents the MCMC algorithm and model selection procedure for the model. An illustrative application using household tobacco consumption data is given in Section 4. Section 5 concludes the paper.
2. Zero-Inflated Bivariate Ordered Probit Model
2.1. The Basic Model
We consider the basic Bayesian approach to a bivariate latent variable regression model with excess of zeros. To develop notation, let and denote the bivariate latent variables. We consider two observed ordered response variables and taking on values , for . Define two sets of cut-off parameters , , where the restrictions , , and have been imposed. We assume that follows a bivariate regression model where is a -variate of regressors for the th individual () and are the error terms. For subsequent analysis, let , , and Analogous to the univariate case, the observed bivariate-dependent variables are defined as where . Let .
We introduce inflation at the point , called the zero-zero state. As in the univariate case, define the participation model: In the context of the zero-inflation model, the observed response random vector takes the form We observe when either the individual is a non-participant () or the individual is a zero-consumption participant ( and ). Likewise, we observe positive outcome (consumption) when the individual is a positive consumption participant for at least one good ( and ).
Let and denote the respective cumulative distribution and probability density functions of standardized normal evaluated at . Assuming normality and that is uncorrelated with , but corr, and each component with unit variance, the zero-inflated bivariate ordered probit (ZIBOP) distribution is where . Further, for in (2.6), we have for so that where is the cdf for the standardized bivariate normal. Likewise, in (2.6) are given by The ensuing likelihood contribution for -independent observations is where if and , and otherwise. Here, the vector consists of , and the parameters associated with the trivariate distribution of .
Regarding identification of the parameters in the model defined by (2.1) through (2.5) with normality assumption, we note that the mean parameter (joint choice probability associate with the observed response vector ) depends nonlinearly on the probability of zero inflation () and choice probability () coming from the BOP submodel. Since the likelihood function for ZIBOP depends separately on the two regression components, the parameters of ZIBOP model with covariates are identified as long as the model is estimated by full maximum likelihood method. The same or different sets of covariates can affect the two components via and . When using quasi-likelihood estimation or generalized estimating equations methods rather than full ML, the class of identifiable zero-inflated count and ordered data models is generally more restricted; see, for example, Hall and Shen [16] and references there in. Although the parameters in the ZIBOP model above are identified through a nonlinear functional form estimated by ML, for more robust identification we can use traditional exclusion restrictions by including instrumental variables in the inflation equation, but excluding them from the ordered choice submodel. We follow this strategy in the empirical section.
About 2/3 of the observations in our tobacco application below have a double-zero-state, . Consequently, we focused on a mixture constructed from a point mass at and a bivariate ordered probit. In addition to allowing for inflation in the double-zero-state, our approach can be extended to allow for zero-inflation in each component.
2.2. Marginal Effects
It is common to use marginal or partial effects to interpret covariate effects in nonlinear models; see, for example, Liu et al. [17]. Due to the nonlinearity in zero-inflated ordered response models and in addition to estimation of regression parameters, it is essential to obtain the marginal effects of changes in covariates on various probabilities of interest. These include the effects of covariates on probability of nonparticipation (zero-inflation), probability of participation, and joint and/or marginal probabilities of choice associated with different levels of consumption.
From a practical point of view, we are less interested in the marginal effects of explanatory variables on the joint probabilities of choice from ZIBOP. Instead, we focus on the marginal effects associated with the marginal distributions of for . Define a generic (scalar) covariate that can be a binary or approximately continuous variable. We obtain the marginal effects of a generic covariate on various probabilities assuming that the regression results are based on ZIBOP. If is a binary regressor, then the marginal effect of on probability, say , is the difference in the probability evaluated at 1 and 0, conditional on observable values of covariates: . For continuous explanatory variables, the marginal effect is given by the partial derivative of the probability of interest with respect to , .
Regressor can be a common covariate in vectors of regressors and or appears in either or . Focusing on the continuous regressor case, the marginal effects of in each of the three cases are presented below. First, consider the case of common covariate in participation and main parts of the model, that is, in both and . The marginal effect on the probability of participation is given by where again is the probability density function (pdf) of the standard normal distribution and is the coefficient in the inflation part associated with variable . In terms of the zeros category, the effect on the probability of nonparticipation (zero inflation) is while represents the marginal effect on the probability of zero-consumption. Here the scalar is the coefficient in the main part of the model associated with .
Continuing with the case of common covariate, the marginal effects of on the probabilities of choice are given as follows. First, the total marginal effect on the probability of observing zero-consumption is obtained as a sum of the marginal effects in (2.11) and (2.12); that is, The effects for the remaining choices for outcomes are as follows:
Now consider case 2, where a generic independent variable is included only in , the main part of the model. In this case, covariate has obviously no direct effect on the inflation part. The marginal effects of on various choice probabilities can be presented as follows: with , , and . The marginal effects in (2.15) can be obtained by simply setting in (2.13) and (2.14).
For case 3, where appears only in , its marginal effects on participation components given in (2.10) and (2.11) will not change. Since in case 3, the partial effects of on various choice probabilities take the form: Again, we impose the restrictions , and .
As noted by a referee, it is important to understand the sources of covariate effects and the relationship between the marginal effects and the coefficient estimates. Since for , the total effect of a generic covariate on probability of consumption at level comes from two (weighted) sources: the participation part () and the main ordered probit part () such that with , and . This shows that is the same as —the participation effect in (2.18)—but is not necessarily the same as the sign of (). The latter is particularly true in the left tail of the distribution, where the coefficient () and the main (unweighted) effect in (2.19) have opposite signs because is negative. In this case, a positive effect coming from the main part requires to be negative. By contrast, is positive in the right tail, but can be positive or negative when the terms and are on the opposite sides of the mode of the distribution. This shows that a given covariate can have opposite effects in the participation and main models. Since the total effect of an explanatory variable on probability of choice is a weighted average of (2.18) and (2.19), interpretation of results should focus on marginal effects of covariates rather than the signs of estimated coefficients. This is the strategy adopted in the empirical analysis below.
2.3. A Special Case
Since the zero-inflated univariate ordered probit (ZIOP) model has not been analyzed previously in the Bayesian framework, we provide a brief sketch of the basic framework for ZIOP. The univariate ordered probit model with excess of zeros can be obtained as a special case of the ZIBOP model presented previously. To achieve this, let in the ZIBOP model and focus on the first ordered outcome with . In the standard ordered response approach, the model for the latent variable is given by (2.1) with . The observed ordered variable can be presented compactly as where is the indicator function equal to 1 or 0 according to whether or not. Again are unknown threshold parameters, where we set , , and .
Zero-inflation is now introduced at point . Using the latent variable model (2.4) for the zero inflation, the observed binary variable is given by , where if , and 0 otherwise. In regime 1, or for participants (e.g., smokers), while, in regime 0, or for nonparticipants. In the context of the zero-inflation model, the observed response variable takes the form . We observe when either the individual is a non-participant or the individual is a zero-consumption participant and . Likewise, we observe positive outcome (consumption) when the individual is a positive consumption participant .
Assume that and are independently distributed. Harris and Zhao [15] also consider the case where and are correlated. In the context of our application, the correlated model did not provide improvements over the uncorrelated ZIOP in terms of deviance information criterion. The zero-inflated ordered multinomial distribution, say , arises as a mixture of a degenerate distribution at zero and the assumed distribution of the response variable as follows: where, for any parameter vector associated with the distribution of , with . For simplicity, dependence on latent variables, covariates, and parameters has been suppressed on the right-hand side of (2.22). The likelihood based on -independent observations takes the form where, for example, , and if individual chooses outcome , or otherwise.
Different choices of the specification of the joint distribution of give rise to various zero-inflated ordered response models. For example, if the disturbance terms in the latent variable equations are normally distributed, we get the zero-inflated ordered probit model of Harris and Zhao [15]. The zero-inflated ordered logit model can be obtained by assuming that and are independent, each of the random variables following the logistic distribution with cumulative distribution function defined as . Unlike the ordered probit framework, the ordered logit cannot lend itself easily to allow for correlation between bivariate discrete response outcomes. Henceforth, we focus on the ordered probit paradigm in both univariate and bivariate settings.
Assuming that and are independently normally distributed, each with mean 0 and variance 1, the required components in (2.22) and consequently (2.23) are given by: The marginal effects for the univariate ZIOP are given by Harris and Zhao [15]. Bayesian analysis of the univariate ZIOP will be treated as a special case of the zero-inflated bivariate order probit model in the next section.
3. Bayesian Analysis
3.1. Prior Distributions
The Bayesian hierarchical model requires prior distributions for each parameter in the model. For this purpose, we can use noninformative conjugate priors. There are two reasons for adopting noninformative conjugate priors. First, we prefer to let the data dictate the inference about the parameters with little or no influence from prior distributions. Secondly, the noninformative priors facilitate resampling using Markov Chain Monte Carlo algorithm (MCMC) and have nice convergence properties. We assume noninformative (vague or diffuse) normal priors for regression coefficients , with mean and variance which are chosen to make the distribution proper but diffuse with large variances. Similarly, .
In choosing prior distributions for the threshold parameters, ’s, caution is needed because of the order restriction on them. One way to avoid the order restriction is to reparameterize them. Following Chib and Hamilton [18] treatment in the univariate ordered probit case, we reparameterize the ordered threshold parameters with the inverse map For , let so that . We choose normal prior without order restrictions for ’s.
The only unknown parameter associate with the distribution of in (2.1) and (2.4) is , the correlation between and . The values of by definition are restricted to be in the −1 to 1 interval. Therefore, the choice for prior distribution for can be uniform or a proper distribution based on reparameterization. Let denote the hyperbolic arc-tangent transformation of , that is, and taking hyperbolic tangent transformation of gives us back . Then parameter is asymptotically normal distributed with stabilized variance, , where is the sample size. We may also assume that .
3.2. Bayesian Analysis via MCMC
For carrying out a Bayesian inference, the joint posterior distribution of the parameters of the ZIBOP model in (2.6) conditional on the data is obtained by combining the likelihood function given in (2.9) and the above-specified prior distributions via Bayes’ theorem, as: where and the parameter vector now consists of , , and . Here ;;; are defined in (3.1), and are given via the inverse map (3.2).
Full conditional posterior distributions are required to implement the MCMC algorithm [19–22], and they are given as follows: (1)fixed effects:(a)zero state: (b)nonzero state: (2)thresholds: (3)bivariate correlation:
The MCMC algorithm simulates direct draws from the above full conditionals iteratively until convergence is achieved. A single long chain [23, 24] is used for the proposed model. Geyer [23] argues that using a single longer chain is better than using a number of smaller chains with different initial values. We follow this strategy in our empirical analysis.
The Bayesian analysis of the univariate ZIOP follows as a special case of that of the ZIBOP presented above. In particular, the joint posterior distribution of the parameters of the ZIOP model in (2.22) conditional on the data is obtained by combining the likelihood function given in (2.23) and the above-specified prior distributions (with modified notations) via Bayes' theorem, as follows: where, using notation of Section 2.3 for and the other parameter vectors, ;; , and . Apart from dropping the bivariate correlation, we basically replace the bivariate normal cumulative distribution by the univariate counterpart . Details are available upon request from the authors.
Apart from Bayesian estimation of the regression parameters, the posterior distributions of other quantities of interest can be obtained. These include posteriors for marginal effects and probabilities for nonparticipation, zero-consumption, and joint outcomes of interest. These will be considered in the application section. Next, we summarize model selection procedure.
The commonly used criteria for model selection like BIC and AIC are not appropriate for the multilevel models (in the presence of random effects), which complicates the counting of the true number of free parameters. To overcome such a hurdle, Spiegelhalter et al. [25] proposed a Bayesian model comparison criterion, called Deviance Information Criterion (DIC). It is given as where the “goodness-of-fit” is measured by the deviance for and complexity is measured by the “effective number of parameters”: that is, posterior mean deviance minus deviance evaluated at the posterior mean of the parameters. The DIC is then defined analogously to AIC as The idea here is that models with smaller DIC should be preferred to models with larger DIC. Models are penalized both by the value of , which favors a good fit, but also (similar to AIC and BIC) by the effective number of parameters . The advantage of DIC over other criteria, for Bayesian model selection, is that the DIC is easily calculated from the MCMC samples. In contrast, AIC and BIC require calculating the likelihood at its maximum values, which are not easily available from the MCMC simulation.
4. Application
4.1. Data
We consider an application to tobacco consumption behavior of individuals based on the 2001 household Tobacco Prevalence survey data from Bangladesh. The Survey was conducted in two administrative districts of paramount interest for tobacco production and consumption in the country. Data on daily consumption of smoking and chewing tobacco along with other socioeconomic and demographic characteristics and parental tobacco consumption habits were collected from respondents of 10 years of age and above. The data set has been used previously by Gurmu and Yunus [26] in the context of binary response models. Here we focus on a sample consisting of 6000 individual respondents aged between 10 and 101 years.
The ordinal outcomes used in this paper correspond roughly to zero, low, moderate, and high levels of tobacco consumption in the form of smoking or chewing tobacco , respectively. The first dependent variable for an individual's daily cigarette smoking intensities assumes the following 4 choices: if nonsmoker, if smoking up to 7 cigarettes per day, if smoking between 8 and 12 cigarettes daily, and if smoking more than 12 cigarettes daily; likewise, for the intensity of chewing tobacco, if reported not chewing tobacco, if uses up to 7 chewing tobacco, and if consuming 7 or more chewing tobacco. The frequency distribution of cigarette smoking and tobacco chewing choices in Table 1 shows that nearly 66% of the respondents identify themselves as nonusers of tobacco. Our modeling strategy recognizes that these self-identified current nonusers of tobacco may include either individuals who never smoke or chew tobacco (genuine nonusers) or those who do, but not during the reporting period (potential users of tobacco). For example, potential tobacco users may include those who wrongly claim to be nonusers, previous tobacco users that are currently nonusers, and those most likely to use tobacco in the future due to changes in, say, prices and income. Table 1 also shows that 76% of the respondents are non-smokers and nearly 87% identify themselves as nonusers of tobacco for chewing. Given the extremely high proportion of observed zeros coupled with sparse cells on the right tail, we employ the zero-inflated bivariate ordered probit framework.
Table 2 gives definition of the explanatory variables as well as their means and standard deviations. The respondents are more likely to be Muslim, married, in early thirties, live in rural area, and have about 7 years of formal schooling. Although the country is mostly agrarian, only around 11% of the respondents were related to agricultural occupation in either doing agricultural operations on their own farms or working as agricultural wage laborers. About 12% of the respondents belong to the service occupation. The benchmark occupational group consists of business and other occupations. More than one-half of the fathers and slightly less than two-thirds of the mothers of the respondents currently use or have used tobacco products in the past.
Among the variables given in Table 2, the two indicators of parental use of tobacco products are included in as part of the participation equation (2.4). The rest of the variables are included in and of (2.1) and (2.4). To allow for nonlinear effects, age and education enter all three equations using a quadratic form. Due to lack of data on prices, our analysis is limited to the study of other economic and demographic determinants of participation, smoking, and chewing tobacco.
4.2. Results
We estimate the standard bivariate ordered probit (BOP) and zero-inflated bivariate ordered probit regression models for smoking and chewing tobacco and report estimation results for parameters, marginal effects, and choice probabilities, along measures of model selection. An earlier version of this paper reports results from the standard ordered probit model as well as the uncorrelated and correlated versions of the univariate zero-inflated ordered probit model for smoking tobacco. Convergence of the generated samples is assessed using standard tools (such as trace plots and ACF plots) within WinBUGS software. After initial 10,000 burn-in iterations, every 10th MCMC sample thereafter was retained from the next 100,000 iterations, obtaining 10,000 samples for subsequent posterior inference of the unknown parameters. The slowest convergence is observed for some parameters in the inflation submodel. By contrast, the autocorrelations functions for most of the marginal effects die out quickly relative to those for the associated parameters.
Table 3 reports the goodness-of-fit statistics for the standard bivariate ordered probit model and its zero-inflated version, ZIBOP. The ZIBOP regression model clearly dominates BOP in terms of DIC and its components; compare the DIC of 11330 for the former and 11447 for the latter model. Table 4 gives posterior means, standard deviations, medians, and the 95 percent credible intervals (in terms of the 2.5 and 97.5 percentiles) of the parameters and choice probabilities from ZIBOP model. For comparison, the corresponding results from BOP are shown in Table 6 of the appendix. Both models predict significant negative correlation between the likelihood of smoking and chewing tobacco. The posterior estimates of the cut-off points are qualitatively similar across models. In what follows, we focus on discussion of results from the preferred ZIBOP model. The 95% credible interval for the correlation parameter from the zero-inflated model is in the range −0.25 to −0.12, indicating that smoking and chewing tobacco are generally substitutes. Results of selected predicted choice probabilities (bottom of Table 4) show that the ZIBOP regression model provides very good fit to the data. The posterior mean for the probability of (zero, zero)-inflation is about 24% while the 95% credible interval is [0.15, 0.32], indicating that a substantial proportion of zeros may be attributed to nonparticipants. These results underscore the importance of modeling excess zeros in bivariate ordered probit models.
To facilitate interpretation of results, we report in Tables 5 and 7 the same set of posterior estimates for the marginal effects from ZIBOP and BOP models, respectively. Since age and education enter the three equations non-linearly, we report the total marginal effects coming from the linear and quadratic parts. We examine closely the marginal effects on the unconditional marginal probabilities at all levels of smoking and chewing tobacco (; ). The marginal effects reported in Table 5 show that the results for covariates are generally plausible. Age has a negative impact on probabilities of moderate and heavy use of tobacco. For heavy smokers, education has a significant negative impact on the probability of smoking cigarettes. An additional year of schooling on average decreases probability of smoking by about 6.9% for heavy smokers. Among participants, being male or married has positive impact on probability of smoking, while the effects for being Muslim, urban resident, and student are largely negative. Male respondents are more likely to smoke cigarettes while women respondents are more likely to use chewing tobacco with heavy intensity, a result which is in line with custom of the country [26].
Using (2.13), we decompose the marginal effect on probability of observing zero-consumption into two components: the effect on nonparticipation (zero inflation) and zero-consumption. For each explanatory variable, this decomposition is shown in Table 5 in the first three rows for smoking and in rows 1, 7, and 8 for chewing tobacco. For most variables, the effects on probabilities of nonparticipation and zero-consumption are on average opposite in sign, but this difference seems to diminish at the upper tail of the distribution. For example, looking at the posterior mean for age under smoking, getting older by one more year decreases probability of nonparticipation by about 2.6% but increases probability of zero-consumption by 4.6%, implying a net increase of 2.0% in predicted probability of observing zero. The effect of age in the case of chewing tobacco is qualitatively similar, negative effect on genuine nonusers and positive effect on potential tobacco users, with the latter dominating in the overall effect.
Income has opposite effects on probability of nonparticipation and zero-consumption, predicting on average that tobacco is an inferior good for nonparticipants and a normal good for participants. However, the 95% credible interval contains zero, suggesting that the effect of income is weak. Generally, the opposing effects on probabilities of nonparticipation and zeroconsumption would have repercussions on both the magnitude and the statistical significance of the full effect of observing zero-consumption. Similar considerations apply to positive levels of consumption since the marginal effect on probability of observing consumption level can be decomposed into the marginal effects on (i) participation and (ii) levels of consumption conditional on participation, . These results show that policy recommendations that ignore excess zeros may lead to misleading conclusions.
5. Conclusion
In this paper we analyze the zero-inflated bivariate ordered probit model in a Bayesian framework. The underlying model arises as a mixture of a point mass distribution at for nonparticipants and the bivariate ordered probit distribution for participants. The Bayesian analysis is carried out using MCMC techniques to approximate the posterior distribution of the parameters. Using household tobacco survey data with substantial proportion of zeros, we analyze the socioeconomic determinants of individual problem of smoking and chewing tobacco. In our illustration, we find evidence that accounting for excess zeros provides very good fit to the data. The use of a model that ignores zero-inflation masks differential effects of covariates on nonusers and users at various levels of consumption, including zeros. The Bayesian approach to modeling excess zeros provides computational flexibility of generalizing to multivariate ordered response models as well as ordinal panel data models.
The proposed zero-inflated bivariate model is particularly useful when most of the bivariate ordered outcomes are zero . In addition to allowing for inflation in the double-zero state, our approach can be extended to allow for zero inflation in each component. If needed, other states in an ordered regression model may be inflated as well. These extensions need to be justified empirically on a case-by-case basis and are beyond the scope of this paper.
Appendices
A.
For more details see Tables 6 and 7.
B.
WinBUGS Code for Fitting the Proposed Models (see Algorithm 1).
| ||
Acknowledgments
The authors thank Alfonso Flores-Lagunes, the editor, two anonymous referees and seminar participants at the Conference on Bayesian Inference in Econometrics and Statistics, the Joint Statistical Meetings, the Southern Economics Association Conference, and Syracuse University for useful comments. Mohammad Yunus graciously provided the data used in this paper.