Methodology Used to Estimate the Bayes Significance Level for the Null Hypothesis of
No Change between the 2013-2014 and 2014-2015 Population Percentages
Comparisons between the state population percentages displayed in Tables 1 to 15 in this document are based on the 2013 through 2015 National Surveys on Drug Use and Health (NSDUHs). The moving average state estimates for the overlapping 2013-2014 and 2014-2015 time periods were obtained from independent applications of the survey-weighted hierarchical Bayes (SWHB) methodology; that is, the 2014-2015 models were fit independently of the previously fitted 2013-2014 models. This independent analysis approach was followed because there was no desire to revise the previously published 2013-2014 estimates. Moreover, the same fixed predictor variables were used in the 2013-2014 and 2014-2015 models, but annual updates were made when more current versions became available.1 The age group-specific fixed predictor variables were defined at five levels (namely, the person level, census block group level, tract level, county level, and state level). Also, each age group model had 51 state-level random effects and 250 "within-state" area-level random effects.2
To estimate change in state population percentages, let 
and 
denote the 2013-2014 and 2014-2015 prevalence rates, respectively, for state-s and age group-a. The change between and is defined in terms of the log-odds ratio (
) as opposed to the simple difference because the posterior distribution of the is closer to Gaussian than the posterior distribution of the simple difference (
). The is defined as
, D
where ln denotes the natural logarithm. The p value given is computed to test the null hypothesis of no change (i.e., 
or equivalently 
). An estimate of is given by
, D
where the 
represents the previously published 2013-2014 state estimates and the 
represents the 2014-2015 state estimates. To compute the variance of 
that is, 
let 
and 
, then
, D
where 
denotes the covariance between 
and 
. This covariance is defined in terms of the associated correlation as follows:
. D
Note that 
and 
used here to calculate are the same variances used in calculating the previously published 2013-2014 Bayesian confidence intervals and the 2014-2015 Bayesian confidence intervals, respectively.
The correlation between and was obtained by simultaneously modeling the 2013, 2014, and 2015 NSDUH data. This simultaneous modeling approach was adopted based on the results of the validation study3 conducted for measuring change in the 1999-2000 and 2000-2001 state population percentages. For this simultaneous model, 4 age groups (12 to 17, 18 to 25, 26 to 34, and 35 or older) by 3 years (2013, 2014, and 2015), that is, 12 subpopulation-specific models, were fitted, each with its own set of fixed and random effects. In this case, the general covariance matrices for the state and within-state random effects4 were 12 × 12 matrices corresponding to the 12 element (age group × year) vectors of random effects. Note that the survey-weighted, Bernoulli-type log likelihood employed in the SWHB methodology was appropriate for this simultaneous model because the 12 age group × year subpopulations were nonoverlapping. The correlation 
was approximated by the correlation calculated using the posterior distributions of 
and 
from the simultaneous model.
To calculate the p value for testing the null hypothesis of no difference (
), it is assumed that the posterior distribution of 
is normal with 
and 
. With (
), the Bayes p value or significance level for the null hypothesis of no difference is 
, where 
is a standard normal random variate, 
, and 
denotes the absolute value of 
. This Bayesian significance level (or p value) for the null value of , say 
, is defined following Rubin5 as the posterior probability for the collection of the values that are less likely or have smaller posterior density 
than the null (no change) value . That is, 
. With the posterior distribution of approximately normal, 
is given by the above expression.
