2014-2015
National Survey on Drug Use and Health:
Guide to State Tables and Summary of Small Area Estimation Methodology

Section A: Overview of NSDUH and Model-Based State Estimates

A.1 Introduction

This document provides information on the model-based small area estimates of substance use and mental disorders in states based on data from the combined 2014-2015 National Surveys on Drug Use and Health (NSDUHs). These estimates are available online along with other related information.¹ NSDUH is an annual survey conducted from January through December of the civilian, noninstitutionalized population aged 12 or older and is sponsored by the Substance Abuse and Mental Health Services Administration (SAMHSA). The survey collects information from individuals residing in households, noninstitutionalized group quarters (e.g., shelters, rooming houses, dormitories), and civilians living on military bases. In 2014-2015, NSDUH collected data from 135,974 respondents aged 12 or older and was designed to obtain representative samples from the 50 states and the District of Columbia. NSDUH is planned and managed by SAMHSA's Center for Behavioral Health Statistics and Quality (CBHSQ). Data collection and analysis are conducted under contract with RTI International.² A summary of NSDUH's methodology is given in Section A.2. Section A.3 lists all of the tables and files associated with the 2014-2015 state small area estimates and when and where they can be found. Information is given in Section A.4 on the confidence intervals and margins of error and how to make interpretations with respect to the small area estimates. Section A.5 discusses related substance use measures and warns users about not drawing conclusions by subtracting small area estimates from two different measures. Section A.6 discusses NSDUH questionnaire changes from 2015 and how these changes affect the 2014-2015 small area estimates.

The survey-weighted hierarchical Bayes (SWHB) estimation methodology used in the production of state estimates from the 1999 to 2014 surveys also was used in the production of the 2014-2015 state estimates. The SWHB methodology is described in Appendix E of the 2001 state report (Wright, 2003b) and in Folsom, Shah, and Vaish (1999). The goals and implementation of small area estimation (SAE) modeling remain the same and are described in Appendix E of the 2001 state report (Wright, 2003b). A general model description is given in Section B.1 of this document. A list of measures for which small area estimates are produced is given in Section B.2. Predictors used in the 2014-2015 SAE modeling are listed and described in Section B.3.

Small area estimates obtained using the SWHB methodology are design consistent (i.e., the small area estimates for states with large sample sizes are close to the robust design-based estimates). The state small area estimates when aggregated using the appropriate population totals result in national small area estimates that are very close to the national design-based estimates. However, to ensure internal consistency, it is desirable to have national small area estimates³ exactly match the national design-based estimates. The benchmarked state-level estimates are also potentially less biased than the unbenchmarked state-level estimates. Beginning in 2002, exact benchmarking was introduced, as described in Section B.4.⁴ Tables of the estimated numbers of individuals associated with each measure are available online,⁵ and an explanation of how these counts and their respective Bayesian confidence intervals⁶ are calculated can be found in Section B.5. Section B.6 discusses the method to compute aggregate estimates by combining two age groups. Section B.7 discusses the method to compare the estimates of a particular measure between two states. For all measures except major depressive episode (MDE, i.e., depression), serious mental illness (SMI), any mental illness (AMI), and past year serious thoughts of suicide, the age groups for which estimates are provided are 12 to 17, 18 to 25, 26 or older, 18 or older, and 12 or older.⁷ Estimates of underage (aged 12 to 20) alcohol use were also produced. Alcohol consumption is expected to differ significantly across the 18 to 25 age group because of the legalization of alcohol at age 21. Therefore, it was decided that it would be useful to produce small area estimates for individuals aged 12 to 20.

In Section C, the 2013, 2014, 2015, pooled 2013-2014, and pooled 2014-2015 survey sample sizes, population estimates, and response rates are included in Tables C.1 to C.14, respectively. Table C.15 lists all of the measures and the years for which small area estimates were produced going back to the 2002 NSDUH, and Table C.16 lists all of the measures by age groups for which small area estimates were produced. In addition, Table C.17 provides a summary of milestones implemented in the SAE production process from 2002 to 2015.

A.2 Summary of NSDUH Methodology

This section provides a brief overview of the NSDUH methodology, specifically the sample design. For additional details on NSDUH's methodology, see Section A.2 of the 2011-2012 state SAE methodology document.⁸

The 1999 through 2001 National Household Surveys on Drug Abuse (NHSDAs)⁹ and the 2002 through 2013 NSDUHs employed a 50-state design with an independent, multistage area probability sample for each of the 50 states and the District of Columbia. For the 50-state design, 8 states were designated as large sample states (California, Florida, Illinois, Michigan, New York, Ohio, Pennsylvania, and Texas) with target sample sizes of 3,600 per year. For the remaining 42 states and the District of Columbia, the target sample size was 900 per year. This approach ensured that there was sufficient sample in every state to support SAE while at the same time maintaining efficiency for national estimates. The design also oversampled youths and young adults, so that each state's sample was approximately equally distributed among three major age groups: 12 to 17 years, 18 to 25 years, and 26 years or older.

A coordinated design was developed for the 2014 through 2017 NSDUHs. Similar to the 1999 through 2013 surveys, the coordinated 4-year design is state-based with an independent, multistage area probability sample within each state and the District of Columbia. This design designates 12 states as large sample states. These 12 states have the following target sample sizes per year: 4,560 interviews in California; 3,300 interviews in Florida, New York, and Texas; 2,400 interviews in Illinois, Michigan, Ohio, and Pennsylvania; and 1,500 interviews in Georgia, New Jersey, North Carolina, and Virginia. Making the sample sizes more proportional to the state population sizes improves the precision of national NSDUH estimates. This change also allows for a more cost-efficient sample allocation to the largest states while slightly increasing the sample sizes in smaller states to improve the precision of state estimates (note that the target sample size per year in the small states is 960 interviews with the exception of Hawaii where the target sample size is 967 interviews). The fielded sample sizes for each state in 2015 are provided in Table C.5, and the combined 2014-2015 sample sizes are provided in Table C.9.

Starting in 2014, the allocation of the NSDUH sample is 25 percent for adolescents aged 12 to 17, 25 percent for adults aged 18 to 25, and 50 percent for adults aged 26 or older. The sample of adults aged 26 or older is further divided into three subgroups: aged 26 to 34 (15 percent), aged 35 to 49 (20 percent), and aged 50 or older (15 percent). For more information on the 2014 through the 2017 NSDUH sample design and for differences between the 2013 and 2014 surveys, refer to the 2014 NSDUH sample design report (CBHSQ, 2015b).

Nationally in 2014-2015, 259,815 addresses were screened, and 135,974 individuals responded within the screened addresses (see Table C.9). The screening response rate (SRR) for 2014-2015 combined averaged 80.8 percent, and the interview response rate (IRR) averaged 70.2 percent, for an overall response rate (ORR) of 56.8 percent (Table C.9). The ORRs for 2014-2015 ranged from 42.7 percent in New York to 72.8 percent in Utah. Estimates have been adjusted to reflect the probability of selection, unit nonresponse, poststratification to known census population estimates, item imputation, and other aspects of the estimation process. These procedures are described in detail in the 2013, 2014, and 2015 NSDUHs' methodological resource books (MRBs) (CBHSQ, 2014, 2015a, in press).

A.3 Presentation of Data

In addition to this methodology document for the 2014-2015 state SAE results, the following files are available at http://www.samhsa.gov/data/:

• 2014-2015 NSDUH: Model-Based Prevalence Estimates (50 States and the District of Columbia) (Tables 1 to 15, by Age Group): Tables of percentages and associated 95 percent Bayesian confidence intervals are included for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, adults 18 or older, and all individuals aged 12 or older. Also included are tables for underage (12 to 20) use of alcohol. These tables are available in Excel and PDF format. In order to increase the precision of small area estimates and rankings especially for small sample states and to detect year-to-year changes more efficiently, an SAE expert panel¹⁰ recommended producing annual state estimates based on 2 consecutive years of pooled NSDUH data and to base comparisons of estimates (to measure change) on 2-year moving averages.
• 2014-2015 NSDUH National Maps of Prevalence Estimates, by State (Figures 1a to 15d): The color of each state on these U.S. maps indicates how the state ranks relative to other states for each measure. States could fall into one of five groups according to their ranking by quintiles. Because 51 states were ranked for each measure, the middle quintile was assigned to 11 states, and the remaining quintiles were assigned 10 states each. In some cases, a "quintile" could have more or fewer states than desired because two (or more) states had the same estimate (to two decimal places). When such ties occurred at the "boundary" between two quintiles, all of the states with the same estimate were conservatively assigned to the lower quintile. Those states with the highest rates for a given measure are in red, and those states with the lowest estimates are in white.
  
  Note that because the average annual incidence of marijuana for the 26 or older age group and past year heroin use for youths aged 12 to 17 was so low and had such an abbreviated range, no U.S. map was included for them.
• 2014-2015 NSDUH State Estimates Categorized into Five Groups, by Age Group: This Excel table shows the ranges of percentages for each outcome categorized into five groups (used to form the U.S. maps described above) from the lowest to highest estimate for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, adults aged 18 or older, and all individuals aged 12 or older. Also included are ranges for underage (12 to 20) alcohol use.
• 2014-2015 NSDUHs: Model-Based Estimated Totals (in Thousands) (50 States and the District of Columbia) (Tables 1 to 15): Tables showing estimated numbers (counts in thousands) and confidence intervals are included for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, adults aged 18 or older, and all individuals aged 12 or older. Also included are tables for underage (12 to 20) alcohol use. These tables are available in Excel and PDF format.
• 2014-2015 NSDUH State-Specific Tables (Tables 1 to 112): Tables are provided for each individual state and the District of Columbia, as well as for the total United States and the four census regions (i.e., Northeast, Midwest, South, and West). The tables (two per area) show the percentages and the numbers of individuals (counts in thousands).
• NSDUH: Comparison of 2013-2014 and 2014-2015 Population Percentages (50 States and the District of Columbia) (Tables 1 to 15): Tables are presented that show the 2013-2014 (previously published data) and 2014-2015 NSDUH state estimates and an indication of the statistical significance of the difference or change (p value). Estimates are shown for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, adults aged 18 or older, and all individuals aged 12 or older. Also included are tables for underage (12 to 20) alcohol use. Because annual state-level estimates are based on 2 years of pooled NSDUH data, two consecutive sets of estimates have a 1-year overlap (e.g., 2013-2014 and 2014-2015). If the population totals across the 3 years (e.g., 2013, 2014, and 2015) were the same, then the null hypothesis of no difference between the log odds of the 2013-2014 and 2014-2015 prevalence rates would be equivalent to testing the null hypothesis that the difference between the 2013-2014 and 2014-2015 prevalence rates is zero, which in turn would be equivalent to testing that the difference between the 2013 and 2015 prevalence rates is zero. The methodology used to compare these percentages is provided at the end of the tables.
• NSDUH: Comparison of 2008-2009 and 2014-2015 Population Percentages (50 States and the District of Columbia) (Tables 1 to 14): Tables are presented that show the 2008-2009 and 2014-2015 NSDUH state estimates and an indication of the statistical significance of the difference or change (p value). Estimates are shown for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, adults aged 18 or older, and all individuals aged 12 or older. Also included are tables for underage (12 to 20) alcohol use. This comparison is done between the most recent estimates (in this case, 2014-2015) and the earliest comparable estimates for all outcomes including mental health (based on 2008-2009 NSDUH data). Note that for earlier comparison tables, current small area estimates were compared with small area estimates from 2002-2003. However, because NSDUH estimates for AMI, SMI, and past year thoughts of suicide were not generated until 2008-2009, it was decided to establish 2008-2009 as the new baseline for comparisons with 2014-2015 data and beyond in order to include them in these tables. The methodology used to compare these percentages is provided at the end of the tables.
• 2014-2015 NSDUH: Other Sources of State-Level Data: This document compares two outcomes (cigarette and alcohol use) from NSDUH with data from the Behavioral Risk Factor Surveillance System (BRFSS).

A.4 Confidence Intervals and Margins of Error

At the top of each of the 15 state model-based estimate tables¹¹ is the design-based national estimate along with a 95 percent design-based confidence interval, all of which are based on the survey design, the survey weights, and the reported data. The state estimates are model-based statistics (using SAE methodology) that have been adjusted (benchmarked) such that the population-weighted mean of the estimates across the 50 states and the District of Columbia equals the design-based national estimate. For more details on this benchmarking, see Section B.4. The region-level estimates are also benchmarked and are obtained by taking the population-weighted mean of the associated state-level benchmarked estimates. Associated with each state and regional estimate is a 95 percent Bayesian confidence interval. These intervals indicate the uncertainty in the estimate due to both sampling variability and model fit. For example, the state with the highest estimate of past month use of marijuana for young adults aged 18 to 25 was Vermont, with an estimate of 35.0 percent and a 95 percent confidence interval that ranged from 30.8 to 39.3 percent (see Table 2 of the state model-based estimates' tables). Assuming that sampling and modeling conditions held, the Bayes posterior probability was 0.95 that the true percentage of past month marijuana use in Vermont for young adults aged 18 to 25 in 2014-2015 was between 30.8 and 39.3 percent. As noted earlier in a Section A.1 footnote, the term "prediction interval" (PI) was used in the 2004-2005 NSDUH state report and prior reports to represent uncertainty in the state and regional estimates. However, that term also is used in other applications to estimate future values of a parameter of interest. That interpretation does not apply to NSDUH state model-based estimates, so PI was replaced with "Bayesian confidence interval."

Margin of error is another term used to describe uncertainty in the estimates. For example, if is a 95 percent symmetric confidence interval for the population proportion (p) and is an estimate of p obtained from the survey data, then the margin of error of is given by or . Because is a symmetric confidence interval, will be the same as . In this case, the probability is 0.95 that the interval ± or ± will contain the true population value (p). The margin of error defined above will vary for each estimate and will be affected not only by the sample size (e.g., the larger the sample, the smaller the margin of error), but also by the sample design (e.g., telephone surveys using random digit dialing and surveys employing a stratified multistage cluster design will, more than likely, produce a different margin of error) (Scheuren, 2004).

The confidence intervals shown in NSDUH reports are asymmetric, meaning that the distance between the estimate and the lower confidence limit will not be the same as the distance between the upper confidence limit and the estimate. For example, Utah's past month marijuana use estimate is 11.1 percent for adults aged 18 to 25 years, with a 95 percent confidence interval equal to (8.9 − 13.7) (see Table 2 of the state model-based estimates' tables). Therefore, Utah's estimate is 2.2 (i.e., 11.1 − 8.9) percentage points from the lower 95 percent confidence limit and 2.6 (i.e., 13.7 − 11.1) percentage points from the upper limit. These asymmetric confidence intervals work well for small percentages often found in NSDUH tables and reports while still being appropriate for larger percentages. Some surveys or polls provide only one margin of error for all reported percentages. This single number is usually calculated by setting the sample percentage estimate () equal to 50 percent, which will produce an upper bound or maximum margin of error. Such an approach would not be feasible in NSDUH because the estimates vary from less than 1 percent to over 75 percent; hence, applying a single margin of error to these estimates could significantly overstate or understate the actual precision levels. Therefore, given the differences mentioned above, it is more useful and informative to report the confidence interval for each estimate instead of a margin of error.

When it is indicated that a state has the highest or lowest estimate, it does not imply that the state's estimate is significantly higher or lower than the next highest or lowest state's estimate. Additionally, two significantly different state estimates (at the 5 percent level of significance) may have overlapping 95 percent confidence intervals. For details on a more accurate test to compare state estimates, see Section B.6.

A.5 Related Substance Use Measures

Small area estimates are produced for a number of related drug measures, such as marijuana use and illicit drug use. It might appear that one could draw conclusions by subtracting one from the other (e.g., subtracting the percentage who used illicit drugs other than marijuana in the past month from the percentage who used illicit drugs in the past month to find the percentage who only used marijuana in the past month). Because related measures have been estimated with different models (i.e., separate models by age group and outcome), subtracting one measure from another related measure at the state or census region level can give misleading results, perhaps even a "negative" estimate, and should be avoided. However, these comparisons can be made at the national level because these estimates are design-based estimates. For example, at the national level, subtracting cigarette use estimates from tobacco use estimates will give the estimate of individuals who did not use cigarettes, but used other forms of tobacco, such as cigars, pipes, and smokeless tobacco.

A.6 2015 NSDUH Changes and Their Effects on Small Area Estimates

In 2015, a number of changes were made to the NSDUH questionnaire and data collection procedures. These changes were intended to improve the quality of the data that were collected and to address the changing needs of substance use and mental health policy and research.¹² This section briefly summarizes the effect of the redesign on the comparability between the 2015 NSDUH and earlier NSDUHs, specifically related to the SAE outcomes. For a more detailed discussion of the questionnaire redesign and its effect, see Section C of the 2015 NSDUH's methodological summary and definitions report (CBHSQ, 2016a) and a brief report summarizing the implications of the changes for data users (CBHSQ, 2016b).

In the alcohol section of the questionnaire, the threshold for defining binge alcohol use among females was revised from five or more drinks on an occasion to four or more drinks on an occasion to ensure consistency with federal definitions.¹³ The threshold for males in 2015 remained at five or more drinks on an occasion. Consequently, a new baseline was established in 2015 for estimates of binge alcohol for the overall population. Thus, small area estimates for past month binge alcohol use using combined 2014 and 2015 data were not produced. Note that this change did not affect estimates for alcohol use or alcohol use disorder.

Several changes were made to the various illicit drug modules. Specifically, changes were made to the hallucinogen, inhalant, methamphetamine, and prescription psychotherapeutic modules. For details on these specific changes, see Section C.1 of the 2015 NSDUH methodological summary and definitions report (CBHSQ, 2016a). These changes resulted in the need to revise the baseline for the following SAE outcomes: illicit drug use in the past month, nonmedical use of pain relievers in the past year,¹⁴ illicit drug use disorder, and needing but not receiving treatment for illicit drugs.

Additionally, changes to some of the drug modules might have affected the set of respondents in 2015 who were eligible to be asked questions about treatment for substance use. Hence, SAE outcomes on needing but not receiving treatment (for illicit drugs and alcohol) were potentially affected. Thus, substance use treatment estimates were not produced using combined 2014 and 2015 NSDUH data.

Finally, although questions on the perceptions of risk of harm from using different substances did not change in 2015, data quality checks on preliminary data and the full 2015 data showed deviations from the expected trends for these measures. A survey redesign carries the risk that preceding changes to the questionnaire will affect how respondents answer later questions (e.g., context effects). A context effect may be said to take place when the response to a question is affected by information that is not part of the question itself. For example, the content of a preceding question may affect the interpretation of a subsequent question. Or a respondent may answer a subsequent question in a manner that is consistent with responses to a preceding question if the two questions are closely related to each other. The set of questions preceding the risk and availability module in the 2015 questionnaire had undergone a number of significant changes that could have affected the way in which respondents answered the perceived risk and availability questions. Because of these deviations, the perception of risk estimates were not produced using combined 2014 and 2015 NSDUH data.

To summarize, several changes in the 2015 questionnaire had impacts on the comparability of the 2014 and 2015 NSDUH data. It was decided, therefore, that for those measures data across those 2 years could not be pooled, and estimates for those measures could not be produced using 2014 and 2015 NSDUH data. For a complete list of outcomes for which small area estimates are available using 2014-2015 NSDUH data, refer to Section B.2.

Section B: State Model-Based Estimation Methodology

B.1 General Model Description

The model can be characterized as a complex mixed¹⁵ model (including both fixed and random effects) of the following form:

,     D

where is the probability of engaging in the behavior of interest (e.g., using marijuana in the past month) for person-k belonging to age group-a in grouped state sampling region (SSR)-j of state-i.¹⁶ Let denote a vector of auxiliary (predictor) variables associated with age group-a (12 to 17, 18 to 25, 26 to 34, and 35 or older) and denote the associated vector of the regression parameters. The age group-specific vectors of the auxiliary variables are defined for every block group in the nation and also include person-level demographic variables, such as race/ethnicity and gender. The vectors of state-level random effects and grouped SSR-level random effects are assumed to be mutually independent with and , where is the total number of individual age groups modeled (generally, ). For hierarchical Bayes (HB) estimation purposes, an improper uniform prior distribution is assumed for , and proper Wishart prior distributions are assumed for and . The HB solution for involves a series of complex Markov Chain Monte Carlo (MCMC) steps to generate values of the desired fixed and random effects from the underlying joint posterior distribution. The basic process is described in Folsom et al. (1999), Shah, Barnwell, Folsom, and Vaish (2000), and Wright (2003a, 2003b).

Once the required number of MCMC samples (1,250 in all) for the parameters of interest are generated and tested for convergence properties (see Raftery & Lewis, 1992), the small area estimates for each race/ethnicity × gender cell within a block group can be obtained for each age group. These block group-level small area estimates then can be aggregated using the appropriate population count projections for the desired age group(s) to form state-level small area estimates. These state-level small area estimates are benchmarked to the national design-based estimates as described in Section B.4.

B.2 Variables Modeled

The 2015 National Survey on Drug Use and Health (NSDUH) data were pooled with the 2014 NSDUH data, and age group-specific state estimates for 14 binary (0, 1) measures were produced for the following outcomes:

1. past year use of marijuana,
2. past month use of marijuana,
3. average annual rate of first use of marijuana,¹⁷
4. past year use of cocaine,
5. past year use of heroin,
6. past month use of alcohol,¹⁸
7. past month use of tobacco products,
8. past month use of cigarettes,
9. past year alcohol use disorder,
10. past year alcohol dependence,
11. serious mental illness (SMI) in the past year,¹⁹
12. any mental illness (AMI) in the past year,
13. serious thoughts of suicide in the past year, and
14. past year major depressive episode (MDE, i.e., depression).

Comparisons between the 2013-2014 and the 2014-2015 state estimates also were produced for all of these measures. For details on how measures on mental illness, dependence or abuse, and average annual rate of first use of marijuana are defined, see "2011-2012 National Surveys on Drug Use and Health: Guide to State Tables and Summary of Small Area Estimation Methodology" at http://www.samhsa.gov/data/. Note that data on past year heroin use are presented in the 2014-2015 state small area estimation (SAE) tables and maps for the first time. Also, as discussed in Section A.6, some measures are not comparable between 2014 and 2015 because of questionnaire changes in 2015. Therefore, these measures are omitted from this report. Table C.15 shows all of the SAE outcomes and the years they are available; thus, this table can be used to see outcomes for which small area estimates were produced using 2013-2014 NSDUH data, but are not available based on 2014-2015 data.

B.3 Predictors Used in Mixed Logistic Regression Models

Local area data used as potential predictor variables in the mixed logistic regression models were obtained from a number of sources, as noted in the following discussion. Note that the predictors used to produce the 2014-2015 state small area estimates were the same as the predictors used to produce the 2013-2014 state small area estimates; however, values of the data were updated when possible. No new variable selection was done for 2014-2015, with the exception of the heroin use outcome. Variable selection was done using combined 2014 and 2015 data for past year heroin use. Fixed-effect predictors for this new outcome variable were selected using the method described by Wright and Sathe (2005).

Sources and potential data items used in the 2014-2015 modeling are provided in the following text and lists.

• Nielsen Claritas. This demographic data package from Nielsen Claritas, a market research firm headquartered in San Diego, California, contains data for 2013 with projections to 2018. The population projections are used to update these predictor variables each year. The 2014 and 2015 population projections were used for producing the 2014-2015 state small area estimates.
• U.S. Census Bureau. The 2010 census (demographic and geographic variables) and 2013 food stamp participation estimates were used (http://www.census.gov/did/www/saipe/inputdata/cntysnap.xls). The Census Bureau's Small Area Income and Poverty Estimates (SAIPE) program obtains Food Stamp program (now known as the Supplemental Nutrition Assistance Program [SNAP]) participation estimates from the U.S. Department of Agriculture, Food and Nutrition Service. Also, the Census Bureau's 2010-2014 American Community Survey (ACS) 5-year ACS demographic and socioeconomic variables at the tract level and poverty variable at the county level were used (http://www.census.gov/programs-surveys/acs/).
• Federal Bureau of Investigation (FBI). Uniform Crime Report (UCR) arrest totals were obtained from http://www.icpsr.umich.edu/icpsrweb/NACJD/archive.jsp. The most current data used are from 2012 for most counties, with previous years' data substituted in a few cases.
• Bureau of Labor Statistics (BLS). The 2015 county-level unemployment estimates were used (http://www.bls.gov/lau/tables.htm). The BLS uses results from the Current Population Survey (CPS) to provide county-level unemployment estimates. The CPS is a monthly survey of households conducted by the Census Bureau for the BLS.
• Bureau of Economic Analysis (BEA). The 2014 county-level per capita income estimates were used (http://bea.gov/iTable/index.cfm). These county-level per capita income estimates are produced by the Regional Income Division of the BEA.
• National Center for Health Statistics (NCHS). Mortality data using International Classification of Diseases, 10th revision (ICD-10), 2007-2012, were used. The ICD-10 death data are from the NCHS at the Centers for Disease Control and Prevention (CDC).
• SAMHSA, Center for Behavioral Health Statistics and Quality (CBHSQ, formerly the Office of Applied Studies [OAS]). Data were used from the National Survey of Substance Abuse Treatment Services (N-SSATS), formerly known as the Uniform Facility Data Set (UFDS). The 2012-2013 data on drug and alcohol treatment estimates were obtained. Maintenance of effort expenditures, block grant awards, cost of services, and total taxable resources data were also used.

The following lists provide the specific independent variables that were potential predictors in the models.

B.4 Benchmarking the Age Group-Specific Small Area Estimates

The self-calibration built into the survey-weighted hierarchical Bayes (SWHB) solution ensures that the population-weighted average of the state small area estimates will closely match the national design-based estimates. The national design-based estimates in NSDUH are based entirely on survey-weighted data using a direct estimation approach, whereas the state and census region estimates are model-based. Given the self-calibration ensured by the SWHB solution, for state reports prior to 2002, the standard Bayes prescription was followed; specifically, the posterior mean was used for the point estimate, and the tail percentiles of the posterior distribution were used for the Bayesian confidence interval limits.

Singh and Folsom (2001) extended Ghosh's (1992) results on constrained Bayes estimation to include exact benchmarking to design-based national estimates. In the simplest version of this constrained Bayes solution where only the design-based mean is imposed as a benchmarking constraint, each of the 2014-2015 state-by-age group small area estimates is adjusted by adding the common factor , where is the design-based national estimate and is the population-weighted mean of the state small area estimates for age group-a. The exactly benchmarked state-s and age group-a small area estimates then are given by . Experience with such additive adjustments suggests that the resulting exactly benchmarked state small area estimates will always be between 0 percent and 100 percent because the SWHB self-calibration ensures that the adjustment factor is small relative to the size of the state-level small area estimates.

Relative to the Bayes posterior mean, these benchmark-constrained state small area estimates are biased by the common additive adjustment factor. Therefore, the posterior mean squared error (MSE) for each benchmarked state small area estimate has the square of this adjustment factor added to its posterior variance. To achieve the desirable feature of exact benchmarking, this constrained Bayes adjustment factor was implemented for the state-by-age group small area estimates. The associated Bayesian confidence (credible) intervals can be recentered at the benchmarked small area estimates on the logit scale with the symmetric interval end points based on the posterior root mean squared errors (RMSEs). The adjusted 95 percent Bayesian confidence intervals are defined below:

,     D

where

,     D

,     D     and

.     D

The associated posterior coverage probabilities for these benchmarked intervals are very close to the prescribed 0.95 value because the state small area estimates have posterior distributions that can be approximated exceptionally well by a Gaussian distribution.

B.5 Calculation of Estimated Number of Individuals Associated with Each Outcome

Tables 1 to 15 of "2014-2015 NSDUHs: Model-Based Estimated Totals (in Thousands) (50 States and the District of Columbia)" show the estimated numbers of individuals associated with each of the 14 outcomes of interest.²⁰ To calculate these numbers, the benchmarked small area estimates and the associated 95 percent Bayesian confidence intervals are multiplied by the average population across the 2 years (in this case, 2014 and 2015) of the state by the age group of interest.

For example, past month use of alcohol among 18 to 25 year olds in Alabama was 51.99 percent.²¹ The corresponding Bayesian confidence intervals ranged from 48.05 to 55.90 percent. The population count for 18 to 25 year olds averaged across 2014 and 2015 in Alabama was 530,600 (see Table C.10 in Section C of this methodology document). Hence, the estimated number of 18 to 25 year olds using alcohol in the past month in Alabama was 0.5199 × 530,600, which is 275,859.²² The associated Bayesian confidence intervals ranged from 0.4805 × 530,600 (i.e., 254,953) to 0.5590 × 530,600 (i.e., 296,605). Note that when estimates of the number of individuals are calculated for Tables 1 to 15 in "2014-2015 NSDUHs: Model-Based Estimated Totals (in Thousands) (50 States and the District of Columbia)" (follow the link in footnote 23), the unrounded percentages and population counts are used, then the numbers are reported to the nearest thousand. Hence, the number obtained by multiplying the published estimate with the published population estimate may not exactly match the counts that are published in these tables because of rounding differences.

The only exception to this calculation is the production of the estimated numbers of marijuana initiates. Those estimates cannot be directly calculated as the product of the percentage estimate of first use of marijuana and the population counts available in Section C. That is because the denominator of that percentage estimate is defined as the number of person years at risk for marijuana initiation, which is a combination of individuals who never used marijuana and one half of the individuals who initiated in the past 24 months.

B.6 Calculation of Aggregate Age Group Estimates and Limitations

Tables 1 to 15 of "2014-2015 NSDUHs: Model-Based Prevalence Estimates (50 States and the District of Columbia)" show estimates for the following age groups: 12 to 17, 18 to 25, 26 or older, 18 or older, and 12 or older.²³ If a user was interested in producing aggregated estimates, such as for those aged 12 to 25, the aggregated estimates could be calculated using prevalence estimates along with the population totals shown in Section C of this document. However, with the information that is provided in the tables, the confidence intervals cannot be calculated. Below is an example of this calculation for a given state.

For example, past month use of alcohol in Alabama among youths 12 to 17 was 8.76 percent, and among young adults 18 to 25 it was 51.99 percent.²⁴ The population counts for 12 to 17 year olds and 18 to 25 year olds averaged across 2014 and 2015 in Alabama were 380,801 and 530,600, respectively (see Table C.10 in Section C of this methodology document). Hence, one would calculate the estimate for individuals aged 12 to 25 by first finding the number of users aged 12 to 25, which is 309,217 ([0.0876 × 380,801] + [0.5199 × 530,600]), then dividing that number by the population aged 12 to 25, which results in a rate of 33.93 percent (309,217 ÷ [380,801 + 530,600]).

B.7 Comparison of Two 2014-2015 Small Area Estimates

This section describes a method for determining whether differences between two 2014-2015 state population percentages are statistically significant. This procedure can be used for any two state population percentages representing the same age group (e.g., young adults aged 18 to 25) and time period (e.g., 2014-2015).

Let and denote the 2014-2015 age group-a specific prevalence rates for two different states, and , respectively. The null hypothesis of no difference, that is, , is equivalent to the log-odds ratio equal to zero, that is, , where is defined as , where ln denotes the natural logarithm. An estimate of is given by , where and are the 2014-2015 state estimates given in the "2014-2015 NSDUH: Model-Based Prevalence Estimates (50 States and the District of Columbia) (Tables 1 to 15, by Age Group)" (follow the link in footnote 24). To compute the variance of , that is, , let and , then , where denotes the covariance between and . This covariance is defined in terms of the associated correlation as follows:

.     D

The quantities and can be obtained by using the 95 percent Bayesian confidence intervals given in the "2014-2015 NSDUH: Model-Based Prevalence Estimates (50 States and the District of Columbia) (Tables 1 to 15, by Age Group)" (follow the link in footnote 24). For this purpose, let and denote the 95 percent Bayesian confidence intervals for the two states, and , respectively. Then

,     D

where .

For all practical purposes, the correlation between and is assumed to be negligible; hence, can be approximated by . The correlation is assumed to be negligible because each state was a stratum in the first level of stratification; therefore, each state sample is selected independently. However, the correlation between the two state estimates is theoretically nonzero because state estimates share common fixed-effect parameters in the SAE models. Hence, the test statistic (defined below) might result in a different conclusion in a few cases when the correlation between the state estimates is incorporated in calculating . To calculate the p value for testing the null hypothesis of no difference (), it is assumed that the posterior distribution of is normal with and . With the null value of , the Bayes p value or significance level for the null hypothesis of no difference is , where is a standard normal random variate, , and denotes the absolute value of . This Bayesian significance level (or p value) for the null value of , say , is defined following Rubin (1987) as the posterior probability for the collection of the values that are less likely or have smaller posterior density than the null (no change) value . That is, . With the posterior distribution of approximately normal, is given by the above expression.

Hence, to test whether differences between two 2014-2015 state estimates are statistically significant, the test statistic and the associated p value can be used. If p ≤ 0.05, then the two state estimates can be considered different at the 5 percent level of significance. Because age group estimates within a state are correlated, the method described here cannot be used to test whether differences between two age group estimates within a state are statistically significant.

When comparing estimates for two states, it is tempting and often convenient to look at their 95 percent Bayesian confidence intervals to decide whether the difference in the state estimates is significant. If the two Bayesian confidence intervals overlap, one would conclude that the difference is not statistically significant. If the two Bayesian confidence intervals do not overlap, it implies that the state estimates are significantly different from each other. However, the type-I error for the overlapping 95 percent Bayesian confidence intervals test may be as low as 0.6 percent (assuming that the two state estimates are uncorrelated and have the same variances) as compared with the 5 percent type-I error of the test based on the statistics defined above (Payton, Greenstone, & Schenker, 2003).

As discussed in Schenker and Gentleman (2001), the method of overlapping Bayesian confidence intervals is more conservative (i.e., it rejects the null hypothesis of no difference less often) than the standard method based on statistics when the null hypothesis is true. Even if Bayesian confidence intervals for two states overlap, the two estimates may be declared significantly different by the test based on statistics. Hence, the method of overlapping Bayesian confidence intervals is not recommended to test the difference of two state estimates. A detailed description of the method of overlapping confidence intervals and its comparison with the standard methods for testing of a hypothesis is given in Schenker and Gentleman (2001) and Payton et al. (2003).

Example. The percentages for past month alcohol use among 12 to 17 year olds in New Jersey and Oklahoma are shown in the following exhibit and also in Table 6 of the "2014-2015 NSDUH: Model-Based Prevalence Estimates (50 States and the District of Columbia)" at http://www.samhsa.gov/data/. Looking at the two 95 percent Bayesian confidence intervals, it would appear that the Oklahoma and New Jersey percentages for past month alcohol use are not statistically different at the 5 percent level of significance because the two Bayesian confidence intervals overlap:

However, in the following example, the test based on the statistic described earlier concludes that they are significantly different at the 5 percent level of significance.

Let , , , , , . Then,

,     D

,     D

,     D

,     D

,     D   and

.     D

Because the computed absolute value of is greater than or equal to 1.96 (the critical value of the statistic), then at the 5 percent level of significance, the hypothesis of no difference (Oklahoma prevalence rate = New Jersey prevalence rate) is rejected. Thus, the two state prevalence rates are statistically different. The Bayes p value or significance level for the null hypothesis of no difference is calculated as follows:

.     D

Section C: Sample Sizes, Response Rates, and Population Estimates