Part I Chapter 3

Point Estimates and Design-Based Uncertainty

Nearly half of adults have hypertension. How many know it, treat it, and control it, and how sure can we be?

DataNHANES 2021–2023 · NHIS · BRFSS · Taylor linearization

A standard error of what?

The software printed a standard error. Is it the standard error of anything? You have a public-use health file open and a difference between two weighted shares, and one call gives it a p-value of 0.022 while another gives 0.081. One of them goes in the paper, and nothing in the output says which. The two calls agree on the estimate. They disagree about a single input: a statement about the repetition that makes an estimate vary, which one call was given and the other was not. This chapter is about that input, what it costs to leave it out, and how to tell when leaving it out changes a conclusion.

The difference in question is a change in blood pressure control among men. Nearly half of U.S. adults have high blood pressure: in August 2021–August 2023, 47.7% (95% CI 45.4%–49.9%) met the definition the National Center for Health Statistics (NCHS) uses, a reading of 130/80 mm Hg or higher in the examination room or current use of prescribed blood-pressure medication. Hypertension is treatable in a specific sequence, and public health counts the steps as a cascade. Among adults with hypertension, 59.2% had been told so by a health professional, 51.2% were taking medication for it, and 20.7% had their blood pressure under control. The shares come from the National Health and Nutrition Examination Survey (NHANES), which measures blood pressure in a mobile examination center rather than asking about it, and NCHS published them in Data Brief 511 (Fryar et al. 2024). The brief also answers the question clinicians asked as the pandemic receded: did control get worse? Between 2017–March 2020 and August 2021–August 2023, NCHS reported no detectable change in awareness, treatment, or control.

20.7%

of adults with hypertension had it under control, August 2021–August 2023

95% CI 18.8%–22.7%

That conclusion is easy to overturn by accident. Among men with hypertension, the estimated change in control is −3.0 (95% CI −6.4 to 0.4) percentage points. An analyst who computes weighted shares with a group-by and attaches the textbook standard error reports a significant decline, p = 0.022. An analyst who declares NHANES's sampling design reports p = 0.081. Same files, same weights, same point estimate. Men's control is also one of 24 period comparisons, four stages of the cascade in six groups of adults, so one small p-value among them deserves suspicion before the design is even declared.

The cascade, counted the way NCHS counts it

The target population is U.S. civilian noninstitutionalized adults 18 and older, excluding pregnant women, in August 2021–August 2023. Blood pressure is the average of up to three oscillometric readings taken in the mobile examination center (MEC). Hypertension is a systolic pressure of 130 mm Hg or more, a diastolic pressure of 80 or more, or a yes to "Are you now taking any medication prescribed by a doctor for your high blood pressure?" Awareness is a yes to ever having been told of hypertension by a doctor or other health professional. Treatment is the medication question. Control is systolic below 130 and diastolic below 80. Awareness, treatment, and control are shares of adults with hypertension, not of all adults and not of those at the previous stage. The analytic sample is 6,084 examined adults, 3,242 of them with hypertension, the brief's counts exactly.

The hypertension cascade, August 2021–August 2023

Adults 18+; awareness, treatment and control are shares of adults with hypertension; 95% intervals

Design-based
External benchmark

Show data

Group	Estimate for	Estimate	95% CI	n
This chapter (Taylor, df 15)	Prevalence (all adults)	47.7%	45.4%–49.9%	6,084
NCHS Data Brief 511 (published)	Prevalence (all adults)	47.7%	45.4%–49.9%	6,084
This chapter (Taylor, df 15)	Awareness (with hypertension)	59.2%	56.5%–61.9%	3,240
NCHS Data Brief 511 (published)	Awareness (with hypertension)	59.2%	56.4%–61.9%	3,240
This chapter (Taylor, df 15)	Treatment (with hypertension)	51.2%	48.7%–53.7%	3,238
NCHS Data Brief 511 (published)	Treatment (with hypertension)	51.2%	48.6%–53.7%	3,238
This chapter (Taylor, df 15)	Control (with hypertension)	20.7%	18.8%–22.7%	3,242
NCHS Data Brief 511 (published)	Control (with hypertension)	20.7%	18.8%–22.8%	3,242

Figure 3.1.Hypertension is common and rarely controlled. Prevalence is a share of all adults; awareness, treatment, and control are each shares of adults with hypertension. The estimates match NCHS's published values within a tenth of a percentage point.Note: Chapter intervals: estimate ± t(15)·SE. NCHS publishes Korn–Graubard intervals, so endpoints can differ in the last digit.Source: NHANES August 2021–August 2023; NCHS Data Brief 511; chapter calculations

Figure 3.1 sets each estimate beside the published one. Agreement at the published rounding shows that the code reproduces NCHS's definitions. It does not show that the numbers are true. The brief is another estimate from the same sample, with the same protocol and the same nonresponse; benchmarks are not truth.Interval endpoints can differ in the last digit: NCHS publishes Korn–Graubard intervals, and this chapter uses the estimate plus or minus a t critical value times the standard error.

Two traps in the August 2021–August 2023 files

The examination weight is WTMEC2YR; WTMECPRP belongs to the pre-pandemic file. The medication item also changed: BPQ_L replaced BPQ040A and BPQ050A with BPQ150 and a new skip pattern, so only a yes counts as treated, and 3 adults with normal readings and unknown medication status stay in the denominator as not hypertensive.

Analysis Ledger

How many adults have hypertension, know it, treat it, and control it? (NHANES August 2021–August 2023)

Target population: U.S. civilian noninstitutionalized adults 18 and older, excluding pregnant women, August 2021–August 2023
Estimand: Finite-population proportions: hypertension among adults; awareness, treatment and control among adults with hypertension (NCHS Data Brief 511 definitions). Also the prevalence directly standardized to the 2000 U.S. population
Estimator: Weighted ratio (Hájek) means over domains of the declared design; age-adjusted prevalence = Σ_a s_a p̂_a over 18–39, 40–59, 60+
Explicit weights: WTMEC2YR, the examination weight: base weights for unequal selection probabilities with nonresponse and calibration adjustments. s_a = 2000 standard-population shares (0.4203, 0.3572, 0.2225) in the age-adjusted estimate
Implicit weights: None beyond the explicit weights; every estimate is a weighted mean of a 0/1 indicator
Source of randomness: Selection of PSUs and of people within PSUs in a stratified multistage sample; nonresponse treated as handled by the weight adjustments
Variance estimator: Taylor linearization with 15 masked strata and 30 PSUs (with-replacement approximation); design df 15; 95% intervals estimate ± t(15)·SE
Assumptions: Weights correct for differential nonresponse; the oscillometric protocol measures blood pressure comparably across participants; medication use is self-reported; averaging up to three readings defines the participant's blood pressure
Numbers reported: 47.7%percent
59.2%percent
51.2%percent
20.7%percent
44.5%percent

Three standard errors for one number

The prevalence estimate is a weighted ratio, the Hájek estimator:

\hat p = \frac{\sum_i w_i y_i}{\sum_i w_i},

where $y_i$ is 1 for hypertension and $w_i$ is the examination weight. With correct inclusion weights, $\hat p$ is a consistent estimator of one specified finite-population quantity: the share of the target population with hypertension. The weights define what the formula estimates. They do not say how far the estimate would move if the survey were run again.

Three standard errors answer three different questions about that repetition. The naive standard error imagines repeated simple random samples of examined adults and ignores the weights. The weights-only standard error charges for unequal weights but still treats each respondent as an independent draw. The full-design standard error describes the sample NHANES draws: a few dozen primary sampling units (PSUs), counties or groups of counties, selected within strata, with hundreds of people examined in each.

Predict first

The sampler below is the one from Chapter 0. Its second stage draws repeated samples of the same size two ways: simple random samples of people, and samples of neighborhoods with a fixed number of residents in each. Before you open that stage, guess whether the clustered estimates spread more or less than the simple random ones, and how the answer changes as fewer neighborhoods supply more residents each.

One population, three questions

A synthetic population of 20,000 adults in 200 neighbourhoods (version 1.0.0, population seed 20260910). Each stage changes one thing.

Stage

1. Who enters the sample2. How the sample was drawn3. What the regression targets

How much more likely a member of group G is to be sampled4×

Selection probability: 13.8% for G, 3.4% for everyone else. Design weight d = 1/π.

Seed 1n = 1,000

Population

G 15%

not G

This sample

G 42%

not G

Across 200 samples at this setting the unweighted mean averages 16.5% and the weighted mean 12.3%, against a truth of 12.4%. Members of G are uninsured more often (26% against 10%), so a sample that over-represents them overstates the rate unless each person counts 1/π times.

What is held fixed. The population, the outcome (uninsured), and the sample size. Only who is likely to enter the sample changes; the design weights d = 1/π undo exactly that.

The clustered estimates spread more than the simple random ones, the gap grows as fewer neighborhoods supply more residents each, and the center of the distribution does not move. Two people examined in the same county share its diet, its incomes, its doctors, and its air; the second one tells the survey less than a stranger from a new county would. That is the pattern behind every NHANES standard error.

Predict first

For hypertension prevalence, will the full-design standard error be smaller than the naive one, about the same, or larger? By how much? And which of the two simpler standard errors will it be closer to?

Taylor linearization turns the picture into a formula. Replace each respondent by a linearized value $z_i = w_i(y_i - \hat p)/\sum_j w_j$ , add those values within each PSU to get a total $z_{hj}$ for PSU $j$ in stratum $h$ , and compare PSUs within strata:

\widehat{\Var}(\hat p) = \sum_{h} \frac{n_h}{n_h - 1} \sum_{j=1}^{n_h} \left(z_{hj} - \bar z_h\right)^2 .

PSUs enter as the units of independent information. Strata enter only through the within-stratum comparison, which keeps differences between strata out of the variance; that is where stratification's precision gain comes from (Lumley 2010; Heeringa, West, and Berglund 2017). The weights-only formula is the same expression with every respondent as its own PSU and a single stratum, and the naive formula is $\sqrt{\hat p(1-\hat p)/(n-1)}$ .

One estimate, three standard errors

Table of five hypertension estimates for August 2021–August 2023 with their naive, weights-only and full-design standard errors, design effects between about 1.7 and 3.1, and 15 design degrees of freedom.
Estimate	Value	n	SE, naive (pp)	SE, weights only (pp)	SE, full design (pp)	DEFF	Design df
Hypertension, crude	47.7%	6,084	0.64	0.79	1.06	2.72×	15
Hypertension, age-adjusted	44.5%	6,084	0.64	0.77	1.12	3.07×	15
Awareness	59.2%	3,240	0.86	1.09	1.26	2.12×	15
Treatment	51.2%	3,238	0.88	1.09	1.18	1.80×	15
Control	20.7%	3,242	0.71	0.82	0.93	1.70×	15

Figure 3.2.The full-design standard error is well above both simpler ones, and by how much depends on the estimand.Note: Naive: √(p(1−p)/(n−1)), ignoring weights and design. Weights only: every respondent its own PSU, no strata (the HC1 sandwich for a weighted mean). Full design: Taylor linearization with SDMVSTRA and SDMVPSU. DEFF = (full-design SE / naive SE)².Source: NHANES August 2021–August 2023; chapter calculations

For prevalence, the three standard errors are 0.64, 0.79, and 1.06 percentage points (Figure 3.2). The ratio of the design variance to the naive variance is the design effect, $\DEFF$ (Kish 1965): here 2.72. It factors exactly into two pieces. The weighting factor, the squared ratio of the weights-only to the naive standard error, is 1.51. The clustering-and-stratification factor, the squared ratio of the design to the weights-only standard error, is 1.80. For prevalence, the 6,084 examined adults carry about as much information as 2,239 adults drawn at random.

Caution

A slogan that needs correcting in both halves

"Weights get the point estimate, the design gets the uncertainty." The weights help define the estimator, and they also shape its variance twice over: unequal weights inflate it, and the weights are themselves built from the sample, since nonresponse and calibration adjustments are estimates. The repeated-sampling variance therefore depends on the sampling design and on how the weights were constructed. Taylor linearization applied to the final weights treats them as fixed; replicate methods that repeat the weighting steps in every replicate do not.

Degrees of freedom are the other half of the story. The public August 2021–August 2023 file has 30 masked PSUs in 15 strata, and the usual Taylor rule, PSUs minus strata, gives 15 degrees of freedom: the number of independent within-stratum comparisons behind the variance estimate. With so few, the 97.5th percentile of t is 2.13 rather than 1.96. Together with the larger standard error, the design-based interval is 1.8 times as wide as the naive one. The design declaration behind these numbers is two lines in either language (The tools).

There is no single NHANES design effect

A design effect belongs to an estimator, for an outcome, in a domain, under a design; it is not a constant of the survey. For August 2021–August 2023 it is 2.72 for prevalence, 2.12 for awareness, 1.80 for treatment, 1.70 for control, and 2.30 for prevalence among adults 18–39 (Figure 3.3). The pre-pandemic file gives 5.29 for the same prevalence estimand. Most of that difference is weighting. The 2017–March 2020 sample oversampled by race, Hispanic origin, and income, and its weighting factor is 2.34; the August 2021–August 2023 sample did not oversample. An effective sample size is always an effective sample size for something: $\ndesign(\hat\theta) = n/\DEFF(\hat\theta)$ .

There is no single NHANES design effect

Design effect by estimand and survey period

Show data

Group	Estimate for	Estimate	n
Aug 2021–Aug 2023	Prevalence	2.72	6,084
Aug 2021–Aug 2023	Awareness	2.12	3,240
Aug 2021–Aug 2023	Treatment	1.80	3,238
Aug 2021–Aug 2023	Control	1.70	3,242
Aug 2021–Aug 2023	Prevalence, ages 18–39	2.30	1,712
2017–Mar 2020	Prevalence	5.29	7,948
2017–Mar 2020	Awareness	3.25	4,297
2017–Mar 2020	Treatment	3.15	4,295
2017–Mar 2020	Control	1.62	4,305
2017–Mar 2020	Prevalence, ages 18–39	3.03	2,608

Figure 3.3.Design effects vary across estimands and survey periods; no single number describes NHANES.Note: Subgroup pre-specified before estimation. The 2017–March 2020 sample oversampled by race, Hispanic origin and income; the August 2021–August 2023 sample did not.Source: NHANES 2017–March 2020 and August 2021–August 2023; chapter calculations

The Moulton bridge

Predict first

Suppose two people in the same PSU are barely more alike than two strangers, an intraclass correlation well under one percent. Can that nearly double a variance?

Economists know the clustering component under another name. For a regressor that is constant within clusters, with equal cluster sizes $\bar m$ and equicorrelated errors, Moulton (1990) showed that conventional variances understate the truth by the factor

1 + (\bar m - 1)\rho .

NHANES examines about 203 analytic-sample adults per PSU, so an intraclass correlation of only 0.004 reproduces the clustering factor of 1.80. The answer to the prediction is yes: with clusters that large, a correlation that small nearly doubles the variance. Read the formula as a bridge, not an identity. The Moulton factor is a special-case regression analogue of the clustering component of a design effect: it assumes equal cluster sizes, leaves out the stratification gain, and says nothing about the weighting factor. Chapter 6 takes up clustering in regression.

Did control get worse?

The comparison needs two independent samples measured the same way. The pre-pandemic file (2017–March 2020, weight WTMECPRP, 25 design degrees of freedom) and the August 2021–August 2023 file both used the oscillometric device. The separate 2017–2018 cycle stays out: the pre-pandemic file already contains its participants under new identifiers, so pooling the two would count them twice.

Nationally, nothing moved detectably. Awareness was 60.2% before and 59.2% after, treatment 50.3% and 51.2%, control 22.2% and 20.7%, a change of −1.5 (95% CI −4.0 to 1.0) percentage points. Naive and full-design tests agree that no stage changed (for control, p = 0.126 and 0.244), which matches the NCHS finding.

The conclusions part company in subgroups. Across the 24 pre-specified comparisons, four stages in all adults, men, women, and three age groups, naive intervals flag 3, weights-only intervals flag 1, and full-design intervals flag 1 (Figure 3.4; Figure 3.5 lists all 24). The design-based test adds the two periods' Taylor variances and uses a t reference with Satterthwaite degrees of freedom built from 15 and 25. Men's control, from the opening, is one of the naive flags. Prevalence among adults 60 and older is another: a change of −2.5 (95% CI −6.2 to 1.1) percentage points, with a naive p of 0.035 and a design p of 0.170. The one change whose design-based interval excludes zero is awareness among adults 18–39 with hypertension, −8.4 (95% CI −16.6 to −0.2) percentage points (naive p = 0.004, design p = 0.044). With 24 tests at the 5 percent level, about $24 \times 0.05 = 1.2$ rejections are expected even if nothing changed, so this one is not established as a real change either. The design moved no estimate; it widened the intervals, and two of the three naive flags no longer exclude zero.

Did the cascade change after the pandemic?

Aug 2021–Aug 2023 minus 2017–March 2020, percentage points, 95% intervals

Naive
Design-based

Show data

Group	Estimate for	Estimate	95% CI
Naive iid interval	Prevalence	−0.4	−2.1–1.2
Full-design interval	Prevalence	−0.4	−3.8–2.9
Naive iid interval	Prevalence, ages 60+	−2.5	−4.9–−0.2
Full-design interval	Prevalence, ages 60+	−2.5	−6.2–1.1
Naive iid interval	Awareness	−1.0	−3.3–1.2
Full-design interval	Awareness	−1.0	−4.7–2.7
Naive iid interval	Awareness, ages 18–39	−8.4	−14.2–−2.7
Full-design interval	Awareness, ages 18–39	−8.4	−16.6–−0.2
Naive iid interval	Treatment	0.9	−1.4–3.2
Full-design interval	Treatment	0.9	−2.7–4.5
Naive iid interval	Control	−1.5	−3.3–0.4
Full-design interval	Control	−1.5	−4.0–1.0
Naive iid interval	Control, men	−3.0	−5.6–−0.4
Full-design interval	Control, men	−3.0	−6.4–0.4

Figure 3.4.Naive intervals exclude zero for three subgroup changes; full-design intervals exclude zero for one.Note: Overall stages plus the three subgroup comparisons (of 24) that naive intervals flag. Design intervals use t with Satterthwaite df from 15 and 25.Source: NHANES 2017–March 2020 and August 2021–August 2023; chapter calculations

All 24 period comparisons

Table of 24 comparisons of hypertension cascade estimates between 2017–March 2020 and August 2021–August 2023 by stage and group, with naive and full-design p-values.
Stage	Group	2017–Mar 2020	Aug 2021–Aug 2023	Change (pp)	p, naive	p, full design
Prevalence	All adults	48.1%	47.7%	−0.4	0.615	0.799
Prevalence	Men	50.5%	50.8%	0.3	0.808	0.885
Prevalence	Women	45.7%	44.6%	−1.1	0.335	0.582
Prevalence	Ages 18–39	23.4%	23.4%	0.0	0.978	0.986
Prevalence	Ages 40–59	52.4%	52.5%	0.0	0.992	0.995
Prevalence	Ages 60+	74.1%	71.6%	−2.5	0.035	0.170
Awareness	All adults	60.2%	59.2%	−1.0	0.375	0.585
Awareness	Men	58.2%	55.2%	−3.0	0.064	0.262
Awareness	Women	62.3%	63.6%	1.3	0.430	0.557
Awareness	Ages 18–39	35.6%	27.2%	−8.4	0.004	0.044
Awareness	Ages 40–59	55.3%	56.7%	1.4	0.510	0.698
Awareness	Ages 60+	73.9%	73.7%	−0.2	0.880	0.920
Treatment	All adults	50.3%	51.2%	0.9	0.432	0.613
Treatment	Men	46.3%	46.7%	0.4	0.808	0.878
Treatment	Women	54.5%	56.1%	1.6	0.329	0.460
Treatment	Ages 18–39	16.6%	13.9%	−2.6	0.250	0.349
Treatment	Ages 40–59	43.3%	47.1%	3.8	0.072	0.249
Treatment	Ages 60+	69.3%	69.1%	−0.3	0.846	0.894
Control	All adults	22.2%	20.7%	−1.5	0.126	0.244
Control	Men	21.9%	18.9%	−3.0	0.022	0.081
Control	Women	22.5%	22.8%	0.3	0.850	0.889
Control	Ages 18–39	5.5%	4.5%	−1.0	0.454	0.484
Control	Ages 40–59	18.8%	18.1%	−0.7	0.649	0.746
Control	Ages 60+	31.6%	29.2%	−2.4	0.100	0.329

Figure 3.5.Naive and full-design p-values for every stage and group; highlighted rows reach the 5 percent level under at least one method.Note: Highlighted rows: p < 0.05 under at least one method. No adjustment for multiple comparisons.Source: NHANES; chapter calculations

Analysis Ledger

Did awareness, treatment, or control change after the pandemic?

Target population: U.S. civilian noninstitutionalized adults 18+ with hypertension (pregnant women excluded) in 2017–March 2020 and in August 2021–August 2023
Estimand: Differences between the two periods in finite-population proportions, overall and in five pre-specified groups (24 comparisons)
Estimator: Difference of two independent weighted ratio means
Explicit weights: WTMECPRP (2017–March 2020 pre-pandemic file) and WTMEC2YR (August 2021–August 2023)
Implicit weights: None
Source of randomness: Two independent stratified multistage samples
Variance estimator: Sum of the two Taylor variances (15 and 25 df), t with Satterthwaite df; compared with naive iid and weights-only variances
Assumptions: Comparable definitions and instruments (oscillometric in both periods; medication question changed wording and skip pattern in 2021–2023); no adjustment for multiple comparisons, as in NCHS reports
Numbers reported: −1.5percentage points
60.2%percent
50.3%percent
22.2%percent
3
1

That answers the opening question, for a comparison of two independent samples. The same data hold four further lessons about weights and designs, and a second survey supplies the case researchers meet more often: two groups inside one sample.

Age standardization: a third kind of weight

Predict first

Crude prevalence is higher among men than among women by 6.3 percentage points, and women in the population are older on average. After both groups are put on one age distribution, will the gap shrink or grow?

Survey weights $w_i$ repair representation: they make the sample stand for the population that existed in 2021–2023, with its own age structure. Standardization weights $s_a$ do something else. They replace that age structure with a fixed standard, which redefines the population in which the comparison is made:

\hat p^{\text{adj}} = \sum_{a} s_a \hat p_a, \qquad s_a = (0.4203,\ 0.3572,\ 0.2225),

where $\hat p_a$ is the survey-weighted prevalence in the age groups 18–39, 40–59, and 60 and older, and $s_a$ is each group's share of the 2000 U.S. standard population, the NCHS convention. The $s_a$ are constants, so the variance comes from linearizing the three $\hat p_a$ jointly under the design.

Crude prevalence is 47.7%; age-adjusted prevalence is 44.5% (95% CI 42.1%–46.9%) (Figure 3.6). For men the two are 50.8% and 48.8%; for women, 44.6% and 40.1%. The sex gap widens from 6.3 (95% CI 2.9 to 9.6) to 8.7 (95% CI 5.8 to 11.5) percentage points, because the crude comparison sets an older female population against a younger male one. Neither number is "the" prevalence. The crude share describes the actual population; the adjusted share describes a hypothetical one with the standard's age distribution, which is what makes groups and periods comparable. The pre-pandemic values, 48.1% and 45.1%, show the same wedge.

Survey weights and standardization weights answer different questions

Hypertension prevalence, crude and age-adjusted to the 2000 U.S. standard population, 95% intervals

Show data

Group	Estimate for	Estimate	95% CI	n
Aug 2021–Aug 2023	All adults, crude	47.7%	45.4%–49.9%	6,084
Aug 2021–Aug 2023	All adults, age-adjusted	44.5%	42.1%–46.9%	6,084
Aug 2021–Aug 2023	Men, crude	50.8%	48.5%–53.2%	2,776
Aug 2021–Aug 2023	Men, age-adjusted	48.8%	46.1%–51.4%	2,776
Aug 2021–Aug 2023	Women, crude	44.6%	41.4%–47.7%	3,308
Aug 2021–Aug 2023	Women, age-adjusted	40.1%	37.3%–42.9%	3,308
2017–Mar 2020	All adults, crude	48.1%	45.4%–50.7%	7,948
2017–Mar 2020	All adults, age-adjusted	45.1%	42.6%–47.5%	7,948

Figure 3.6.Age adjustment lowers prevalence and widens the gap between men and women: standardization changes the comparison population, not the representation of the sample.Note: Standardization weights s_a = 0.4203 (18–39), 0.3572 (40–59), 0.2225 (60+).Source: NHANES; 2000 U.S. standard population (Census P25-1130); chapter calculations

Domains: subset the design, not the data

Awareness, treatment, and control are domain estimates. They describe adults with hypertension, a subgroup whose sample size is itself random. The textbook rule is to declare the design on the full sample and then restrict the estimate, never to delete rows first. What the rule buys is easiest to see in the National Health Interview Survey (NHIS) 2023 (Blewett et al. 2025), which has many more PSUs: 662 PSUs in 52 strata and 610 design degrees of freedom.The NHIS analysis uses the sample-adult weight SAMPWEIGHT and a (STRATA, PSU) key built within the year, because PSU numbers repeat across strata. For self-reported hypertension, 32.3% (95% CI 31.6%–32.9%) of 29,471 sample adults, the design effect is 1.64.

Start with the defensible-looking mistake: filter to the domain, then declare the design on what remains. For a dense domain, cost-related delay in medical care among adults with hypertension (6.4% (95% CI 5.8%–7.0%) of 11,019 adults), the delete-rows-first standard error is only 0.05% smaller than the correct one. For a sparse domain, uninsured adults in poor health with 43 respondents, the correct standard error of the same share is 9.46 percentage points and the delete-rows-first one is 8.26, which is 12.7% too small. For the weighted count of adults who delayed care, the shortfall is 44%.The sparse proportion, with an effective sample size of 28, would fail the NCHS presentation standards (Parker et al. 2017); it appears here only for its standard error.

The mechanism is collapsed strata, not subsetting as such. The domain's members sit in 41 PSUs spread over 30 strata. Once the other rows are gone, 22 of those strata contain a single PSU, and the within-stratum comparison has nothing to compare. Declared on the full design, the same strata keep all their PSUs. PSUs with no domain members contribute totals of zero, which is how the formula learns that the domain's size varies from sample to sample, the source of variation that matters most for totals.

One sparse domain, four answers

Table comparing standard errors for delayed care among uninsured adults in poor health: the declared-design SE versus three delete-rows-first SEs under certainty, adjust and average lonely-PSU rules, for the proportion and for the weighted count.
Approach	A single-PSU stratum	SE, proportion (pp)	Ratio	SE, weighted count	Ratio
Declare the design, then estimate the domain	not applicable: all 662 PSUs kept	9.46	1.00×	60,491	1.00×
Delete rows first; certainty or remove	contributes zero	8.26	0.87×	33,776	0.56×
Delete rows first; adjust	deviation from the grand mean	10.46	1.11×	46,575	0.77×
Delete rows first; average	average of the other strata	15.99	1.69×	65,406	1.08×

Figure 3.7.In a sparse domain, deleting rows first makes the standard error too small or too large, depending on a rule the analyst may never have chosen.Note: Domain: uninsured sample adults in poor self-rated health with a valid delayed-care response (n = 43). After deleting rows, 22 of 30 surviving strata have one PSU. R's default rule (fail) and svy's default both refuse to compute.Source: IPUMS NHIS 2023 sample adults; chapter calculations; checked against svy 0.28 and R survey 4.5

The software has to be told what a lonely PSU contributes, and each answer is a modeling decision (Figure 3.7). Under certainty or remove, it contributes zero, as if that PSU had been selected with certainty. Under adjust, it is centered at the grand mean rather than at its stratum mean. Under average, the other strata's variance is scaled up to stand in for it. For the sparse domain these rules give 8.26, 10.46, and 15.99 percentage points, against the correct 9.46. R's default, survey.lonely.psu = "fail", stops with an error, and svy raises SingletonError. Both defaults force the decision into the open.

How sparse is too sparse? A controlled experiment gives a gradual answer, not a threshold (Figure 3.8). Draw 200 random domains of each size from the sample adults and compare the two standard errors for self-reported hypertension. Down to domains of about one percent of the sample, the median ratio of the delete-rows-first standard error to the correct one stays within a percent of one (0.994). At 59 respondents, a median of 19 strata are left with one PSU, the median ratio falls to 0.81, and 80% of the domains get a standard error more than 10 percent too small.

Thinning one domain: the understatement grows as strata collapse

Table: as a random domain of NHIS 2023 sample adults shrinks from 100% to 0.2% of the sample, PSUs with members fall from 662 to about 56, single-PSU strata rise from 0 to about 19, and the median ratio of the delete-rows-first SE to the correct SE falls from 1.00 to about 0.81.
Share of adults kept	n	PSUs with members	Single-PSU strata	df after deletion	SE ratio, median	10th pct	Draws >10% too small
100.0%	29,471	662	0	610	1.000	1.000	0%
10.0%	2,947	637	0	585	1.000	0.999	0%
2.0%	589	380	0	328	0.998	0.986	0%
1.0%	295	234	3	183	0.994	0.967	0%
0.5%	147	131	10	83	0.965	0.904	9%
0.2%	59	56	19	22	0.806	0.641	80%

Figure 3.8.As the domain shrinks and strata lose PSUs, the delete-rows-first standard error drifts gradually below the correct one; the understatement becomes serious only in very small domains.Note: Each row below 100% summarizes 200 seeded random domains of fixed size (seed 20260910). SE ratio = SE after deleting non-domain rows (single-PSU strata as certainty units) / SE from the declared design. Medians shown.Source: IPUMS NHIS 2023 sample adults; outcome: ever told hypertension; chapter calculations

Two reporting rules follow. Put the PSUs retained and the number of single-PSU strata next to every domain standard error; every NHANES domain in this chapter keeps all PSUs with no lonely stratum. And state the degrees of freedom separately from the variance: NCHS counts only the PSUs and strata that contain domain members, and R and svy report the same count, 11 for the sparse domain, although the full design has 610.

Analysis Ledger

A sparse domain, estimated two ways (NHIS 2023)

Target population: U.S. civilian noninstitutionalized adults 18+ without health insurance and in poor self-rated health, 2023
Estimand: Share (and number) who delayed medical care because of cost in the past 12 months
Estimator: Weighted ratio mean and weighted total within the domain
Explicit weights: SAMPWEIGHT, the sample-adult weight
Implicit weights: None
Source of randomness: Selection of PSUs within 52 strata and of households and adults within PSUs; domain membership is itself random
Variance estimator: Taylor linearization on the full declared design, (STRATA, PSU) composite key (662 PSUs); contrasted with deleting non-domain rows first under certainty, adjust, and average lonely-PSU rules
Assumptions: With-replacement PSU approximation; the domain proportion is shown only to study its standard error (effective n below 30 fails NCHS presentation standards)
Numbers reported: 9.46percentage points
8.26percentage points
12.7%
44%

Two groups in one sample

Most published comparisons are not between two surveys. They are between two groups in one: the South against the Northeast, women against men, citizens against noncitizens. Each group is a domain, and the difference between two domain means is the estimate that carries the argument. Its standard error has to be computed under the design, and it has a term the naive formula never contains: the covariance between the two estimates, which is not zero when the two groups were interviewed in the same PSUs.

Predict first

Take the uninsured share among NHIS adults 18–64 and compare it between the Northeast and the Midwest, and then between men and women. For which comparison does the design inflate the naive standard error more? And for which does it matter that the two estimates come from the same PSUs?

The family is fixed before any result is seen: the uninsured share among sample adults 18–64, the Medicaid-relevant age range that Lab 3 ranks by state, compared between every pair of groups along six dimensions, region, race and Hispanic origin, education, citizenship, sex, and age group, 27 comparisons in all. Nationally the share is 10.8% (95% CI 10.1%–11.5%). Most of the 27 differences are large, and every method agrees on them: naive tests flag 26, weights-only tests 26, full-design tests 25, and the verdict changes for 1 (Figure 3.9). That one is the comparison a regional story would be built on. The uninsured share is 6.3% in the Northeast and 8.1% in the Midwest, a difference of −1.8 (95% CI −3.7 to 0.0) percentage points. Its naive standard error is 0.61 points (p = 0.003), the weights-only standard error is 0.74 (p = 0.014), and the full-design standard error is 0.94 (p = 0.055). The opening's two p-values were of this kind.

One sample, 27 group comparisons of the uninsured share

Table of 27 pairwise comparisons of the uninsured share among adults 18 to 64 in NHIS 2023 across region, race and Hispanic origin, education, citizenship, sex, and age group, with naive and full-design p-values; the region comparisons show the largest inflation of the standard error and the Northeast-Midwest comparison changes verdict.
Dimension	Group A	Group B	A	B	A − B (pp)	SE ratio, design ÷ naive	Covariance ratio	p, naive	p, full design
Census region	Northeast	Midwest	6.3%	8.1%	−1.8	1.54×	1.00×	0.003	0.055
Census region	Northeast	South	6.3%	14.8%	−8.6	1.65×	1.00×	0.000	0.000
Census region	Northeast	West	6.3%	9.8%	−3.6	1.47×	1.00×	0.000	0.000
Census region	Midwest	South	8.1%	14.8%	−6.8	1.54×	1.00×	0.000	0.000
Census region	Midwest	West	8.1%	9.8%	−1.8	1.33×	1.00×	0.003	0.027
Census region	South	West	14.8%	9.8%	5.0	1.46×	1.00×	0.000	0.000
Race and Hispanic origin	White NH	Black NH	6.8%	10.2%	−3.4	1.26×	1.00×	0.000	0.000
Race and Hispanic origin	White NH	Hispanic	6.8%	24.7%	−17.9	1.42×	0.98×	0.000	0.000
Race and Hispanic origin	White NH	Asian/PI NH	6.8%	4.2%	2.6	1.18×	1.01×	0.000	0.000
Race and Hispanic origin	White NH	Other/Multiple NH	6.8%	14.4%	−7.6	1.22×	0.99×	0.000	0.000
Race and Hispanic origin	Black NH	Hispanic	10.2%	24.7%	−14.5	1.36×	0.99×	0.000	0.000
Race and Hispanic origin	Black NH	Asian/PI NH	10.2%	4.2%	6.0	1.21×	1.00×	0.000	0.000
Race and Hispanic origin	Black NH	Other/Multiple NH	10.2%	14.4%	−4.2	1.23×	0.99×	0.008	0.032
Race and Hispanic origin	Hispanic	Asian/PI NH	24.7%	4.2%	20.5	1.31×	0.96×	0.000	0.000
Race and Hispanic origin	Hispanic	Other/Multiple NH	24.7%	14.4%	10.3	1.26×	0.98×	0.000	0.000
Race and Hispanic origin	Asian/PI NH	Other/Multiple NH	4.2%	14.4%	−10.2	1.22×	0.99×	0.000	0.000
Education	Less than HS	HS	28.3%	14.7%	13.6	1.25×	0.97×	0.000	0.000
Education	Less than HS	Some college	28.3%	8.7%	19.6	1.24×	0.96×	0.000	0.000
Education	Less than HS	BA+	28.3%	4.3%	24.0	1.27×	0.98×	0.000	0.000
Education	HS	Some college	14.7%	8.7%	6.0	1.18×	0.95×	0.000	0.000
Education	HS	BA+	14.7%	4.3%	10.4	1.22×	0.99×	0.000	0.000
Education	Some college	BA+	8.7%	4.3%	4.4	1.15×	0.95×	0.000	0.000
Citizenship	Not a citizen	Citizen	37.0%	7.6%	29.3	1.38×	0.98×	0.000	0.000
Sex	Men	Women	12.5%	9.1%	3.4	1.22×	0.87×	0.000	0.000
Age group	18-29	30-44	12.7%	12.3%	0.3	1.15×	0.91×	0.617	0.664
Age group	18-29	45-64	12.7%	8.3%	4.3	1.12×	0.92×	0.000	0.000
Age group	30-44	45-64	12.3%	8.3%	4.0	1.21×	0.89×	0.000	0.000

Figure 3.9.Region comparisons carry the largest design inflation because regions are built from whole PSUs; comparisons between groups that share PSUs have a covariance the independence formula misses. One of 27 verdicts changes.Note: Uninsured at interview (HINOTCOVE). Naive: independent simple random samples, normal reference. Full design: Taylor variance of the difference on the declared (STRATA, PSU) design, covariance included, t reference with the df of PSUs and strata containing either group. Covariance ratio = design SE of the difference ÷ the design SE that treats the two groups as independent. Highlighted rows change verdict at the 5 percent level. No adjustment for multiple comparisons.Source: IPUMS NHIS 2023 sample adults 18–64; chapter calculations

Two patterns in Figure 3.9 answer the prediction. First, geography is where clustering bites hardest. A region is a union of whole PSUs, so a comparison between regions is a comparison between two sets of counties, and every PSU's shared conditions land entirely on one side of it. Over the six region comparisons the full-design standard error is a median 1.50 times the naive one; over the other 21 it is 1.22 times. Second, the covariance runs the other way. Two regions share no PSU, and the design standard error of their difference equals the value the independence formula $\sqrt{\SE_a^2 + \SE_b^2}$ gives (median ratio 1.00). Men and women live in the same households and the same counties, their estimates move together across repeated samples, and the design standard error of their difference is 0.87 times the independence value: the positive covariance makes the comparison more precise, not less. Overlapping confidence intervals are therefore not a test in either direction. Even for independent estimates the standard error of a difference is $\sqrt{2}$ times one of them and not twice, so two intervals can overlap while the interval for the difference excludes zero, and shared PSUs change the answer again. Estimate the difference itself, on the declared design, with both groups kept in the frame.

Analysis Ledger

Which groups of adults differ in coverage? (NHIS 2023)

Target population: U.S. civilian noninstitutionalized adults 18–64 in 2023, by Census region, race and Hispanic origin, education, citizenship, sex, and age group
Estimand: Differences between pairs of groups in the finite-population share without health insurance at interview; 27 pre-specified comparisons
Estimator: Difference of two weighted ratio (Hájek) means, each a domain of the full sample-adult design
Explicit weights: SAMPWEIGHT, the sample-adult weight
Implicit weights: None
Source of randomness: One stratified multistage sample; both groups in every comparison come from the same PSUs, so their estimates covary
Variance estimator: Taylor linearization of the difference on the declared design, (STRATA, PSU) composite key, covariance included; t with the df of PSUs and strata containing either group; compared with naive iid and weights-only variances
Assumptions: With-replacement PSU approximation; item nonresponse on coverage excluded; no multiplicity adjustment; descriptive contrasts, not effects
Numbers reported: 10.8%percent
27
26
25
−1.8percentage points

Two complex surveys, opposite uncertainty

"Complex survey" names a family, not a property. The Behavioral Risk Factor Surveillance System (BRFSS) 2024 (CDC 2024) interviewed 457,670 adults by telephone in 2,393 strata, and each respondent is a PSU: 457,670 PSUs and 455,277 design degrees of freedom. Its raked weights (Chapter 2) vary widely, with a coefficient of variation of 2.03. For fair or poor self-rated health, 19.6% (95% CI 19.3%–19.8%), the design effect is 5.24: a weighting factor of 5.26 times a clustering-and-stratification factor of 1.00. NHANES is the mirror image, with a weighting factor of 1.51, a clustering factor of 1.80, and 15 degrees of freedom. NHIS sits between them (Figure 3.10).

The contrast changes what the analyst should worry about. In BRFSS the t critical value is 1.96 and the degrees of freedom never bind, but unequal weights use up most of the sample's information. In NHANES the weights cost less, and a handful of PSUs determines both the variance and how precisely the variance itself is estimated. Even BRFSS needs a lonely-PSU rule: 110 of its strata contain a single respondent, and this chapter uses adjust.BRFSS 2024 omits Tennessee, so its target population is adults in the participating jurisdictions.

Complex surveys, opposite sources of uncertainty

Table comparing NHANES, NHIS and BRFSS: NHANES has 30 PSUs and 15 design degrees of freedom and its design effect comes mostly from clustering; BRFSS has one respondent per PSU and about 455,000 degrees of freedom and its design effect comes almost entirely from unequal weights.
Survey	Example estimand	Analytic n	Strata	PSUs	Design df	t(0.975)	1 + CV(w)²	Weighting factor	Clustering × strata factor	DEFF
NHANES Aug 2021–Aug 2023	Hypertension, measured or medicated	6,084	15	30	15	2.13	1.52×	1.51×	1.80×	2.72×
NHIS 2023 sample adults	Ever told hypertension	29,471	52	662	610	1.96	1.42×	1.30×	1.26×	1.64×
BRFSS 2024	Fair or poor general health	456,360	2,393	457,670	455,277	1.96	5.13×	5.26×	1.00×	5.24×

Figure 3.10.NHANES uncertainty comes from few PSUs and BRFSS uncertainty from unequal weights; both surveys are called complex.Note: DEFF = weighting factor × clustering-and-stratification factor, by construction: (weights-only SE / naive SE)² × (design SE / weights-only SE)². BRFSS 2024 omits Tennessee; its Taylor SE uses the adjust rule for 110 single-respondent strata.Source: NHANES August 2021–August 2023; IPUMS NHIS 2023; CDC BRFSS 2024; chapter calculations

Interlude: weighting repairs representation, not measurement

Weights can correct for who answered. They cannot correct what was measured. Three surveys ask adults whether a health professional ever told them they had high blood pressure: BRFSS 2023 gives 34.4% (95% CI 34.0%–34.7%), NHIS 2023 gives 32.3%, and NHANES's own interview gives 30.8%. NHANES's measured definition gives 47.7%, and the older 140/90 threshold applied to the same readings gives 32.4% (Figure 3.11). The gap between self-report and measurement is not a representation error that better weights could close. The surveys estimate different constructs: a remembered diagnosis on one side, a blood pressure reading plus medication use on the other.BRFSS asks this item only in odd years, and 2023 omits Kentucky and Pennsylvania, so its population is adults in 48 states and the District of Columbia.

Three surveys, two constructs

Adults 18+, share with high blood pressure, 95% full-design intervals

Show data

Group	Estimate for	Estimate	95% CI	n
Self-reported diagnosis	BRFSS 2023: told high blood pressure	34.4%	34.0%–34.7%	423,197
Self-reported diagnosis	NHIS 2023: told hypertension	32.3%	31.6%–32.9%	29,471
Self-reported diagnosis	NHANES 2021–23: told high blood pressure	30.8%	28.6%–32.9%	6,080
Measured	NHANES 2021–23: measured ≥140/90 or medication	32.4%	29.9%–34.9%	6,084
Measured	NHANES 2021–23: measured ≥130/80 or medication	47.7%	45.4%–49.9%	6,084

Figure 3.11.Self-reported diagnosis agrees across three surveys; measured hypertension is a different construct, and no weighting reconciles the two.Note: BRFSS 2023 covers 48 states and DC (Kentucky and Pennsylvania did not meet data standards). Calendar periods differ slightly.Source: CDC BRFSS 2023; IPUMS NHIS 2023; NHANES August 2021–August 2023; chapter calculations

NHANES shows the same lesson inside one survey. In 2017–2018, NCHS measured the same examinees with both the old auscultatory protocol and the new oscillometric device, in random order (NCHS 2024). For those 4,633 adults, with identical weights, mean systolic pressure was 1.5 (95% CI 0.8 to 2.2) mm Hg higher by auscultation, and prevalence under the 130/80-or-medication definition was 48.4% by auscultation against 47.6% by the device, a difference of 0.7 (95% CI −1.4 to 2.8) percentage points (1.1 points for the readings-alone definition). Representation is identical by construction; only the instrument changed. That is why the period comparison above uses two oscillometric samples.

Replicate weights, briefly

NHANES releases masked strata and PSUs, so Taylor linearization is the producer's method. Other surveys ship replicate weights instead: the American Community Survey provides 80 alternative weight columns and the CPS ASEC provides 160. Replicate weights are the survey world's pre-packaged repeated sampling: re-estimate once with each column, and the spread of the replicate estimates, scaled by a method-specific constant, approximates the sampling variance. Balanced repeated replication, Fay's method (Fay and Train 1995), the jackknife, successive difference replication, and the bootstrap are distinct procedures with different constants and degrees of freedom; calling them all "a bootstrap" hides exactly the constants an analyst must get right. Because the producer repeats the weighting steps inside each replicate, the replicate variance also carries the weight-construction variability that fixed-weight linearization leaves out. Lab 3 works through the American Community Survey case.

The tools

Every design-based number in this chapter comes from one declaration: the stratum, the PSU, and the weight. The two calls below return the same estimate, standard error, design effect, and degrees of freedom to machine precision, and the chapter's build checks both against a hand-written Taylor routine.

python

import polars as pl
import svy
 
# nh: one row per MEC-examined participant (WTMEC2YR > 0) with 0/1 columns
# htn and analytic built as defined above. PSU codes repeat across strata.
nh = nh.with_columns((pl.col("SDMVSTRA") * 10 + pl.col("SDMVPSU")).alias("psu"))
design = svy.Design(stratum="SDMVSTRA", psu="psu", wgt="WTMEC2YR")
sample = svy.Sample(nh, design=design)
prev = sample.estimation.mean("htn", where=svy.col("analytic") == 1, deff="wr")

des <- svydesign(ids = ~SDMVPSU, strata = ~SDMVSTRA, weights = ~WTMEC2YR,
                 nest = TRUE, data = nh)
svymean(~htn, subset(des, analytic), deff = "replace")

The domain rule is the where argument in svy and subset() on the design object in R; filtering the data frame first is the mistake from the domain section. A difference between two domains is one more line: svyby(..., covmat = TRUE) followed by svycontrast() keeps the covariance between the two group estimates, which is how the men-minus-women standard error in Figure 3.9 was checked.

python

# Wrong: delete rows, then declare. svy refuses to guess.
sub = svy.Sample(adults.filter(pl.col("sparse") == 1), design=design)
sub.estimation.mean("delay")            # raises SingletonError
sub.singleton.handle("certainty")       # an explicit rule you must report
# Right: keep every PSU and restrict the estimate.
svy.Sample(adults, design=design).estimation.mean("delay", where=svy.col("sparse") == 1)

What we learned about the question

Hypertension is common and poorly controlled. In August 2021–August 2023, 47.7% of adults met the 130/80-or-medication definition; of those, 59.2% knew it, 51.2% were treated, and 20.7% had it controlled. None of the four national shares changed detectably from 2017–March 2020. Two of the three subgroup changes that naive intervals flag do not exclude zero under the declared design, and the third is one of 24 comparisons. In the NHIS, 25 of 27 group differences in the uninsured share among adults 18–64 hold under the design; the Northeast–Midwest difference, −1.8 percentage points, does not.

What the method changed

The point estimates never changed. What changed was the width of every interval (for prevalence, 1.8 times the naive width, with 15 degrees of freedom), the number of subgroup changes whose intervals exclude zero (from 3 to 1), the standard errors of sparse domains, the standard error of a difference between two groups in one sample, which grew by a median factor of 1.50 across regions and fell below the independence value for men against women, and, through standardization weights, the population in which men and women are compared.

What this analysis does not establish

This analysis does not show that the cascade was stable in every subgroup: the interval for the national change in control, −1.5 (95% CI −4.0 to 1.0) percentage points, is wide enough to hide a meaningful decline, and subgroup intervals are wider still. It establishes no trend beyond two periods and nothing about why control did or did not change; the pandemic is a date here, not a cause. Treatment rests on self-reported medication use, asked differently in the two periods. Agreement with NCHS Data Brief 511 validates the code, not the population values. The NHIS group differences are descriptive contrasts, tested one at a time without a multiplicity adjustment, and the Northeast–Midwest comparison is not shown to be zero, only not distinguishable from it. The sparse NHIS proportion is not a publishable estimate, and the implied intraclass correlation is a back-of-envelope reading of an equal-cluster formula.