Skip to content
Draft: numbers and prose are under review
Contents
Weights, Design, and Estimands

Appendix B

Scoped Derivations

Linearization, the Moulton factor, implicit regression weights, and four effective sample sizes, each with its conditions.

You will need this appendix when a chapter's claim is one you would have to defend: when a referee asks why a cluster-robust standard error should equal the survey package's, or whether the Moulton factor is "the design effect," or how a regression with positive effects in every cell can return a negative coefficient. Each question has a short answer in the chapter and its conditions here. Come for the derivation you need; the table below says which one that is.

Five derivations carry the chapters' arguments, and none of them is new. What this appendix adds is the scope: each result is stated with the conditions under which it holds, next to the algebra, with a note on what happens when a condition fails. Every identity and approximation was also checked numerically on synthetic data built to satisfy its conditions; Figure B.1 at the end summarizes the checks.

DerivationThe question it answersUsed inKind of result and the condition that matters most
Horvitz–Thompson and HájekWhy does weighting by inverse inclusion probabilities point an estimator at a population total or mean?Chapters 0 and 2Exactly unbiased for a total, approximately for a mean; needs known, positive inclusion probabilities and complete response
Taylor linearization and CRV1When does a cluster-robust standard error equal the design-based one?Chapters 3 and 7Identical with one stratum; with several strata, close only in expectation, and only when the strata do not predict the outcome
The Moulton factorHow much does clustering inflate a conventional regression variance?Chapters 3 and 4Exact with equal cluster sizes and equicorrelated errors; an approximation otherwise
Variance weighting in OLSWhich average of heterogeneous contrasts does a regression on a binary treatment report?Chapters 0, 4, and 5Exact in sample with a saturated set of cells, also for adjusted outcomes when other controls enter additively; with linear controls, weights can turn negative
Four effective samplesWhat does each "effective sample size" measure?Chapters 0, 2, 3, and 5Each holds under its own conditions, and none converts into another

Horvitz–Thompson and Hájek

A finite population UU of NN units carries fixed values yiy_i. A probability design selects a sample ss; unit ii is included with probability πi=Pr⁡(i∈s)\pi_i = \Pr(i \in s), and units ii and jj together with probability πij\pi_{ij}. Write Ii=1{i∈s}I_i = \mathbf{1}\{i \in s\} and di=1/πid_i = 1/\pi_i.

The Horvitz–Thompson estimator of the total T=∑i∈UyiT = \sum_{i \in U} y_i is design-unbiased (Horvitz and Thompson 1952). The Hájek estimator of the mean is a ratio of two such estimators; it is consistent and approximately unbiased, and it is what software computes when asked for a weighted mean.

Derivation
T^HT=∑i∈sdi yi=∑i∈UIiyiπi,Ep ⁣[T^HT]=∑i∈Uyiπi Ep[Ii]=∑i∈Uyi=T.\hat T_{\text{HT}} = \sum_{i \in s} d_i\, y_i = \sum_{i \in U} I_i \frac{y_i}{\pi_i}, \qquad \E_p\!\left[\hat T_{\text{HT}}\right] = \sum_{i \in U} \frac{y_i}{\pi_i}\,\E_p[I_i] = \sum_{i \in U} y_i = T .

The expectation Ep\E_p is over repeated samples from the design with the yiy_i held fixed. Its variance is

Var⁡p ⁣(T^HT)=∑i∈U∑j∈U(πij−πiπj)yiπiyjπj,πii=πi,\Var_p\!\left(\hat T_{\text{HT}}\right) = \sum_{i \in U}\sum_{j \in U} \left(\pi_{ij} - \pi_i \pi_j\right) \frac{y_i}{\pi_i}\frac{y_j}{\pi_j}, \qquad \pi_{ii} = \pi_i ,

which can be estimated without bias when every πij>0\pi_{ij} \gt 0.

The Hájek mean divides by the estimated population size:

Yˉ^H=∑i∈sdiyi∑i∈sdi=T^yN^,Yˉ^H−Yˉ=T^y−YˉN^N^≈1N∑i∈sdi (yi−Yˉ).\hat{\bar Y}_{\text{H}} = \frac{\sum_{i \in s} d_i y_i}{\sum_{i \in s} d_i} = \frac{\hat T_{y}}{\hat N}, \qquad \hat{\bar Y}_{\text{H}} - \bar Y = \frac{\hat T_y - \bar Y \hat N}{\hat N} \approx \frac{1}{N}\sum_{i \in s} d_i\,(y_i - \bar Y).

The right-hand side is a Horvitz–Thompson estimator of the total of the residuals yi−Yˉy_i - \bar Y, divided by NN. So the Hájek mean is unbiased to first order, with a bias of order 1/n1/n, and its variance is approximately the Horvitz–Thompson variance of the residuals. When yiy_i is roughly constant but N^\hat N varies from sample to sample, as under Poisson sampling, that variance is far smaller than the variance of T^HT/N\hat T_{\text{HT}}/N.

Conditions. The design must give every unit of the target population a positive, known πi\pi_i, and the weights must be 1/πi1/\pi_i. Units with πi=0\pi_i = 0 are outside what any weighting can recover: that is coverage, not estimation. Responses must be complete and measured without error. Once the weights are wi=digiw_i = d_i g_i with nonresponse and calibration adjustments, the estimator is a calibration estimator: design-consistent under conditions on the control totals, approximately unbiased only if the response model holds, and with a variance that should reflect the calibration, either by linearizing with the calibration residuals or with replicate weights that repeat the weighting. The binding statement follows: correct inclusion weights give design-unbiased or consistent estimators for specified finite-population quantities, and which of the two depends on the estimand and the estimator.

Taylor linearization for a ratio mean, and when it coincides with CRV1

Take a stratified design with strata h=1,…,Hh = 1, \dots, H, with nh≥2n_h \ge 2 primary sampling units in stratum hh treated as sampled with replacement, and weights whijw_{hij} for unit ii of PSU jj in stratum hh. The ratio R=Y/XR = Y/X is estimated by R^=Y^/X^\hat R = \hat Y/\hat X, with Y^=∑whijyhij\hat Y = \sum w_{hij} y_{hij} and X^=∑whijxhij\hat X = \sum w_{hij} x_{hij}. A weighted mean is the case x≡1x \equiv 1.

Derivation: the linearized variance

A first-order expansion around (Y,X)(Y, X) gives

R^−R≈1X(Y^−RX^)=1X∑h,i,jwhij(yhij−R xhij).\hat R - R \approx \frac{1}{X}\left(\hat Y - R\hat X\right) = \frac{1}{X}\sum_{h,i,j} w_{hij}\left(y_{hij} - R\,x_{hij}\right).

Define the linearized values uhij=(yhij−R^ xhij)/X^u_{hij} = (y_{hij} - \hat R\,x_{hij})/\hat X, their weighted PSU totals zhj=∑iwhijuhijz_{hj} = \sum_i w_{hij} u_{hij}, and stratum means zˉh=nh−1∑jzhj\bar z_h = n_h^{-1}\sum_j z_{hj}. The with-replacement variance estimator is

V^TL(R^)=∑h=1Hnhnh−1∑j=1nh(zhj−zˉh)2,\hat V_{\text{TL}}(\hat R) = \sum_{h=1}^{H} \frac{n_h}{n_h - 1}\sum_{j=1}^{n_h}\left(z_{hj} - \bar z_h\right)^2 ,

with the common design degrees of freedom ∑hnh−H\sum_h n_h - H (Binder 1983; Wolter 2007).

Now compare the cluster-robust variance of a weighted least squares regression of yy on a constant, with the PSUs as clusters (G=∑hnhG = \sum_h n_h), strata ignored, and residuals ei=yi−β^e_i = y_i - \hat\beta:

β^=R^,V^CRV1=GG−1⋅Nobs−1Nobs−K⋅∑g=1G(∑i∈gwiei)2(∑iwi)2.\hat\beta = \hat R, \qquad \hat V_{\text{CRV1}} = \frac{G}{G-1}\cdot\frac{N_{\text{obs}} - 1}{N_{\text{obs}} - K}\cdot \frac{\sum_{g=1}^{G}\left(\sum_{i \in g} w_i e_i\right)^2}{\left(\sum_i w_i\right)^2}.
Derivation: when the two coincide

With x≡1x \equiv 1, X^=∑iwi\hat X = \sum_i w_i, so ∑i∈gwiei/∑iwi=zg\sum_{i \in g} w_i e_i / \sum_i w_i = z_g, the PSU total of the linearized values. The normal equations give ∑gzg=0\sum_g z_g = 0 exactly.

One stratum. With H=1H = 1, zˉ=0\bar z = 0 and K=1K = 1, so

V^TL=GG−1∑gzg2=V^CRV1.\hat V_{\text{TL}} = \frac{G}{G-1}\sum_{g} z_g^2 = \hat V_{\text{CRV1}} .

Several strata. Decompose the uncentered sum by strata:

∑h,jzhj2=∑h,j(zhj−zˉh)2+∑hnh zˉh2,V^CRV1=GG−1[∑h,j(zhj−zˉh)2+∑hnh zˉh2].\begin{gathered} \sum_{h,j} z_{hj}^2 = \sum_{h,j}\left(z_{hj} - \bar z_h\right)^2 + \sum_h n_h\,\bar z_h^2 , \\[4pt] \hat V_{\text{CRV1}} = \frac{G}{G-1}\left[\sum_{h,j}\left(z_{hj} - \bar z_h\right)^2 + \sum_h n_h\,\bar z_h^2\right]. \end{gathered}

Taylor linearization keeps only the within-stratum part, with the factors nh/(nh−1)n_h/(n_h - 1). The term ∑hnhzˉh2\sum_h n_h \bar z_h^2 is between-stratum variation that the stratified design removed by construction. CRV1 keeps it, so it omits the stratification gain and is conservative when strata predict the outcome. When the stratum means of the linearized totals are near zero, the two estimators differ only through the factors G/(G−1)G/(G-1) and nh/(nh−1)n_h/(n_h - 1). With two PSUs per stratum the latter equals two; it is not a small correction, and it compensates, in expectation, for centering each stratum on a two-PSU mean.

Regression coefficients. The same structure holds with ui=(X′WX)−1xiwieiu_i = (X'WX)^{-1}x_i w_i e_i in place of the scalar linearized value. With one stratum, CRV1 equals Taylor linearization times (Nobs−1)/(Nobs−K)(N_{\text{obs}} - 1)/(N_{\text{obs}} - K).

Conditions. The two are identical in a given sample with one stratum, the same clusters (PSUs nested in strata and identified by the stratum-PSU pair), the with-replacement approximation in both (no finite population correction), the same weights, and either an intercept-only model or acceptance of the (Nobs−1)/(Nobs−K)(N_{\text{obs}} - 1)/(N_{\text{obs}} - K) factor. With several strata they are not identical. When the strata do not predict the outcome, the two agree approximately in expectation over repeated samples, not sample by sample: CRV1's between-stratum term and Taylor linearization's larger finite-sample factors offset on average. When the strata do predict the outcome, CRV1 keeps between-stratum variation that the design removed, and it is conservative. Intervals also need matching degrees of freedom: PSUs minus strata for Taylor linearization, commonly G−1G - 1 for CRV1. On real data the difference is visible. For adult cigarette smoking in NHIS 2024, CRV1 on the stratum-PSU key reproduces Taylor linearization with the strata ignored, and it is 1.11×1.11×EstimandCRV1 (composite PSU key) SE divided by the Taylor SE with strata, NHIS 2024 adult smokingVarianceNone reportedSourceIPUMS NHIS 2024 (ipums/analysis/nhis/nhis_core/part_2015_2024.parquet) the Taylor standard error that uses the strata; clustering on the PSU code alone, which merges PSUs from different strata, gives 1.21×1.21×EstimandCRV1 (PSU code alone) SE divided by the Taylor SE with strataVarianceNone reportedSourceIPUMS NHIS 2024 (ipums/analysis/nhis/nhis_core/part_2015_2024.parquet) (Chapter 8).

The Moulton factor as a special case

Predict first

Groups of 20 people share a common shock that accounts for a tenth of the outcome's variance, and the regressor of interest takes one value per group. By how much does the conventional OLS standard error understate the true one?

Moulton's factor answers that question for a regression coefficient (Moulton 1990). The survey design effect answers a related question for an estimator under a sampling design. They coincide in one special case.

Derivation

Let ygi=β0+β1xgi+ugiy_{gi} = \beta_0 + \beta_1 x_{gi} + u_{gi} with GG clusters of equal size mm and ugi=αg+εgiu_{gi} = \alpha_g + \varepsilon_{gi}, where the αg\alpha_g and εgi\varepsilon_{gi} are independent with variances σα2\sigma_\alpha^2 and σε2\sigma_\varepsilon^2, σ2=σα2+σε2\sigma^2 = \sigma_\alpha^2 + \sigma_\varepsilon^2, and ρu=σα2/σ2\rho_u = \sigma_\alpha^2/\sigma^2.

When xx is constant within clusters, xgi=xgx_{gi} = x_g,

β^1−β1=∑g(xg−xˉ) uˉg∑g(xg−xˉ)2,Var⁡(uˉg)=σα2+σε2m=σ2m[1+(m−1)ρu],\hat\beta_1 - \beta_1 = \frac{\sum_g (x_g - \bar x)\,\bar u_g}{\sum_g (x_g - \bar x)^2}, \qquad \Var(\bar u_g) = \sigma_\alpha^2 + \frac{\sigma_\varepsilon^2}{m} = \frac{\sigma^2}{m}\left[1 + (m-1)\rho_u\right],

so

Var⁡(β^1∣x)σ2/(m∑g(xg−xˉ)2)=1+(m−1)ρu.\frac{\Var(\hat\beta_1 \mid x)}{\sigma^2 \big/ \left(m\sum_g (x_g - \bar x)^2\right)} = 1 + (m-1)\rho_u .

The denominator is the conventional OLS variance. With unequal cluster sizes and a regressor that varies within clusters, the standard approximation is

Var⁡(β^1)Var⁡OLS(β^1)≈1+[Var⁡(mg)mˉ+mˉ−1]ρx ρu,\frac{\Var(\hat\beta_1)}{\Var_{\text{OLS}}(\hat\beta_1)} \approx 1 + \left[\frac{\Var(m_g)}{\bar m} + \bar m - 1\right]\rho_x\,\rho_u ,

where ρx\rho_x is the within-cluster correlation of the regressor.

Now take single-stage cluster sampling: GG clusters of size mm drawn with equal probability and with replacement, equal weights, and the sample mean of yy. Its variance relative to a simple random sample of the same size is 1+(m−1)ρy1 + (m-1)\rho_y, with ρy\rho_y the intraclass correlation of yy (Kish 1965). The sample mean is the OLS coefficient on a constant, a regressor that is constant within clusters, so Moulton's ratio with ρx=1\rho_x = 1 reproduces this clustering component.

For the prediction above, the variance ratio is 1+19(0.1)=2.91 + 19(0.1) = 2.9, so the conventional standard error is too small by a factor of about 2.9≈1.7\sqrt{2.9} \approx 1.7.

Conditions and scope. The factor needs equicorrelated errors with a common ρu\rho_u (a random-effects structure), homoskedasticity, independent clusters, a fixed regressor, and known cluster sizes. It concerns an unweighted regression coefficient and a model-based repetition, while the design effect concerns an estimator under a sampling design. A full design effect also reflects stratification, which usually lowers variance, and unequal weighting, which Kish's 1+CV(w)21 + \CV(w)^2 approximates under conditions of its own. The product (1+CV(w)2)(1+(mˉ−1)ρ)(1 + \CV(w)^2)\left(1 + (\bar m - 1)\rho\right) is a model-based approximation to the whole, with its own assumptions (Gabler, Häder, and Lahiri 1999). The accurate sentence is therefore the binding one: Moulton's factor is a special-case regression analogue of the clustering component of a design effect, not the design effect itself.

Variance weighting in OLS with a binary treatment

With a binary treatment DD and a discrete covariate XX whose cells are xx, the saturated regression Yi=∑xαx1{Xi=x}+βDi+eiY_i = \sum_x \alpha_x \mathbf{1}\{X_i = x\} + \beta D_i + e_i gives a coefficient that is a variance-weighted average of within-cell contrasts (Angrist 1998):

βOLS=∑xPr⁡(X=x)  px(1−px)  τx∑xPr⁡(X=x)  px(1−px),px=Pr⁡(D=1∣X=x),τx=E[Y∣D=1,X=x]−E[Y∣D=0,X=x].\begin{gathered} \beta_{\text{OLS}} = \frac{\sum_x \Pr(X{=}x)\; p_x (1 - p_x)\; \tau_x}{\sum_x \Pr(X{=}x)\; p_x (1 - p_x)}, \\[4pt] p_x = \Pr(D{=}1 \mid X{=}x), \qquad \tau_x = \E[Y \mid D{=}1, X{=}x] - \E[Y \mid D{=}0, X{=}x]. \end{gathered}
Derivation

By the Frisch–Waugh–Lovell theorem, β\beta is the coefficient from regressing YY on D~=D−E[D∣X]\tilde D = D - \E[D \mid X], the residual of DD on the cell dummies. Because the dummies are saturated, E[D∣X=x]=px\E[D \mid X{=}x] = p_x exactly, so

E[D~2∣X=x]=px(1−px),E[D~ Y∣X=x]=px(1−px)(E[Y∣D=1,x]−E[Y∣D=0,x])=px(1−px) τx,\begin{aligned} \E[\tilde D^2 \mid X{=}x] &= p_x(1-p_x), \\ \E[\tilde D\,Y \mid X{=}x] &= p_x (1-p_x)\left(\E[Y \mid D{=}1, x] - \E[Y \mid D{=}0, x]\right) \\ &= p_x(1-p_x)\,\tau_x , \end{aligned}

and β=E[D~Y]/E[D~2]\beta = \E[\tilde D Y]/\E[\tilde D^2] averages these over the cells. In a sample the identity is exact, with cell counts, treated shares, and differences of cell means in place of the population quantities.

The weights favor cells in which treatment is split evenly; cells in which nearly everyone or nearly no one is treated contribute almost nothing, whatever their size. The targets one might have wanted weight cells differently: a population average effect weights by Pr⁡(X=x)\Pr(X{=}x), the effect on the treated by Pr⁡(X=x∣D=1)\Pr(X{=}x \mid D{=}1), which is proportional to Pr⁡(X=x) px\Pr(X{=}x)\,p_x. A constant τx\tau_x, or the same treated share in every cell, makes all three coincide. Equal treatment variance, the same px(1−px)p_x(1 - p_x) in every cell, is weaker. It makes the regression's weights proportional to Pr⁡(X=x)\Pr(X{=}x), so the regression then targets the population average effect, but it does not bring the effect on the treated along: cells with treated shares pp and 1−p1 - p have the same variance and different weights Pr⁡(X=x) px\Pr(X{=}x)\,p_x.

Chapter 5 adds age, age squared, and sex to the cell dummies, entered additively. The identity survives, for adjusted outcomes.

Derivation: additive controls alongside the cells

The model is Yi=∑xαx1{Xi=x}+βDi+Zi′γ+eiY_i = \sum_x \alpha_x \mathbf{1}\{X_i = x\} + \beta D_i + Z_i'\gamma + e_i. Because (α^,β^,γ^)(\hat\alpha, \hat\beta, \hat\gamma) minimizes the (weighted) sum of squared residuals jointly, (α^,β^)(\hat\alpha, \hat\beta) also minimizes it with γ\gamma held at γ^\hat\gamma. That restricted problem is the saturated regression of the adjusted outcome Y~i=Yi−Zi′γ^\tilde Y_i = Y_i - Z_i'\hat\gamma on DiD_i and the cell dummies, to which the derivation above applies, with or without survey weights. So β^\hat\beta is the variance-weighted average of within-cell differences in mean adjusted outcomes, exactly in sample. The contrasts are net of one profile Zi′γ^Z_i'\hat\gamma fitted on the whole sample, whose coefficients depend on how DD and ZZ covary within cells; they are not the contrasts that separate regressions within each cell would give.

Survey weights change the distribution the average is taken over, not its form. Weighted least squares with inclusion weights replaces the sample's cell shares and treated shares with their population counterparts, so the survey-weighted coefficient estimates the population version of the variance-weighted average: the population linear projection coefficient of this saturated model, which in general is not the population average effect. Solon, Haider, and Wooldridge put the point in estimand terms: under unmodeled heterogeneity, weighting delivers the population projection rather than the population average of the heterogeneous effects, and the constructive response is to model the heterogeneity and then summarize it over the population of interest (Solon, Haider, and Wooldridge 2015). A gap between the weighted and unweighted coefficients is then a diagnostic, of heterogeneity interacting with the design, of misspecification, or of endogenous sampling, rather than a verdict on which is right (DuMouchel and Duncan 1983).

Conditions. The result requires a binary treatment and a saturated set of controls for a discrete covariate; it is exact in sample and holds in the population under the usual large-sample conditions. Reading τx\tau_x as a causal effect requires conditional ignorability; without it the τx\tau_x are descriptive within-cell contrasts, which is how Chapters 4 and 5 use them.

With controls entered linearly rather than saturated, Aronow and Samii show that if E[D∣X]\E[D \mid X] is linear in XX, the coefficient converges to E[ωiτi]/E[ωi]\E[\omega_i \tau_i]/\E[\omega_i] with ωi=(Di−E[Di∣Xi])2\omega_i = (D_i - \E[D_i \mid X_i])^2, whose conditional mean is p(Xi)(1−p(Xi))p(X_i)(1 - p(X_i)) (Aronow and Samii 2016). That is the regression-effective sample of Chapter 5. Słoczyński shows the complementary result that, with linear controls, the coefficient is approximately a convex combination of the effects on the treated and the untreated in which the weight on the effect on the treated is roughly the untreated share: the smaller group gets the larger weight (Słoczyński 2022).

Predict first

Every cell's treatment effect is positive: one, one, and five. Can the OLS coefficient on the treatment be negative?

It can, once the control is linear and the treatment probability is not: with three cells whose effects are one, one, and five, a linear control and a treatment probability that bends produce a negative coefficient.

The arithmetic of a negative implicit weight

Let L(X)L(X) denote the linear projection of DD on (1,X)(1, X). If treatment is ignorable given XX and E[Y(0)∣X]\E[Y(0) \mid X] is linear in XX, then

βOLS=E[p(X) {1−L(X)} τ(X)]E[p(X) {1−L(X)}],\beta_{\text{OLS}} = \frac{\E\left[p(X)\,\{1 - L(X)\}\,\tau(X)\right]}{\E\left[p(X)\,\{1 - L(X)\}\right]},

and the implicit weight p(x){1−L(x)}p(x)\{1 - L(x)\} is negative wherever L(x)>1L(x) \gt 1. Take X∈{0,1,2}X \in \{0, 1, 2\} with equal probabilities and p(x)=(0.05,0.90,0.95)p(x) = (0.05, 0.90, 0.95). The projection is L(x)=0.183+0.45xL(x) = 0.183 + 0.45x, so L(2)=1.083L(2) = 1.083, and the normalized weights are (0.140, 1.131, −0.271)(0.140,\ 1.131,\ -0.271). With cell effects τ(x)=(1,1,5)\tau(x) = (1, 1, 5),

βOLS=0.140(1)+1.131(1)−0.271(5)≈−0.086.\beta_{\text{OLS}} = 0.140(1) + 1.131(1) - 0.271(5) \approx -0.086 .

Every effect is positive and the coefficient is negative. Saturating the control restores the nonnegative weights px(1−px)p_x(1 - p_x). With linear controls, the implied weights on individual units can also be negative, the regression extrapolating beyond the data, which is why they are worth computing (Chattopadhyay and Zubizarreta 2023). The binding statement follows: equal case weights do not imply equal weighting of heterogeneous conditional effects.

Four effective samples

The Kish effective sample size

Hold the weights fixed and let the yiy_i be independent with common variance σ2\sigma^2, unrelated to the weights. Then

Var⁡ ⁣(∑iwiyi∑iwi)=σ2∑iwi2(∑iwi)2=σ2nKish,nKish=(∑iwi)2∑iwi2=n1+CV(w)2,\begin{gathered} \Var\!\left(\frac{\sum_i w_i y_i}{\sum_i w_i}\right) = \sigma^2 \frac{\sum_i w_i^2}{\left(\sum_i w_i\right)^2} = \frac{\sigma^2}{\nKish}, \\[4pt] \nKish = \frac{\left(\sum_i w_i\right)^2}{\sum_i w_i^2} = \frac{n}{1 + \CV(w)^2}, \end{gathered}

where CV(w)2=n−1∑i(wi−wˉ)2/wˉ2\CV(w)^2 = n^{-1}\sum_i (w_i - \bar w)^2 / \bar w^2 uses the divisor nn. The second equality follows from ∑iwi2=n(σw2+wˉ2)\sum_i w_i^2 = n(\sigma_w^2 + \bar w^2) and (∑iwi)2=n2wˉ2(\sum_i w_i)^2 = n^2 \bar w^2.

Conditions: an outcome unrelated to the weights, no clustering, and weights treated as fixed. When the weights are correlated with the outcome, as with selection proportional to a size that predicts the outcome, weighting can reduce variance and Kish's formula misleads. It never says anything about bias (Kish 1965).

The design effective sample size

The design effect of an estimator is

DEFF(θ^)=Var⁡design(θ^)Var⁡SRS(θ^),ndesign(θ^)=nDEFF(θ^),\DEFF(\hat\theta) = \frac{\Var_{\text{design}}(\hat\theta)}{\Var_{\text{SRS}}(\hat\theta)}, \qquad \ndesign(\hat\theta) = \frac{n}{\DEFF(\hat\theta)},

where the denominator is the variance under simple random sampling of the same size, with or without replacement; the choice should be stated. The numerator carries clustering, stratification, and weighting at once, and it depends on the outcome through its intraclass correlation and its relation to the strata and weights. A survey therefore has one design effect per estimator and outcome, not one per file.

The regression-effective sample

Under the conditions of the Aronow–Samii result above, the coefficient is ∑iω^iτi/∑iω^i\sum_i \hat\omega_i \tau_i / \sum_i \hat\omega_i with ω^i\hat\omega_i the squared residual from regressing DD on XX. The regression-effective sample is the analytic sample reweighted by ω^i\hat\omega_i. It is summarized by comparing ω^\hat\omega-weighted covariate means with population means, not by a count. A Kish-type scalar (∑iω^i)2/∑iω^i2(\sum_i \hat\omega_i)^2/\sum_i \hat\omega_i^2 measures how concentrated the implicit weights are, not how precise the coefficient is, and with a misspecified treatment model the weights on cells can be signed.

Meng's data-defect effective sample size

Let RiR_i indicate that unit ii of the finite population is recorded, n=∑iRin = \sum_i R_i, and f=n/Nf = n/N. With population moments computed with divisor NN,

yˉs−Yˉ=1n∑i∈URi (yi−Yˉ)=NnCov⁡U(R,y)=ρR,Y 1−ff  σY,\bar y_s - \bar Y = \frac{1}{n}\sum_{i \in U} R_i\,(y_i - \bar Y) = \frac{N}{n}\Cov_U(R, y) = \rho_{R,Y}\,\sqrt{\frac{1-f}{f}}\;\sigma_Y ,

an algebraic identity that holds for every recording pattern (Meng 2018). Averaging its square over the recording mechanism gives a mean squared error of ER[ρR,Y2] σY2(1−f)/f\E_R[\rho_{R,Y}^2]\,\sigma_Y^2 (1-f)/f. A simple random sample of size n′n' has variance close to σY2/n′\sigma_Y^2/n' when n′n' is small relative to NN, so the sample of equal mean squared error has size

nDDC≈f1−f⋅1ER ⁣[ρR,Y2].\nDDC \approx \frac{f}{1-f}\cdot\frac{1}{\E_R\!\left[\rho_{R,Y}^2\right]} .

With N=3.3×108N = 3.3 \times 10^8, a recorded fraction f=0.01f = 0.01, and a correlation of 0.0050.005 between recording and the outcome, nDDC≈404\nDDC \approx 404.

Conditions: the estimand is a finite-population mean; the identity is exact, but the correlation ρR,Y\rho_{R,Y} is unknown and must be bounded with outside information; weighted estimators have an analogous identity with the recording indicator replaced by a weighted one. The Kish effective sample size prices weight dispersion and the design effective sample size prices the whole design, both under the assumption that the design is a probability design; the data-defect effective sample size prices what happens when it is not.

Numerical checks

Numerical checks of the stated results

Each derivation's claim, checked on synthetic data built to satisfy its conditions

Table listing, for each derivation in Appendix B, the claim, the numerical check run in the build script, the result, and whether it is consistent with the claim.
BlockClaim as statedCheckResultStatus
B.1HT total is design-unbiased when every pi_i > 0 and is knownPoisson sampling, 20,000 draws, outcome unrelated to piHT mean minus T = -0.88 Monte Carlo SEsconsistent
B.1Hajek mean is not exactly unbiased; its variance can be far below HT/N'ssame draws, outcome unrelated to piHajek bias -0.0002 (MC SE 0.0006); Var(HT/N) / Var(Hajek) = 102.15consistent
B.1HT total is design-unbiased when every pi_i > 0 and is knownPoisson sampling, 20,000 draws, outcome proportional to piHT mean minus T = -0.82 Monte Carlo SEsconsistent
B.1Hajek mean is not exactly unbiased; its variance can be far below HT/N'ssame draws, outcome proportional to piHajek bias +0.0068 (MC SE 0.0008); Var(HT/N) / Var(Hajek) = 1.42consistent
B.2One stratum, weighted mean: Taylor (with replacement) equals CRV1 exactlyhand formula vs svy vs pyfixest CRV1, 40 PSUssvy vs hand 0.0e+00; CRV1 vs svy 0.0e+00 (relative)consistent
B.2Informative strata: CRV1 omits the stratification gain and is conservativeoutcome with stratum means 0, 3, 6, 9CRV1 SE / Taylor SE = 2.22; with uninformative strata 0.98consistent
B.2Regression (K = 2): CRV1 = Taylor x (N - 1)/(N - K), a small-sample factorslope SE: svy GLM vs hand sandwich vs pyfixest CRV1svy vs hand 1.3e-16; CRV1 / svy = 1.000748 vs sqrt((N-1)/(N-K)) = 1.000748consistent
B.3Moulton: Var(OLS slope) / conventional OLS variance = 1 + (m - 1) rho_x rho_uequal clusters (m = 20), rho_u = 0.1, regressor constant within clustersMonte Carlo ratio 3.01 vs formula 2.90consistent
B.3Moulton: Var(OLS slope) / conventional OLS variance = 1 + (m - 1) rho_x rho_uequal clusters (m = 20), rho_u = 0.1, regressor half within-clusterMonte Carlo ratio 1.89 vs formula 1.93consistent
B.4Saturated OLS slope = sum over cells of n_x p_x(1 - p_x) Delta_x / sum n_x p_x(1 - p_x), exactly in samplethree cells, p = 0.1, 0.5, 0.9; unweighted|OLS - formula| = 4.4e-16consistent
B.4Same identity for WLS with the weighted cell shares and treatment ratessame data with unequal weights|WLS - formula| = 2.2e-16; WLS 1.955 vs weighted cell-average (PATE analogue) 1.908 vs ATT analogue 2.440consistent
B.4Linear control, nonlinear propensity: implicit cell weights p(x)(1 - L(x)) can be negativeX in {0,1,2}, p = 0.05, 0.90, 0.95, cell effects 1, 1, 5; n = 2,000,000weights +0.140, +1.131, -0.271; population slope -0.0857, simulated OLS -0.0936consistent
B.5n_Kish = (sum w)^2 / sum w^2 = n / (1 + CV(w)^2)5,000 lognormal weightsrelative difference 1.8e-16consistent
B.5Meng: sample mean error = rho(R,Y) x sqrt((1 - f)/f) x sigma_Y, an identityN = 10,000, recording depends on Yerror 7.9064 vs identity 7.9064consistent
Figure B.1.Every stated identity holds to machine precision and every stated approximation holds within Monte Carlo error; where a condition is dropped, the result moves in the stated direction.Note: Consistent means the check behaved as the derivation says under its conditions; a check is not a proof.Source: appendix-b-derivations/build.py (seed 20260910); synthetic data only