5  Appendix A: Proof of the Variance Formula

This appendix provides a complete proof of the variance formula for the difference-in-means estimator, stated in the chapter on the variance of \(\widehat{\text{ATE}}\). The proof involves only algebra, but there are many steps and some tricks are helpful.

5.1 Setting Up the Problem

Recall from the previous chapter that we can write the difference-in-means estimator as

\[ \widehat{\text{ATE}} = \frac{1}{N_t} \sum_{i=1}^{N} D_i \cdot Y^{\text{obs}}_i - \frac{1}{N_c} \sum_{i=1}^{N} (1 - D_i) \cdot Y^{\text{obs}}_i, \]

where \(D_i\) is the treatment assignment indicator for individual \(i\) (1 if treated, 0 if control).

To compute the variance, we need to understand how this estimator behaves across all possible randomizations. The subscript \(D\) in \(\text{Var}_D(\cdot)\) and \(E_D[\cdot]\) reminds us that the randomness comes entirely from the treatment assignment \(D_i\).

5.2 The Variance Operator \(\text{Var}[\cdot]\)

The variance operator \(\text{Var}[\cdot]\) is a way of measuring “how spread out” a random variable is around its average value. The variance of a random variable measures its typical squared distance from the mean. You can think of \(\text{Var}(X)\) as asking: “If I could repeat this random process infinitely many times, how far would \(X\) typically be from its average?”

For our purposes, we need five rules about variance. These rules play the same role that the expectation rules played in the previous chapter. They tell us which algebraic moves are allowed.

Rule 1: Variance is defined as the expected squared deviation from the mean.

If \(X\) is random, then

\[ \text{Var}(X) = E\left[(X - E[X])^2\right]. \]

We square the deviation so that negative and positive deviations do not cancel out. If on average you earn $100 per day, but some days you earn $80 and some days you earn $120, the deviations are \(-20\) and \(+20\). Without squaring, they would cancel to zero, hiding the variability.

Rule 2: Adding a constant does not change variance.

If \(c\) is a constant, then \(\text{Var}(X + c) = \text{Var}(X)\).

Shifting a random variable up or down changes its average value, but does not change how spread out it is. If your daily earnings go from averaging $100 to averaging $200 (because of a $100 raise), the spread around that average stays the same.

Rule 3: Multiplying by a constant scales variance by the square.

If \(c\) is a constant, then \(\text{Var}(cX) = c^2 \text{Var}(X)\).

This rule often surprises people because it differs from expectation. For expectation, constants come out unchanged (\(E[cX] = c \cdot E[X]\)). For variance, constants come out squared. If your earnings double, the spread of your earnings quadruples.

Why does the constant get squared? Variance measures squared deviations. If we double \(X\), the deviations from the mean also double. But since we square those deviations, doubling the deviation means quadrupling the squared deviation.

Example. Suppose \(X\) can be either 1 or 3, each with probability 1/2. Then \(E[X] = 2\), and

\[ \text{Var}(X) = \frac{1}{2}(1 - 2)^2 + \frac{1}{2}(3 - 2)^2 = \frac{1}{2}(1) + \frac{1}{2}(1) = 1. \]

Now consider \(2X\), which can be either 2 or 6. Then \(E[2X] = 4\), and

\[ \text{Var}(2X) = \frac{1}{2}(2 - 4)^2 + \frac{1}{2}(6 - 4)^2 = \frac{1}{2}(4) + \frac{1}{2}(4) = 4. \]

The variance is \(4 = 2^2 \times 1 = c^2 \times \text{Var}(X)\).

Rule 4: The variance of a sum includes cross-terms.

If \(X\) and \(Y\) are random, then

\[ \text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y) + 2\text{Cov}(X, Y), \]

where \(\text{Cov}(X, Y)\) is the covariance between \(X\) and \(Y\). The covariance measures how \(X\) and \(Y\) move together. If \(X\) and \(Y\) tend to be high at the same time, their covariance is positive. If one tends to be high when the other is low, their covariance is negative.

This rule matters because our estimator is built from sums. When we take the variance of a sum, we do not simply get “variance of the first part plus variance of the second part.” We also get cross-terms.

In our setting, these cross-terms are especially important. Under completely randomized assignment, treatment indicators for different individuals are not independent. If one individual is assigned to treatment, that makes it slightly less likely that another individual is assigned to treatment (because there are only \(N_t\) slots). This creates negative covariance between treatment assignments.

The covariance term appears because we square a sum. Think back to algebra: \((a + b)^2 = a^2 + 2ab + b^2\). The “cross-term” \(2ab\) is what becomes the covariance.

Start from the definition, so that

\[ \text{Var}(X + Y) = E\left[\left((X + Y) - E[X + Y]\right)^2\right]. \]

Rewrite the mean as \(E[X + Y] = E[X] + E[Y]\), so that

\[ \text{Var}(X + Y) = E\left[\left((X - E[X]) + (Y - E[Y])\right)^2\right]. \]

Expand the square, which gives

\[ \text{Var}(X + Y) = E\left[(X - E[X])^2\right] + E\left[(Y - E[Y])^2\right] + 2E\left[(X - E[X])(Y - E[Y])\right]. \]

The first term is \(\text{Var}(X)\). The second term is \(\text{Var}(Y)\). The last term is \(2\text{Cov}(X, Y)\) by definition.

Special case: independence. If \(X\) and \(Y\) are independent, their covariance is zero, and the variance of the sum is just the sum of the variances. But in our setting, treatment assignments are not independent, so we cannot ignore the covariance.

Rule 5: If the mean is zero, variance equals the expected square.

If \(E[X] = 0\), then

\[ \text{Var}(X) = E[X^2]. \]

This rule is a useful shortcut. By definition, \(\text{Var}(X) = E[X^2] - (E[X])^2\). If \(E[X] = 0\), the second term vanishes, and we are left with just \(E[X^2]\).

We will use this simplification later by rewriting expressions so that they have mean zero.

5.3 Some Preliminary Results

Before tackling the variance formula, we need to establish some basic facts about the treatment indicators \(D_i\) under completely randomized assignment.

5.3.1 Facts About \(D_i\)

In a completely randomized experiment with \(N\) individuals where \(N_t\) are assigned to treatment, each individual has the same probability of being treated. We have

\[ E_D[D_i] = \Pr(D_i = 1) = \frac{N_t}{N}. \]

Since \(D_i\) is binary (either 0 or 1), we also have \(D_i^2 = D_i\) (squaring a 0 or 1 gives you the same number). This means

\[ E_D[D_i^2] = E_D[D_i] = \frac{N_t}{N}. \]

From these facts, we can compute the variance of \(D_i\), so that

\[ \text{Var}_D(D_i) = E_D[D_i^2] - (E_D[D_i])^2 = \frac{N_t}{N} - \left(\frac{N_t}{N}\right)^2 = \frac{N_t}{N} \left( 1 - \frac{N_t}{N} \right) = \frac{N_t \cdot N_c}{N^2}. \]

The variance formula \(\text{Var}(X) = E[X^2] - (E[X])^2\) is a standard result. Let’s verify it with a concrete example.

Suppose \(N = 10\) and \(N_t = 4\). Then \(E_D[D_i] = 4/10 = 0.4\). The variance is

\[ \text{Var}_D(D_i) = \frac{4}{10} \left(1 - \frac{4}{10}\right) = 0.4 \times 0.6 = 0.24. \]

This is the variance of a Bernoulli random variable with probability \(p = 0.4\), which is \(p(1-p) = 0.24\).

5.3.2 The Correlation Between Treatment Assignments

Here’s where things get interesting. In a completely randomized experiment, the treatment assignments \(D_i\) and \(D_j\) for different individuals are not independent. If one individual is assigned to treatment, that makes it slightly less likely that another individual is assigned to treatment (because there are only \(N_t\) treatment slots to fill).

For \(i \neq j\), we need to compute \(E_D[D_i \cdot D_j]\). This equals the probability that both individuals \(i\) and \(j\) are assigned to treatment.

We can compute this using conditional probability, so that

\[ E_D[D_i \cdot D_j] = \Pr(D_i = 1) \cdot \Pr(D_j = 1 \mid D_i = 1) = \frac{N_t}{N} \cdot \frac{N_t - 1}{N - 1}. \]

The probability that individual \(i\) is treated is \(N_t/N\). Given that individual \(i\) is treated, there are now \(N_t - 1\) remaining treatment slots and \(N - 1\) remaining individuals. So the probability that individual \(j\) is also treated is \((N_t - 1)/(N - 1)\).

Example. Suppose \(N = 5\) and \(N_t = 2\). The probability both individuals 1 and 2 are treated is

\[ \frac{2}{5} \cdot \frac{1}{4} = \frac{2}{20} = 0.1. \]

We can verify this by counting. There are \(\binom{5}{2} = 10\) ways to choose 2 individuals for treatment. Only 1 of these ways assigns both individuals 1 and 2 to treatment. So the probability is \(1/10 = 0.1\).

5.4 A Helpful Transformation

The algebra becomes cleaner if we transform our treatment indicator. Instead of working directly with \(D_i\), we define a centered treatment indicator \(\tilde{D}_i\), so that

\[ \tilde{D}_i = D_i - \frac{N_t}{N}. \]

This transformation shifts \(D_i\) so that its mean is zero. When \(D_i = 1\) (treated), we have \(\tilde{D}_i = 1 - N_t/N = N_c/N\). When \(D_i = 0\) (control), we have \(\tilde{D}_i = 0 - N_t/N = -N_t/N\).

In summary, the centered indicator is

\[ \tilde{D}_i = \begin{cases} \frac{N_c}{N} & \text{if } D_i = 1 \text{ (treatment)}, \\[6pt] -\frac{N_t}{N} & \text{if } D_i = 0 \text{ (control)}. \end{cases} \]

5.4.1 Properties of \(\tilde{D}_i\)

The centered indicator \(\tilde{D}_i\) has two key properties that make it useful.

Property 1: Zero mean. By construction, \(E_D[\tilde{D}_i] = E_D[D_i] - N_t/N = N_t/N - N_t/N = 0\).

Property 2: Known variance. Since \(\tilde{D}_i\) is just a shifted version of \(D_i\), its variance is the same, so that

\[ \text{Var}_D(\tilde{D}_i) = \text{Var}_D(D_i) = \frac{N_t \cdot N_c}{N^2}. \]

And because \(E_D[\tilde{D}_i] = 0\), we have \(E_D[\tilde{D}_i^2] = \text{Var}_D(\tilde{D}_i) = \frac{N_t \cdot N_c}{N^2}\).

5.4.2 The Cross-Product \(\tilde{D}_i \cdot \tilde{D}_j\)

For the variance calculation, we also need \(E_D[\tilde{D}_i \cdot \tilde{D}_j]\) when \(i \neq j\). The product \(\tilde{D}_i \cdot \tilde{D}_j\) can take three possible values depending on whether both individuals are treated, both are in control, or one of each.

\(D_i\) \(D_j\) \(\tilde{D}_i\) \(\tilde{D}_j\) \(\tilde{D}_i \cdot \tilde{D}_j\)
1 1 \(N_c/N\) \(N_c/N\) \(N_c^2/N^2\)
1 0 \(N_c/N\) \(-N_t/N\) \(-N_t N_c/N^2\)
0 1 \(-N_t/N\) \(N_c/N\) \(-N_t N_c/N^2\)
0 0 \(-N_t/N\) \(-N_t/N\) \(N_t^2/N^2\)

The probabilities of each case are:

  • Both treated: \(\frac{N_t(N_t - 1)}{N(N-1)}\)
  • Both control: \(\frac{N_c(N_c - 1)}{N(N-1)}\)
  • One each: \(\frac{2 N_t N_c}{N(N-1)}\)

After computing the expected value (a homework exercise), we obtain

\[ E_D[\tilde{D}_i \cdot \tilde{D}_j] = -\frac{N_t \cdot N_c}{N^2 \cdot (N - 1)} \quad \text{for } i \neq j. \]

Notice this is negative. This reflects the negative correlation between treatment assignments: if individual \(i\) is more likely to be treated (higher \(\tilde{D}_i\)), individual \(j\) is slightly less likely to be treated (lower \(\tilde{D}_j\)).

5.5 Rewriting the Estimator

Using the centered indicator \(\tilde{D}_i\), we can rewrite the estimator in a form that separates the fixed and random components. After algebraic manipulation (shown in the proof below), we obtain

\[ \widehat{\text{ATE}} = \underbrace{\frac{1}{N} \sum_{i=1}^{N} (Y_i(1) - Y_i(0))}_{\text{fixed: the true ATE}} + \underbrace{\frac{1}{N} \sum_{i=1}^{N} \tilde{D}_i \cdot Y_i^+}_{\text{random: depends on } \tilde{D}_i}, \]

where we define

\[ Y_i^+ = \frac{N}{N_t} Y_i(1) + \frac{N}{N_c} Y_i(0). \]

This decomposition is powerful. It says that our estimate equals the true ATE plus a “noise” term that depends on the randomization.

The first term, \(\frac{1}{N} \sum_{i=1}^{N} (Y_i(1) - Y_i(0))\), is the true ATE. It doesn’t depend on the randomization at all—it’s the same no matter which individuals we assign to treatment.

The second term, \(\frac{1}{N} \sum_{i=1}^{N} \tilde{D}_i \cdot Y_i^+\), is the deviation of our estimate from the truth. This term:

  • Has expected value zero (because \(E_D[\tilde{D}_i] = 0\))
  • Varies depending on the randomization
  • Has variance that we want to compute

The quantity \(Y_i^+\) is a weighted combination of individual \(i\)’s potential outcomes. It captures how much that individual’s outcomes contribute to the variability of the estimator.

This decomposition immediately shows that \(\widehat{\text{ATE}}\) is unbiased. Since \(E_D[\tilde{D}_i] = 0\), we have

\[ E_D[\widehat{\text{ATE}}] = \frac{1}{N} \sum_{i=1}^{N} (Y_i(1) - Y_i(0)) + 0 = \text{ATE}. \]

5.6 The Proof

The proof involves careful algebraic manipulation. We break it into labeled steps.

Step 1: Start with the decomposition.

We established that

\[ \widehat{\text{ATE}} = \frac{1}{N} \sum_{i=1}^{N} (Y_i(1) - Y_i(0)) + \frac{1}{N} \sum_{i=1}^{N} \tilde{D}_i \cdot Y_i^+. \]

Since the first term is constant (not random), the variance is

\[ \text{Var}_D \left(\widehat{\text{ATE}}\right) = \text{Var}_D \left( \frac{1}{N} \sum_{i=1}^{N} \tilde{D}_i \cdot Y_i^+ \right) = \frac{1}{N^2} \text{Var}_D \left( \sum_{i=1}^{N} \tilde{D}_i \cdot Y_i^+ \right). \]

Step 2: Use the variance-of-a-sum formula.

Since \(E_D\left[\sum_{i=1}^{N} \tilde{D}_i \cdot Y_i^+\right] = 0\), we have

\[ \text{Var}_D \left( \sum_{i=1}^{N} \tilde{D}_i \cdot Y_i^+ \right) = E_D \left[ \left( \sum_{i=1}^{N} \tilde{D}_i \cdot Y_i^+ \right)^2 \right]. \]

Step 3: Expand the square.

Expanding the squared sum gives

\[ E_D \left[ \left( \sum_{i=1}^{N} \tilde{D}_i \cdot Y_i^+ \right)^2 \right] = E_D \left[ \sum_{i=1}^{N} \sum_{j=1}^{N} \tilde{D}_i \tilde{D}_j Y_i^+ Y_j^+ \right] = \sum_{i=1}^{N} \sum_{j=1}^{N} E_D[\tilde{D}_i \tilde{D}_j] \cdot Y_i^+ \cdot Y_j^+. \]

We can separate this into diagonal terms (\(i = j\)) and off-diagonal terms (\(i \neq j\)), so that

\[ = \sum_{i=1}^{N} E_D[\tilde{D}_i^2] \cdot (Y_i^+)^2 + \sum_{i=1}^{N} \sum_{j \neq i} E_D[\tilde{D}_i \tilde{D}_j] \cdot Y_i^+ \cdot Y_j^+. \]

Step 4: Substitute the expectations.

Using our earlier results:

  • \(E_D[\tilde{D}_i^2] = \frac{N_t N_c}{N^2}\)
  • \(E_D[\tilde{D}_i \tilde{D}_j] = -\frac{N_t N_c}{N^2(N-1)}\) for \(i \neq j\)

This gives

\[ = \frac{N_t N_c}{N^2} \sum_{i=1}^{N} (Y_i^+)^2 - \frac{N_t N_c}{N^2(N-1)} \sum_{i=1}^{N} \sum_{j \neq i} Y_i^+ Y_j^+. \]

Step 5: Simplify using a variance identity.

There’s a useful algebraic identity: for any values \(z_1, \ldots, z_N\) with mean \(\bar{z}\),

\[ (N-1) \sum_{i=1}^{N} (z_i - \bar{z})^2 = N \sum_{i=1}^{N} z_i^2 - \sum_{i=1}^{N} \sum_{j \neq i} z_i z_j. \]

Applying this identity (with some additional algebra that we omit), we can show that

\[ \sum_{i=1}^{N} (Y_i^+)^2 - \frac{1}{N-1} \sum_{i=1}^{N} \sum_{j \neq i} Y_i^+ Y_j^+ = \frac{N}{N-1} \sum_{i=1}^{N} (Y_i^+ - \overline{Y}^+)^2. \]

Step 6: Substitute the definition of \(Y_i^+\).

After substitution and considerable algebraic manipulation, we can show that

\[ \frac{1}{N-1} \sum_{i=1}^{N} (Y_i^+ - \overline{Y}^+)^2 = \frac{N^2}{N_t^2} S_t^2 + \frac{N^2}{N_c^2} S_c^2 + \frac{2N^2}{N_t N_c} \text{Cov}(Y_i(1), Y_i(0)), \]

where \(\text{Cov}(Y_i(1), Y_i(0))\) is the covariance between the potential outcomes.

Step 7: Relate to \(S_{tc}^2\).

The variance of treatment effects can be written as

\[ S_{tc}^2 = S_t^2 + S_c^2 - 2 \text{Cov}(Y_i(1), Y_i(0)). \]

This allows us to substitute for the covariance term.

Step 8: Combine and simplify.

Putting everything together and simplifying (which involves careful bookkeeping of the various terms), we arrive at

\[ \text{Var}_D \left(\widehat{\text{ATE}}\right) = \frac{S_t^2}{N_t} + \frac{S_c^2}{N_c} - \frac{S_{tc}^2}{N}. \]

This completes the proof. \(\square\)