From Metrics to Decisions

Part III is about closing that gap. Every method in the next five chapters — experiments, regression, fixed effects, difference-in-differences, synthetic control, elasticity — is a different way of building a credible comparison between what we did and what would have happened otherwise. This first article is the framing chapter: it introduces the lightweight tool that should sit upstream of every causal analysis, the Decision Question Card, and lays out the three case families we will return to across Part III.

The Executive Question: What Action Does This Metric Support?

The most common failure in business analytics is launching a modeling project without first defining the lever. A team can spend months building a churn model only to discover that the marketing organization has no retention offer to deploy against the predictions. The model was accurate. The decision was missing.

A useful test: read the question out loud. Does it name a specific action a specific person could take? Compare:

• Metric-focused: "How are pastry sales performing in retail?"
• Decision-ready: "If we run a morning push offering a one-dollar pastry discount to coffee-only weekday app users, will gross margin per user exceed our fifteen-cent threshold over the next two weeks, compared with a randomized holdout?"

The first question gestures at a topic. The second one names the lever, the segment, the outcome, the horizon, the comparison, and the threshold that would justify acting. The second one is what we build analytics around.

The Decision Question Card

Before writing a line of SQL or fitting a single model, fill out a six-line card. It is deliberately short — short enough that a manager and an analyst can agree on it in a fifteen-minute meeting.

1. Action ($D$ or $T$). The specific intervention under managerial control: a price change, a coupon, a feature rollout, a policy shift.
2. Outcome ($Y$). The business metric that should move, including any downstream financial outcome (margin, profit, lifetime value) you do not want to sacrifice.
3. Unit of analysis ($i$). The level at which the action is applied and measured: a customer, a store-week, a region-month.
4. Timing and horizon ($t$). When the intervention starts, when it ends, and the window over which the outcome is measured.
5. Counterfactual comparison. The credible stand-in for what would have happened to the same units in the absence of the action. This is almost always the hardest line of the card.
6. Decision threshold ($\theta$). The minimum effect that would justify acting, after accounting for cost, risk, and operational overhead.

The most common error in this table is not the choice of method, but the choice of unit. If your data warehouse stores transactions but your decision lives at the store-week, analyzing raw transactions silently treats every receipt as an independent observation. Standard errors collapse and the analysis becomes overconfident. The unit of analysis is a decision, not a default.

Why the Counterfactual Is the Whole Game

When a dashboard shows that sales rose after a campaign launched, it is tempting to credit the campaign. That naive conclusion ignores the central equation of decision-making:

The first term we observe. The second term — what the same units would have done in the absence of the action — is never directly observable. Causal analysis is, in the end, the disciplined construction of a credible stand-in for that missing second term.

Part III walks through the four standard ways to construct it:

• Randomization. Random assignment forces the two groups to be statistically identical in expectation. The control group is the counterfactual.
• Regression control. Adjust for observable confounders so that the comparison holds them constant.
• Difference-in-differences. Compare the change in treated units over time with the change in control units, differencing away stable group-level differences and common time shocks.
• Synthetic control. Construct a weighted combination of untreated units that tracks the treated unit's pre-intervention trajectory, then use that combination as the counterfactual after the intervention.

Each method makes the counterfactual more credible at the cost of stronger assumptions. The Decision Question Card forces you to name those assumptions up front.

Looking Ahead: The Part III Case Spine

Part III returns repeatedly to four case families so that students develop depth rather than collecting one-off examples. The cases below appear inside the relevant method chapters as clearly labelled data case sections, not as the conceptual material itself.

1. Milk field quasi-experiment. Scanner data from roughly 1,700 supermarkets. Whole milk is priced flat with low-fat alternatives in some stores and slightly above them in others. We use this case for randomization diagnostics, placebo checks, and heterogeneous treatment effects by ZIP-code income.
2. Zillow Colorado housing study. State-month Zillow Home Value Index series surrounding Colorado's January 2014 cannabis legalization. A natural setting for difference-in-differences and synthetic control when only one unit is treated.
3. Progresso soup scanner panel. Weekly transactions across about 2,000 grocery stores. The workhorse for omitted-variable bias, panel regressions with store and week fixed effects, own- and cross-price elasticities, and optimal pricing.
4. Southwest Airlines route-fare data. Route-level airfares used to ask whether Southwest's presence on a route lowers fares once distance and competition are held constant. This case opens Chapter 6's regression refresher before the chapter climbs to Frisch-Waugh-Lovell, identification, and panel effects.

Chapter 8 brings two of these threads back together: it revisits the Progresso panel to price winter and non-winter elasticities separately with the Lerner rule, then asks whether Progresso's real habit of raising prices in the off-season is actually justified by the data — a bridge into modern dynamic and algorithmic pricing.

A Note on the Artefact Family

The Decision Question Card introduced here is the parent of an artefact family that recurs across the book. The same six-line discipline reappears as the Predictive Task Contract (§9.2) for supervised models, the Model Card (§10.5) for deployed predictions, and the AI Workflow Card (§16.4) for LLM systems. Each card extends the discipline of the one above, and the family converges on the decision memo — the one-page synthesis that ties every form of evidence into a recommendation an executive can sign. The full family map is in §0.1.