# Welcome — what an attribution claim can carry

A 45-minute session on the sentence at the front of most impact reports, and how much weight it can actually bear.

No technical background needed. If you can read a bar chart you can do this session. Bring a claim from a real report — yours or somebody else's.

Mark each step **done**, or **stuck** to call the facilitator over.

# Three sentences

Read these three. They describe the same programme and the same data.

1. "Households in our programme areas saw incomes rise 18 per cent."
2. "Programme households saw incomes rise 18 per cent, against 11 per cent in comparison areas."
3. "Programme households saw incomes rise 18 per cent, against 11 per cent in comparison areas selected to match on district, landholding and baseline income."

Each sentence claims more than the one before it. Each also requires more from the design.

Write down what the first sentence would be consistent with, other than the programme working. Give yourself two minutes and aim for at least three alternatives.

# The counterfactual, stated plainly

Every attribution claim is a comparison with something that did not happen. That is the whole difficulty: the comparison group exists to stand in for a world you cannot observe.

Sentence 1 has no comparison at all. It is consistent with a good monsoon, a price rise, a national scheme, or the simple fact that people who join programmes differ from people who do not.

Sentence 2 has a comparison, but says nothing about how it was chosen. If comparison areas were the ones the programme *declined* to work in, they may differ in exactly the ways that determine income.

Sentence 3 says how. It is still not proof — matching handles what you measured, never what you did not — but it is a claim a reader can interrogate.

#[quiz] Reading a claim

## "Incomes rose 18 per cent in our programme areas" is consistent with all of these EXCEPT:

- [ ] A good agricultural season across the state
- [ ] Selection of more capable households into the programme
- [x] A demonstration that the programme caused the rise
- [ ] A national price movement affecting everyone

## Matching a comparison group on observed characteristics protects against:

- [ ] All differences between the two groups
- [x] Differences in the characteristics you measured, and nothing else
- [ ] Seasonal variation
- [ ] Measurement error in the outcome

# Grade the claim you brought

Take your real claim and place it on this ladder. Be honest; nobody else needs to see the answer.

**Level 0.** A before-and-after in the programme group only.
**Level 1.** A comparison group exists, selection unexplained.
**Level 2.** A comparison group with a stated selection rule you could check.
**Level 3.** A design where assignment itself is defensible — randomised, a genuine discontinuity, a policy change that hit some units and not others for reasons unrelated to the outcome.

Then write the sentence your evidence can actually support. Usually it is one level below the sentence that was published.

# Saying less, honestly

The practical skill is not running better studies. It is writing the claim your design supports and declining to write the one it does not.

Three phrasings that survive scrutiny:

- "Among participating households, incomes rose 18 per cent. We cannot separate the programme's contribution from the season."
- "The difference between programme and comparison areas was 7 percentage points. Comparison areas were chosen on X and Y and may differ on Z."
- "We do not have a design that supports an attribution claim. Here is what we observed and what we think it is consistent with."

The third is not a failure. It is the sentence most honest reports should contain and almost none do.

#[quiz] Writing the claim

## Your design is a comparison group chosen by programme staff for convenience. The strongest defensible sentence is:

- [ ] The programme increased incomes by 7 percentage points
- [ ] The programme likely increased incomes, subject to limitations
- [x] Programme areas grew 7 points faster; the comparison was not chosen in a way that rules out other explanations
- [ ] No claim can be made from these data

# Close

One thing to take away: find the headline claim in your most recent report and rewrite it at the level your design supports. Keep both versions side by side and notice what you lose.

Usually you lose a little rhetorical force and gain something you can defend in a room with a sceptic in it.

**Going further.** [Causal Inference for Development](https://www.impactmojo.in/courses/causal/) works through the designs at each level of that ladder. Free, thirteen modules.