Subject Pack · S8 · Interactive

Media Campaign Evaluation

BCC evaluation -- reach, recall, intent, behaviour. Attribution under noise. Social media metrics vs real-world change. Formative evaluation for iterating campaigns. Walk out with a media campaign evaluation design brief.

4 modules~3 hoursInteractiveIndia-context

Your progress
0% complete
Your Capstone

Media Campaign Evaluation Design Brief

Walk in with a BCC or media campaign. Walk out with an evaluation design brief covering the reach-to-behaviour chain, attribution strategy, digital vs offline measurement, and formative evaluation design.

Module 1 . ~25 min

BCC evaluation -- reach, recall, intent, behaviour

Behaviour Change Communication (BCC) campaigns are India's most common media-based development intervention. Polio, Swachh Bharat, TB-Mukt Bharat, Beti Bachao Beti Padhao -- all rely on media campaigns. Evaluating them requires tracking the full chain from exposure to behaviour change.

The BCC evaluation chain

LevelWhat it measuresHow to measureWhat it tells you
1. ReachDid the audience see/hear the message?Media monitoring, platform analytics, household survey (aided recall)Whether the campaign was distributed, not whether it worked
2. RecallCan the audience remember the message?Unaided and aided recall in survey; message comprehension checkWhether the message was noticed and retained
3. Knowledge/attitude changeDid knowledge or attitudes shift?Pre-post knowledge quiz, attitude scales (Likert)Whether cognitive processing happened
4. IntentDoes the audience intend to act?Behavioural intention questions, commitment devicesWhether the message motivated action (but intent is not behaviour)
5. Behaviour changeDid the audience actually change behaviour?Observed behaviour, service uptake data, self-report with verificationWhether the campaign achieved its goal

Most campaign evaluations stop at Level 2 (recall) and claim success. "70% of the target audience recalled the campaign" means the media plan worked, not that behaviour changed. The gap between recall and behaviour is where most BCC campaigns fail.

Illustrative example (hypothetical figures)

Swachh Bharat Mission's media campaign was one of India's largest BCC efforts. Consider a pattern an evaluation of such a campaign could find (the figures and findings here are hypothetical): campaign recall above 85%, yet recall does not predict toilet construction or use. What predicts use is social norm change at the village level -- once a critical mass of households build toilets, social pressure drives adoption. The media campaign's role would then be to trigger initial demand, while village norms sustain the behaviour. An evaluation would need both media metrics (reach, recall) and community-level outcome data (construction rates, usage observation) to tell this story.

Your BCC Chain Design

Map the chain for your campaign. Answers flow into the capstone.

e.g., "TB-Harega-Desh-Jeetega radio + social media campaign, 3 districts, Madhya Pradesh"
Saved
Self-check
A handwashing campaign reports "82% recall among target mothers." Can you conclude the campaign changed handwashing behaviour?
Yes -- 82% is a high recall rate
No -- recall measures exposure, not behaviour; you need observed handwashing data or structured observation to confirm behaviour change
Yes, if intent was also measured
Only if recall was measured using unaided questions
Correct. Recall is Level 2 in the BCC chain. Behaviour is Level 5. The gap between "I saw the ad" and "I wash hands with soap at critical times" is large. Structured observation of handwashing at key moments (after defecation, before feeding a child) is the gold standard for this outcome.
Module 2 . ~30 min

Attribution under noise

Media campaigns operate in a noisy environment. Your audience sees a great many messages every day. Your handwashing ad competes with product advertising, government messaging, WhatsApp forwards, and peer conversations. Attributing behaviour change to your specific campaign is the hardest problem in media evaluation.

Attribution strategies for media campaigns

The honest position

For most BCC campaigns, clean attribution is impossible. The honest evaluation position is: "We found evidence of a positive association between campaign exposure and behaviour change, but we cannot rule out that other factors contributed." This is more credible than claiming causation from a cross-sectional survey.

Your Attribution Design

Design the attribution approach. These flow into your capstone.

What can and cannot be attributed to the campaign?
Saved
Self-check
You compare handwashing rates between people who recall your campaign (45% handwashing) and those who do not recall it (28% handwashing). Can you say the campaign caused the 17-percentage-point difference?
Yes -- the difference is large and significant
No -- people who recall the campaign are likely different from those who do not (more health-aware, higher education, more media access); the difference reflects self-selection bias alongside any campaign effect
Only if you control for income
Yes, if both groups are from the same district
Correct. Recall-based segmentation suffers from endogeneity. People who recall health campaigns tend to be more health-aware to begin with. The 17-point gap likely overstates the campaign's causal contribution. Use statistical controls (propensity score matching on demographics) or triangulate with a geographic comparison design.