BCC evaluation -- reach, recall, intent, behaviour. Attribution under noise. Social media metrics vs real-world change. Formative evaluation for iterating campaigns. Walk out with a media campaign evaluation design brief.
4 modules~3 hoursInteractiveIndia-context
Your progress
0% complete
Your Capstone
Media Campaign Evaluation Design Brief
Walk in with a BCC or media campaign. Walk out with an evaluation design brief covering the reach-to-behaviour chain, attribution strategy, digital vs offline measurement, and formative evaluation design.
Behaviour Change Communication (BCC) campaigns are India's most common media-based development intervention. Polio, Swachh Bharat, TB-Mukt Bharat, Beti Bachao Beti Padhao -- all rely on media campaigns. Evaluating them requires tracking the full chain from exposure to behaviour change.
The BCC evaluation chain
Level
What it measures
How to measure
What it tells you
1. Reach
Did the audience see/hear the message?
Media monitoring, platform analytics, household survey (aided recall)
Whether the campaign was distributed, not whether it worked
2. Recall
Can the audience remember the message?
Unaided and aided recall in survey; message comprehension check
Whether the message motivated action (but intent is not behaviour)
5. Behaviour change
Did the audience actually change behaviour?
Observed behaviour, service uptake data, self-report with verification
Whether the campaign achieved its goal
Most campaign evaluations stop at Level 2 (recall) and claim success. "70% of the target audience recalled the campaign" means the media plan worked, not that behaviour changed. The gap between recall and behaviour is where most BCC campaigns fail.
Worked example
Swachh Bharat Mission's media campaign was one of India's largest BCC efforts. Campaign recall exceeded 85% nationally. But the evaluation found that recall did not predict toilet construction or use. What predicted toilet use was social norm change at the village level -- when a critical mass of households built toilets, social pressure drove adoption. The media campaign's role was in triggering initial demand, not in sustaining behaviour change. The evaluation needed both media metrics (reach, recall) and community-level outcome data (construction rates, usage observation) to tell this story.
Your BCC Chain Design
Map the chain for your campaign. Answers flow into the capstone.
e.g., "TB-Harega-Desh-Jeetega radio + social media campaign, 3 districts, Madhya Pradesh"
Saved
Self-check
A handwashing campaign reports "82% recall among target mothers." Can you conclude the campaign changed handwashing behaviour?
Yes -- 82% is a high recall rate
No -- recall measures exposure, not behaviour; you need observed handwashing data or structured observation to confirm behaviour change
Yes, if intent was also measured
Only if recall was measured using unaided questions
Correct. Recall is Level 2 in the BCC chain. Behaviour is Level 5. The gap between "I saw the ad" and "I wash hands with soap at critical times" is large. Structured observation of handwashing at key moments (after defecation, before feeding a child) is the gold standard for this outcome.
Module 2 . ~30 min
Attribution under noise
Media campaigns operate in a noisy environment. Your audience sees 3,000+ messages per day. Your handwashing ad competes with product advertising, government messaging, WhatsApp forwards, and peer conversations. Attributing behaviour change to your specific campaign is the hardest problem in media evaluation.
Attribution strategies for media campaigns
Dose-response -- compare behaviour change between high-exposure and low-exposure groups. If higher exposure predicts more change, attribution is stronger. Problem: self-selection (people who seek out health info may be more likely to change).
Geographic variation -- if the campaign ran in some districts but not others, use difference-in-differences. Problem: campaign areas were usually selected for a reason (higher disease burden), which confounds.
Interrupted time series -- track behaviour over time and test whether the campaign launch corresponds to a break in trend. Requires at least 8-10 pre-campaign data points. Problem: other events coincide with campaign timing.
Recall-based segmentation -- split survey respondents into those who recall the campaign and those who do not, then compare behaviour. Problem: recall is endogenous (people who already care about the issue are more likely to recall the campaign).
The honest position
For most BCC campaigns, clean attribution is impossible. The honest evaluation position is: "We found evidence of a positive association between campaign exposure and behaviour change, but we cannot rule out that other factors contributed." This is more credible than claiming causation from a cross-sectional survey.
Your Attribution Design
Design the attribution approach. These flow into your capstone.
What can and cannot be attributed to the campaign?
Saved
Self-check
You compare handwashing rates between people who recall your campaign (45% handwashing) and those who do not recall it (28% handwashing). Can you say the campaign caused the 17-percentage-point difference?
Yes -- the difference is large and significant
No -- people who recall the campaign are likely different from those who do not (more health-aware, higher education, more media access); the difference reflects self-selection bias alongside any campaign effect
Only if you control for income
Yes, if both groups are from the same district
Correct. Recall-based segmentation suffers from endogeneity. People who recall health campaigns tend to be more health-aware to begin with. The 17-point gap likely overstates the campaign's causal contribution. Use statistical controls (propensity score matching on demographics) or triangulate with a geographic comparison design.
Module 3 . ~30 min
Social media metrics vs real-world change
India has 500+ million social media users (2026). Development campaigns increasingly run on Instagram, YouTube, and WhatsApp. Digital metrics are abundant but frequently meaningless for evaluation.
The vanity metrics problem
Impressions -- how many times the ad was displayed. This is media spend, not impact. A bot and a real person count the same.
Engagement -- likes, shares, comments. Engagement measures entertainment value, not behaviour change. A Swachh Bharat meme going viral does not mean anyone built a toilet.
Click-through rate -- useful for tracking interest but does not predict offline behaviour.
Video view counts -- "view" means 3 seconds on Facebook, 30 seconds on YouTube. Neither means comprehension.
Bridging digital and real-world measurement
Digital metric
What it actually measures
Real-world complement needed
Impressions/reach
Distribution success
Survey-based aided recall in target population
Video completion rate
Content holding power
Message comprehension quiz in survey
Engagement rate
Content resonance (not behaviour)
Attitude/norm change survey
Link clicks to services
Intent to act
Service utilisation data (helpline calls, clinic visits)
Worked example
Population Foundation of India's Main Kuch Bhi Kar Sakti Hoon (an entertainment-education TV/web series on family planning and gender) tracked both digital metrics (200M+ YouTube views) and real-world outcomes through a panel survey of regular viewers. The evaluation found that regular viewership was associated with shifts in gender attitudes and increased contraceptive use intent -- but the digital metrics alone could not have established this. The survey component was essential.
Your Digital-to-Real Measurement Plan
Map digital metrics to real-world outcomes. These flow into your capstone.
e.g., link click-through to service data, phone survey of YouTube viewers, etc.
Saved
Self-check
Your menstrual hygiene campaign video got 5 million views on YouTube and 100,000 shares. The funder calls it "a massive success." What is the correct evaluation response?
Agree -- 5M views is exceptional reach
Acknowledge the distribution success, then ask: Did the target audience (adolescent girls in rural UP) see it? Did they understand the message? Did menstrual product use or hygiene practices change?
Disagree -- YouTube views are not evaluation evidence
Suggest running the campaign again with a control group
Correct. 5M views is a media success. But the target population may or may not overlap with YouTube viewers. Even if they do, views do not equal comprehension, and comprehension does not equal behaviour change. The evaluation must bridge from digital metrics to real-world outcomes in the target population.
Module 4 . ~25 min
Formative evaluation for iterating campaigns
The most useful evaluation for a media campaign is often formative, not summative. Formative evaluation provides feedback during the campaign to improve messaging, targeting, and channels in real time.
Formative evaluation methods
Pre-testing -- show campaign materials to a sample of the target audience before launch. Test for: comprehension, emotional resonance, cultural appropriateness, unintended meanings. Can be done in 3-5 focus groups in a week.
Rapid feedback loops -- during the campaign, run weekly online or phone polls (50-100 respondents from the target area) to track recall and message comprehension. Adjust messaging based on findings.
A/B testing -- for digital campaigns, test two versions of the message simultaneously and measure which produces higher engagement or click-through. This is standard in advertising but under-used in development campaigns.
Community listening -- deploy 2-3 field researchers to spend time in communities during the campaign. What are people saying about the campaign? What misunderstandings have emerged? This qualitative intelligence is worth more than recall surveys.
The iteration mindset
Development campaigns are typically designed once and evaluated once. Advertising agencies iterate weekly. The most effective development campaigns (MKBKSH, Polio Sunday) iterated based on real-time feedback. Budget 15-20% of your evaluation resources for formative work during the campaign, not just summative work after it ends.
Your Formative Evaluation Plan
Design the formative component. These flow into your capstone.
Saved
Self-check
Your TB awareness campaign has been running for 3 months. Rapid polls show that 60% of the target audience can recall the campaign but only 15% correctly identify the key message (TB treatment is free at government hospitals). What should you do?
The campaign is working -- 60% recall is high
Revise the messaging immediately -- high recall with low comprehension means the message is noticed but not understood; simplify the key message and re-test
Wait until the campaign ends, then evaluate
Increase the media spend to improve comprehension
Correct. High recall + low comprehension is a messaging problem, not a reach problem. More spend will not fix it. This is exactly the kind of finding that formative evaluation catches and that summative evaluation discovers too late to act on.
Capstone
Your Media Campaign Evaluation Design Brief
Click Build my brief to compile everything.
Media Campaign Evaluation Design Brief
Click "Build my brief" to compile your answers.
Your brief will appear here when you click "Build my brief".