Subject Pack . S5 . Interactive

Health Intervention Evaluation

Outcome framing for NCD, RMNCH+A, and mental health. NFHS-comparable measurement. Health systems indicators vs clinical outcomes. CHW programme evaluation. Walk out with a health evaluation design brief.

4 modules ~3 hours Interactive India-context

Your progress

0% complete

Your Capstone

Health Evaluation Design Brief

Walk in with a health programme. Walk out with an evaluation design brief covering outcome framing, measurement strategy, systems-vs-clinical distinction, and data plan.

Module 1 . ~25 min

Outcome framing: NCD, RMNCH+A, mental health

India's disease burden has shifted. NCDs now account for 65% of deaths (ICMR 2023). But the evaluation infrastructure was built for communicable diseases and maternal/child health. Framing outcomes correctly for the programme type is the first design decision.

Three outcome families

Domain	Typical outcomes	Timeframe to detect change	India data sources
RMNCH+A	ANC registration, institutional delivery, immunisation coverage, exclusive breastfeeding, child stunting/wasting	6-18 months for service uptake; 2-3 years for nutritional outcomes	NFHS, HMIS, ICDS/Poshan Tracker
NCD	Screening rates, diagnosis rates, treatment adherence, blood pressure/glucose control, tobacco cessation	12-24 months for behavioural outcomes; 3-5 years for morbidity/mortality	NFHS-5 NCD module, NCD screening data (NPCDCS), facility records
Mental health	Help-seeking behaviour, symptom severity (PHQ-9, GAD-7), functioning, stigma reduction	3-6 months for help-seeking; 6-12 months for symptom change	No national data source; must collect primary

The outcome ladder

Health evaluations commonly confuse different levels of the outcome ladder:

Output -- camps conducted, screenings done, ASHAs trained (these are NOT outcomes)
Coverage/utilisation -- percentage of target population screened, percentage completing treatment
Behavioural outcome -- treatment adherence, dietary change, tobacco cessation
Health outcome -- blood pressure controlled, HbA1c below 7%, symptom severity reduced
Impact -- morbidity reduction, mortality reduction, DALY averted (rarely measurable in programme evaluations)

Worked example

Sangath's MANAS trial in Goa evaluated collaborative stepped care for depression in primary health centres. The outcome was not "number of counselling sessions delivered" (output) but "remission from depression at 6 months" measured by PHQ-9 score below 5. The RCT found significant remission improvement in PHC settings. This outcome framing -- clinical remission, not service delivery -- made the evidence compelling for NIMHANS and the National Mental Health Programme.

Your Outcome Framing

Fill these for your health programme. Answers flow into the capstone.

Programme name and contexte.g., "NCD screening and management, 50 PHCs, Rajasthan, NPCDCS-linked"

Disease domain

RMNCH+A -- maternal, newborn, child, adolescent health NCD -- non-communicable diseases Mental health Mixed or multi-domain

Primary outcome (one sentence, at which ladder level?)

Secondary outcomes (list 2-3)

Outcome detection timeframe

Saved

Self-check

A diabetes management programme reports: "We conducted 10,000 screenings across 50 PHCs." Is this an outcome finding?

Yes -- screening reach is the key outcome

No -- screenings conducted is an output; the outcome would be screening yield, treatment initiation rate, or glucose control

Yes, if compared to a target

Depends on the sample size

Correct. Screening numbers are outputs. Evaluation-relevant outcomes include: what proportion of screened individuals were diagnosed, what proportion initiated treatment, and what proportion achieved glucose control.

Module 2 . ~30 min

NFHS-comparable measurement

The National Family Health Survey is India's gold-standard household survey for health, nutrition, and demographic indicators. NFHS-5 (2019-21) covers 6.37 lakh households. If your programme targets indicators that NFHS also measures, you have a comparability opportunity -- and a measurement obligation.

Why NFHS comparability matters

Benchmarking -- your programme's district has an NFHS-5 baseline. If you measure the same indicators the same way, you can show where your participants stand relative to the district/state/national average.
Policy credibility -- government decision-makers trust NFHS. If your evaluation uses the same question wording and recall period, your findings are directly comparable.
Secondary data -- NFHS microdata is publicly available. You can construct a quasi-comparison group from NFHS records for your district.

Key NFHS indicators by domain

Domain	NFHS-5 indicator	Exact question wording matters?
ANC	% mothers who had 4+ ANC visits	Yes -- recall period and visit counting differ
Nutrition	% children under 5 stunted (height-for-age below -2SD)	Requires anthropometry with same WHO standards
Immunisation	% children 12-23 months fully immunised	Yes -- card + recall methodology must match
NCD	% women/men with elevated blood pressure	Requires same BP measurement protocol (3 readings, last 2 averaged)
Anaemia	% women 15-49 with Hb below 12 g/dL	Requires HemoCue device or equivalent

The anthropometry trap

Stunting measurement requires trained anthropometrists with calibrated equipment (SECA 213 stadiometer, SECA 876 scale). NFHS uses standardised equipment and measurement protocol. If your evaluation uses different equipment or a different positioning protocol, your stunting estimates are not comparable. Budget Rs 1.5-2L for equipment and training if you plan to measure anthropometry.

Your NFHS Alignment Plan

Map your indicators to NFHS. These flow into your capstone.

Which of your indicators overlap with NFHS-5?

Will you use NFHS-exact question wording? If not, why?

Biomarker or clinical measurement plane.g., BP measurement protocol, anthropometry equipment, HemoCue for Hb

NFHS-5 benchmark values for your district

Will you use NFHS microdata for comparison?

Saved

Self-check

Your evaluation measures stunting using a measuring tape held against a wall, while NFHS uses a SECA stadiometer. Are your stunting estimates comparable?

Yes -- both measure height

No -- measurement error from tape + wall can differ by 1-2 cm, enough to reclassify children across the -2SD threshold

Yes, if you calibrate the tape

Only if both use WHO growth standards

Correct. A 1-2 cm measurement error is enough to move children across the stunting threshold. NFHS comparability requires the same equipment and protocol, not just the same indicator definition.

Module 3 . ~30 min

Health systems indicators vs clinical outcomes

Many health programmes aim to strengthen health systems -- training providers, improving supply chains, building referral networks. Evaluating these is fundamentally different from evaluating a clinical intervention. The distinction matters because funders and programme teams routinely conflate them.

Two different evaluation logics

Clinical intervention evaluation -- does this drug/vaccine/protocol/behaviour change reduce disease? Outcome: clinical endpoint. Design: RCT or quasi-experimental. Example: Does chlorhexidine cord care reduce neonatal sepsis?
Health systems evaluation -- does this organisational change improve service delivery? Outcome: coverage, quality, responsiveness. Design: often implementation research, process evaluation, or stepped-wedge. Example: Does a mentor-training programme for PHC medical officers improve quality of NCD care?

Health systems building blocks (WHO framework)

Building block	Example indicators	India measurement challenge
Service delivery	OPD utilisation, referral completion rate, average wait time	Facility records unreliable; direct observation needed
Health workforce	Vacancy rates, provider competency scores, absenteeism	Sanctioned vs filled posts data from IPHS; absenteeism needs unannounced visits
Health information	HMIS data completeness, reporting timeliness	HMIS data quality varies enormously across states
Essential medicines	Stock-out days, availability of tracer drugs	Facility surveys needed; eAushadhi data available in some states
Governance	VHSNC functionality, Rogi Kalyan Samiti activity	Formal functionality assessments vs actual decision-making

Worked example

Project Asman (Jhpiego India) aimed to reduce maternal and newborn mortality through health systems strengthening in Rajasthan and Maharashtra. The evaluation measured both systems outcomes (provider competency via OSCE, supply chain readiness) and clinical outcomes (case fatality rates, Apgar scores). The key insight: systems improvements preceded clinical improvements by 6-9 months. An evaluation measuring only clinical outcomes at 12 months would have missed that the systems pathway was working.

Your Systems vs Clinical Framework

Distinguish which outcomes are systems-level and which are clinical. These flow into your capstone.

Systems-level outcomes your programme targets

Clinical outcomes (if any) downstream of systems change

Expected time lag between systems and clinical change

Facility-level data collection planFacility survey? Direct observation? Records extraction?

Quality of care measurement approach

Saved

Self-check

A programme trains PHC doctors in NCD management. After 12 months, NCD screening numbers doubled but hypertension control rates did not change. What does this tell you?

The training failed

The systems-level change (screening capacity) worked but the clinical pathway (diagnosis to treatment to control) has a bottleneck downstream

Need more time for clinical outcomes to appear

The data is unreliable

Correct. Increased screening is a systems success. Unchanged control rates suggest a bottleneck -- possibly drug availability, follow-up systems, or patient adherence. The evaluation must measure the full cascade, not just the entry point.

Module 4 . ~25 min

Community health worker programme evaluation

India has the world's largest CHW programme: ~10 lakh ASHAs under the NHM, plus ~14 lakh Anganwadi workers under ICDS. Evaluating CHW programmes requires understanding the dual nature of these workers -- they are simultaneously community members and health system agents.

What to evaluate about CHW programmes

Reach and coverage -- what proportion of the target population does the CHW actually contact? ASHA-soft data shows coverage but not contact quality.
Competency -- does the CHW have the knowledge and skills to deliver the intervention? Measure via OSCE (Objective Structured Clinical Examination) adapted for community-level tasks.
Service quality -- observed quality of counselling, referral appropriateness, follow-up completeness. Requires accompanied home visits.
Community trust and access -- does the community view the CHW as a credible health resource? Qualitative component essential.
CHW well-being -- motivation, burnout, compensation adequacy, safety. ASHA attrition is growing; evaluations that ignore CHW welfare miss a sustainability risk.

The attribution problem in CHW evaluations

ASHAs operate alongside PHC services, Anganwadi services, and community networks. Attributing a health outcome to the ASHA specifically is nearly impossible in routine programme evaluation. Two strategies help:

Dose-response analysis -- compare outcomes for households with high ASHA contact frequency vs low. If the ASHA is driving the outcome, more contact should predict better outcomes.
Process tracing -- for specific cases (e.g., a complicated pregnancy with successful referral), trace the CHW's role in the chain of events. Qualitative, but persuasive for mechanism questions.

The incentive measurement

ASHA compensation is performance-based (Rs 2,000-4,000/month average across states in 2026, with wide variation). Evaluation of CHW programmes must document the compensation structure and its effects on behaviour. If ASHAs are incentivised for institutional deliveries but not for NCD follow-up, do not expect NCD follow-up performance to improve through training alone.

Your CHW Evaluation Design

Design the CHW component. These flow into your capstone.

CHW type and programme role

How will you assess CHW competency?

Service quality measurement plan

Attribution strategy (dose-response, process tracing, or other)

Your honesty-test sentence"This evaluation will tell us ___ and will NOT tell us ___."

Saved

Self-check

An ASHA programme evaluation shows: ASHA knowledge scores improved by 40% after training, but household-level immunisation coverage did not change. What is the most likely explanation?

The training was ineffective

Immunisation coverage was already at ceiling

Knowledge gain did not translate to behaviour change -- possibly due to workload, incentive structure, or vaccine supply constraints outside ASHA control

The immunisation data is unreliable

Correct. This mirrors the teacher-training problem in education: knowledge gain is necessary but not sufficient for behaviour change. The evaluation must examine systemic constraints (vaccine supply, competing demands on ASHA time, incentive alignment) alongside individual competency.

Capstone

Your Health Evaluation Design Brief

Click Build my brief to compile everything.

Health Evaluation Design Brief

Click "Build my brief" -- your module answers will be pulled into the artefact.

Your brief will appear here when you click "Build my brief".

Where to go next on ImpactMojo

Done?

Share this brief with a public health colleague before circulating.

Help us improve: feedback form.

All Practice Packs →