Subject Pack · S7 · Interactive

Public Policy Evaluation

Policy evaluation vs programme evaluation, political economy awareness in design, implementation tracking and regulatory impact, and welfare-effect estimation approaches. Walk out with a policy evaluation design brief.

4 modules~3 hoursInteractiveIndia-context

Your progress
0% complete
Your Capstone

Policy Evaluation Design Brief

Walk in with a public policy or government scheme. Walk out with an evaluation design brief covering the policy-programme distinction, political economy mapping, implementation tracking, and welfare estimation approach.

Module 1 . ~25 min

Policy evaluation vs programme evaluation

A programme is a bounded intervention with defined beneficiaries, a budget, and a start/end date. A policy is a rule, incentive structure, or institutional arrangement that shapes behaviour across an entire population. Evaluating the two requires different logics.

Key differences

DimensionProgramme evaluationPolicy evaluation
ScopeBounded: specific geographies, beneficiariesUniversal or near-universal: affects entire populations
CounterfactualNon-participants or comparison groupOften no untreated group (universal policies); use pre-post, cross-state, or regression discontinuity
ImplementationControlled by implementing agencyMediated through bureaucracy, politics, and state capacity
Timeline2-5 years typicalEffects unfold over decades; political cycles interrupt
DataPrimary collection usually feasibleOften relies on administrative data (NSSO, PLFS, Census, SECC)

Policy evaluation in India

India has a growing culture of policy evaluation but it remains patchy. NITI Aayog's Development Monitoring and Evaluation Office (DMEO) commissions evaluations of centrally sponsored schemes. DMEO was formed in September 2015 by merging the Programme Evaluation Organisation (PEO) and the Independent Evaluation Office (IEO). State-level evaluation capacity varies widely. Some states have their own bodies, such as the Karnataka Evaluation Authority (set up in 2011); many have little capacity.

Worked example

PM-KISAN (Rs 6,000/year cash transfer to farmer families) was implemented nationally in February 2019. No randomisation was possible because it is a universal entitlement. Designs open to an evaluator include: (a) pre/post comparisons using household survey data, (b) regression discontinuity at the 2-hectare cutoff (in the initial design which had a landholding cap), and (c) difference-in-differences across states with faster vs slower DBT rollout. Each design answers a different question and has different limitations.

Your Policy Scoping

Fill these for your policy/scheme. Answers flow into the capstone.

e.g., "PM-KISAN national cash transfer to farmer families, evaluating effect on input use and yields"
Who or what is the comparison, given universal coverage?
Saved
Self-check
The Right to Education Act (2009) mandated free education for all children aged 6-14. How would you design an evaluation of RTE's effect on learning outcomes?
RCT with treatment and control schools
Pre-post comparison using ASER data from 2006-2014, comparing trends across states with different implementation timing and intensity
Survey of teachers asking if RTE improved learning
Cannot be evaluated because it is universal
Correct. Universal policies like RTE require pre-post or cross-state comparison designs. ASER provides comparable learning data every year from 2005 to 2014, and every two years since 2016, which makes it a rare asset for policy evaluation. Variation in RTE implementation timing across states creates a natural experiment.
Module 2 . ~30 min

Political economy awareness in design

Policy evaluation is never politically neutral. The findings will be used, misused, or ignored depending on who commissioned the evaluation, when in the political cycle it is released, and which stakeholders benefit from the current policy.

The four political economy questions

The evaluator's independence

Department of Expenditure rules require a third-party evaluation before a central scheme is continued into a new Finance Commission cycle (OM of 5 August 2016, restated in the OM of 31 October 2025). The evaluator often depends on the same ministry for future work, which creates an incentive towards favourable findings. Document your quality assurance process explicitly: peer review by named experts, pre-registered analysis plan, data deposited in a public repository.

Your Political Economy Mapping

Map the political economy. These flow into your capstone.

Pre-registration, peer review, data deposit, etc.
Saved
Self-check
You are commissioned to evaluate a state government's flagship nutrition programme 6 months before state elections. What is the most important design decision?
Use the most rigorous method possible (RCT)
Pre-register the analysis plan and methodology before data collection, so findings cannot be selectively reported under political pressure
Delay the evaluation until after elections
Only report positive findings to maintain access
Correct. Pre-registration protects both the evaluator and the funder. When the analysis plan is public before data collection, selective reporting becomes visible. This is the strongest independence safeguard available.