Subject Pack · S9 · Interactive

Governance Reform Evaluation

Institutional change measurement, citizen-experience metrics, before-after with structural break design, and the political economy of evaluation findings. Walk out with a governance evaluation design brief.

4 modules~3 hoursInteractiveIndia-context

Your progress
0% complete
Your Capstone

Governance Evaluation Design Brief

Walk in with a governance reform initiative. Walk out with an evaluation design brief covering institutional change measurement, citizen-experience metrics, research design, and political economy mapping.

Module 1 . ~25 min

Institutional change -- what can be measured

Governance reform aims to change how institutions function -- making them more transparent, accountable, responsive, or efficient. These are inherently hard to measure because institutions are complex systems, not bounded interventions.

Four dimensions of institutional change

DimensionWhat it capturesExample indicatorsData source
RulesFormal rule changes (laws, orders, SOPs)New SOP adopted, RTI compliance rate, file disposal timelineAdministrative records, gazette notifications
PracticesHow rules are actually implementedAverage service delivery time, absenteeism rates, meeting frequencyMystery client visits, direct observation, administrative data
NormsInformal expectations and cultureCorruption perception, trust in institution, perceived responsivenessCitizen surveys, staff surveys
OutcomesEnd results for citizensService access, grievance resolution, satisfactionCitizen surveys, service delivery data

Most governance evaluations measure rules and claim outcomes. The gap between "the SOP was changed" (rules) and "citizens experienced faster service" (outcomes) is where governance reforms succeed or fail. Measuring practices is the critical middle layer.

Illustrative example (hypothetical figures)

Rajasthan's Bhamashah Yojana digitised social protection benefit delivery. The design and figures below are hypothetical, to show the method; they are not from a published evaluation. An evaluation could measure: (a) rules -- database created, Aadhaar-linked accounts opened (output), (b) practices -- time from application to benefit receipt, number of trips to office (measured via citizen survey), (c) norms -- corruption perception around benefit delivery (pre/post survey), (d) outcomes -- inclusion errors and exclusion errors in benefit targeting. A finding of this shape would matter most, for example one where digitisation cuts the average number of citizen-office trips from, say, 4.2 to 1.8, while exclusion errors rise for households without Aadhaar.

Your Institutional Change Framing

Map the four dimensions for your reform. Answers flow into the capstone.

e.g., "E-governance reform in land records (Bhoomi-type), 3 districts, Karnataka"
Saved
Self-check
A state government digitises its Public Distribution System (PDS). The evaluation reports: "100% of fair price shops now have electronic Point of Sale devices." Is this an outcome finding?
Yes -- full digitisation is the outcome
No -- device installation is a rules/infrastructure change (output); outcomes would be reduced diversion, fewer ghost beneficiaries, or faster service for citizens
Yes, if the devices are functional
Only if compared to a pre-digitisation baseline
Correct. Installing technology is infrastructure, not reform. The evaluation must measure what changed for citizens (service speed, accuracy, leakage reduction) and for the institution (transparency, accountability). Many e-governance evaluations stop at installation counts.
Module 2 . ~30 min

Citizen-experience metrics

Governance exists to serve citizens. The most credible governance evaluation evidence comes from measuring citizen experience directly -- not from administrative self-reports or expert scorecards.

Three approaches to citizen experience measurement

Indian governance data sources

SourceWhat it providesFrequency
Provider absence surveys (World Bank 2003; national resurvey 2010)Provider absenteeism, infrastructure, service qualityTwo national rounds (2003, 2010)
Good Governance Index (DARPG)State-level governance quality across sectorsTwo editions so far (2019; 2020-21, released December 2021)
DISHA dashboardsDistrict-level scheme implementation dataReal-time
RTI compliance dataResponse rates, timelines, appealsAnnual (CIC annual report)
CPGRAMSGrievance registration and disposalReal-time
The social desirability problem

Citizens in India often under-report negative experiences with government services, especially to unknown surveyors. They fear retaliation or believe it is futile. Use indirect questioning techniques: "Some people in this area say that obtaining a caste certificate takes 3 trips to the office and costs Rs 500 in unofficial fees. In your experience, is this more or less than what it actually takes?" This indirect framing yields more honest responses than "Did you pay a bribe?"

Your Citizen Experience Measurement

Design the citizen-side measurement. These flow into your capstone.

Saved
Self-check
You are evaluating a one-stop-centre reform in district offices. The government reports "average service time reduced from 7 days to 2 days." Your citizen survey finds "average service time is 5 days." Why might these differ?
The citizen survey sample is biased
The government measures from application receipt to file closure; citizens measure from first visit (including document preparation, re-visits for corrections) to actual receipt of service -- the citizen journey is longer than the administrative process
The government data is falsified
Citizens remember incorrectly
Correct. The citizen journey includes steps that administrative data does not capture: gathering documents, making initial inquiries, returning for corrections, waiting for notifications. Both measures are valid; they answer different questions. The evaluation should report both and explain the gap.