Interactive Lab

RCT Readiness Diagnostic

A randomised controlled trial is expensive, slow and often irreversible once its number lands in a report. Before you commit, work through five honest checks: where your evidence really sits, whether your intervention and theory of change are ready, whether your rollout can support randomisation, whether your baseline can even detect the change you hope for — and whether an RCT is the right tool at all. The gate that should come before the Impact Evaluation Designer.

Where does your evidence actually sit?

An RCT is not the summit you must climb to. It answers one narrow question well: did this specific intervention, delivered this way, cause a change in this outcome, on average, against a credible counterfactual? It does not tell you why it worked, for whom, whether it will travel to another district, or whether the intervention is even designed right. Reaching for it too early wastes money and produces a precise-looking number that answers the wrong question.

The evidence hierarchy — and its fine print. The familiar pyramid ranks the internal validity of an efficacy claim. It is silent on relevance, mechanism and transferability. A method sitting lower on the pyramid can be the better choice when your question is "why is this happening?" or "will it work here?" rather than "does it work on average?"

strongest*
Systematic review / meta-analysis. Pools many studies. *Strongest for an established, repeatedly-tested question — useless if the underlying studies asked the wrong one.
RCT
Randomised controlled trial. Randomisation makes treatment and control comparable in expectation. Answers "does it work?" — not "why", "for whom", or "at scale".
quasi-exp
Quasi-experimental. Difference-in-differences, regression discontinuity, matching. Uses a natural or policy-made comparison when you cannot randomise. Often the honest best fit.
theory-based
Theory-based & case methods. Process tracing, contribution analysis, realist evaluation. Test the causal chain and explain mechanism — what an RCT's black box can't.
observational
Observational & monitoring. Before/after, correlations, routine MIS data. Cheap, fast, essential for management — weak for causal claims on its own.
foundational
Descriptive & practitioner knowledge. Needs assessment, ethnography, field-team judgement. Where good questions are born. Never "just anecdote".

Five things you should have before an RCT is even on the table:

A stable, standardised intervention — not one you are still redesigning every cycle.
A theory of change that has been stress-tested — assumptions surfaced, rival explanations considered.
Monitoring / implementation data showing the programme is actually delivered as designed (fidelity).
Process or qualitative evidence that the causal mechanism is plausible, not just hoped for.
A real decision the trial result will inform — to scale, fund, redesign or stop.

Miss these and an RCT measures a moving, poorly-understood target — and its unrefutable-looking number can lock in the wrong conclusion for years.

So — what stage is your programme in?

Pick the description closest to the truth today. This anchors the readiness verdict at the end.

Is the intervention — and its logic — ready to be tested?

An RCT freezes your programme for the length of the trial and asks whether that exact thing works. If the intervention is still changing, or the theory behind it has never been pressure-tested, you are evaluating a target that won't hold still. Answer each honestly — Yes, Partly, or No / not sure.

Evaluability before evaluation. This is what evaluators call an evaluability assessment — checking a programme is coherent and mature enough to be worth evaluating rigorously, before spending on the evaluation. A "No" here is not a failure; it is a cheaper, faster piece of work to do first.

Does your rollout support randomisation?

Even a mature programme with a solid theory can be impossible — or unethical — to randomise. The way you actually deliver on the ground decides whether a clean treatment-vs-control contrast even exists.

The cluster problem. If you randomise groups (villages, schools, panchayats) rather than individuals, statistical power comes mainly from the number of clusters, not the number of people. Fewer than roughly 20–30 clusters per arm makes an RCT fragile no matter how many individuals you survey. Count your clusters honestly before anything else.

Randomisation must be ethically defensible. Withholding a programme from a control group is only justifiable when you have genuine uncertainty about whether it works (equipoise) and a real constraint — over-subscription, phased rollout, or limited budget — that means some people would not get it soon anyway. "We withheld a known benefit purely to generate a clean number" does not pass an ethics board, and it should not pass yours.

Can your baseline even detect the change?

This is the trap that sinks the most RCTs quietly: the outcome is already so high (or so low) at baseline that there is almost no room left to move it — a ceiling (or floor) effect. If 88% of children are already enrolled, an intervention cannot raise enrolment by 20 points; a realistic 3–4 point gain needs a huge, expensive sample to detect. Check the headroom before you fund the trial.

Primary outcome type

Which way should it move?

Baseline level (proportion, 0–1) Current control-group rate, e.g. 0.88 = 88%.

Change you hope to detect (percentage points) Absolute, e.g. 0.05 = a 5-point change. Be realistic, not aspirational.

Units you can realistically randomise Individuals for individual randomisation; clusters (villages/schools) if you randomise groups.

Randomising groups (clusters)?

What this checks. First, headroom — is there enough distance between your baseline and the ceiling/floor for the change you want? Then a first-pass sample size for that change (two-arm, 80% power, α=0.05, inflated by the cluster design effect), compared against the units you actually have. It is a reality check, not a substitute for a statistician's power analysis. First-draft sizing

Your readiness verdict

This pulls together your programme stage, the intervention and rollout gates, and the baseline check into one honest read. It is a conversation-starter for your team and your funder — not a licence or a veto.

Before you sign the RCT contract: the honest cost ledger

An RCT is not free, neutral, or automatically ethical. A cluster RCT in India routinely runs into crores of rupees and 2–4 years from baseline to published result — often longer than the decision it was meant to inform. That money and time is not spent on programming. Ask whether the same decision could be made well with a faster, cheaper, more explanatory design.

On not turning the sector into a lab. There is a real pull — from some funders and researchers — to make an RCT the default proof-of-worth for every programme, which quietly turns communities into study subjects for questions they did not set and evidence they rarely see. Guard against it: an RCT is justified when there is genuine uncertainty, a decision hanging on the answer, and a design that respects the people in it. Randomising for a publication, a funder's checkbox, or methodological fashion is extraction, not evidence. Whose question is being answered, who owns the data, and who sees the result before the next funding cycle are readiness questions too.

If you're not RCT-ready (yet), these earn their keep

Instead of / before an RCT	Answers the question…	Use when
Evaluability assessment	"Is this programme even ready to be evaluated?"	Intervention or theory of change still maturing.
Theory of change stress-test	"Does our causal logic actually hold up?"	Assumptions untested; rival explanations unexplored.
Process tracing	"Through what mechanism did this outcome happen?"	Single case; you need to explain how, not average whether.
Contribution analysis	"How much did we plausibly contribute, alongside other causes?"	No control group possible; many factors at play.
Realist evaluation	"What works, for whom, in what context?"	Effect clearly varies by setting and sub-group.
Quasi-experimental (DiD, RDD, matching)	"Did it work, using a natural comparison?"	Can't randomise, but a policy cutoff or staggered rollout exists.
Rigorous monitoring + rapid feedback	"Is it being delivered, and are people responding?"	Programme still adapting; you need management data now.

When an RCT genuinely is right: a stable, standardised intervention; a real decision hanging on "does it work on average?"; genuine equipoise; a rollout that supports an ethical control group; enough units (and clusters); and a baseline with room to move. If that's you, the next step is designing it well.

Export your readiness memo

Download a plain-text summary of your stage, gate answers, baseline check and verdict — to share with your team, board or funder before anyone commits a budget.

Working through the whole diagnostic is free. Exporting the memo is a Premium feature (Practitioner plan and up).

You've pressure-tested the decision

Knowing when not to run an RCT — and what to do instead — is as much a mark of evidence literacy as knowing how to design one. If you're ready, carry your answers into the Impact Evaluation Designer.

Evidence Hierarchy Evaluability Theory of Change Power & Baseline Research Ethics

Design your evaluation → Stress-test your Theory of Change Explore more Labs