Sampling Design Studio
Walk from your evaluation question to a defensible sampling strategy, and learn the statistics as you go. Built for MEL practitioners, emerging evaluators and NGO teams across South Asia.
Free to design & size · exporting the plan and facilitator mode are Membership · nothing leaves your device · not a substitute for a qualified statistician
Start here New to sampling? Read the basics first: seven short sections, no maths, about ten minutes.
01 Why we don't measure everyone
Say your programme reaches 8,000 women across a district and you want to know how many feel confident speaking up in their group meeting. You could interview all 8,000, but it would take months and a budget you don't have. So you don't. You talk to a well-chosen few hundred and use their answers to say something trustworthy about all 8,000.
That's the whole game. The big group you care about is the population. The few you actually talk to are the sample. Measuring everyone is a census; measuring a slice is sampling.
02 The one idea that matters: representativeness
A sample is only useful if it looks like the whole group. If a quarter of your population is landless, roughly a quarter of your sample should be too. When your method systematically favours some kinds of people over others, that's bias, and a biased sample lies to you confidently.
The classic trap is the convenience sample. You interview whoever is easy to reach, the women already at the centre, the households near the road. Easy, and usually wrong, because “easy to reach” is rarely “typical.”
See it: convenience vs random
A village of 300. The amber dots are the 40% who are landless, the people a market-day interview tends to miss. Draw 30 people two ways and compare what each “finds.”
Run it a few times. The convenience number keeps landing well below the truth; the random one hugs 40%. Same size: only the method differs.
03 Random beats big
Beginners assume a bigger sample is always better. It isn't. A small random sample beats a huge biased one, every time. The famous example: in 1936 a US magazine mailed 10 million ballots and got 2.4 million back predicting the wrong election winner, while a pollster with a few thousand well-chosen people got it right. Size didn't save the biased method.
Random selection (every person having a known, fair chance of being picked) is what earns you the right to generalise from the few to the many. Size only helps after you've got the method right.
04 “How sure?” and “how precise?”
Because you asked a sample and not everyone, your answer carries a little wobble. Two plain words describe it:
Margin of error: how wide the wobble is. “62%, give or take 5 points” means the true figure is very likely between 57% and 67%.
Confidence: how often that range would contain the truth if you repeated the whole exercise. “95% confidence” means if you did it 20 times, about 19 of your ranges would capture the real number.
See it: bigger sample, tighter answer
Drag to change the sample size and watch the margin of error (95% confidence) shrink.
Notice the diminishing returns: going from 400 to 1,600 people (four times the cost) only halves the wobble. Precision gets expensive fast, which is exactly why picking the right sample size is a real decision, not “grab as many as you can.”
05 When your sample comes in clumps (clusters)
Often you simply can't get a list of all 8,000 women to draw randomly from. But you can list the 200 villages they live in. So you randomly pick, say, 30 villages, then interview women within those. That's cluster sampling.
It's practical, but there's a catch. People in the same village tend to be alike: same water source, same school, same local prices. So each extra woman you interview in a village you've already visited teaches you a little less than a brand-new independent person would. To get the same precision, you need a somewhat bigger total sample. Statisticians call that penalty the design effect, and the “how alike are people in a cluster” number behind it the intra-cluster correlation.
06 Describing vs comparing
One last fork, because it changes everything about how big your sample must be:
Knowing which one you're doing is the first question in Step 1 below, and now you'll know why it matters.
07 Your vocabulary cheat-sheet
Every word you'll meet in the steps below, in one place:
That is everything the five steps below assume. Step 1 starts with your evaluation question.
What do you need to measure?
Your question decides the maths. Pick the closest match: you can change it later.
How far does your programme reach?
Scale drives your strategy. The wider and more scattered your population, the more you’ll lean on clusters rather than a simple random sample, because a clean list of every individual rarely exists.
We use the scale + unit to suggest a realistic population size in the next step. You can always overwrite it with your own frame.
Whose responses will you measure?
Tick each group you must report on. Every ticked group gets its own sample, and if you must report them separately, that pushes you toward stratified sampling.
How precise must the answer be?
Precision has two layers. First set the statistical rigor you want in a perfect world; then tell us the field reality, clustering and non-response, that inflates it. We show both, so you can see the gap.
Do I need clustering? (design effect)
If you sample whole villages/schools and then survey people inside them, the people in one cluster resemble each other, so you learn less per person. We inflate the sample by a design effect.
Your sampling strategy
A defensible plan you can drop into a proposal or protocol: with the working shown, so a reviewer can check it.
Show the statistics behind these numbers
Proportion: the sample for a target margin of error e at confidence z is
Mean: replace p(1−p) with the variance σ²:
Two-group comparison (impact): per arm,
Finite population correction (when your population N is small):
Design effect for clustering, with m per cluster and ICC ρ:
Finally we divide by (1 − non-response) to get the number you must approach.
Validation: check our maths against the textbooks
Don't take our word for it. These are standard reference values from the sampling literature, computed live by the same engine that sizes your study. They should match to the rounding.
References: Cochran, Sampling Techniques; Cohen, Statistical Power Analysis; Kish, Survey Sampling (design effect).