Survey Design Lab

Survey Fundamentals

Before writing a single question, you need clarity on three things: who you are asking, what you need to know, and how you will use the answers.

The survey design triangle

Weak foundation

"Let's do a survey about water." No clear objective, no defined population, no analysis plan. The results will be noise.

Strong foundation

"We need to measure household water insecurity in 3 drought-prone villages to target our tanker distribution." Clear, actionable, specific.

Start with the decision. A good rule: for every question, ask "what decision will this answer change?" If the honest answer is "none", cut the question.

Question types at a glance — tap a card

Multiple choice

Single or multiple select. Easy to analyse. Watch for overlapping or non-exhaustive options.

Likert scale

Agreement/frequency scales (often 5- or 7-point). Standardised and comparable. Watch acquiescence and midpoint bias.

Open-ended

Rich qualitative data. Resource-intensive to code. Use sparingly and with a clear coding plan.

Ranking

Forces prioritisation. Harder to analyse. Best kept to a short list of items.

Matrix / grid

Efficient for similar items. Risk of straight-lining. Break long grids into chunks.

Demographic

Age, gender, caste, income. Sensitive — always allow "Prefer not to say".

South Asia tip: In many rural settings, asking direct income questions tends to produce underreporting. Large household surveys such as India's NFHS instead build a wealth index from asset ownership (e.g. TV, two-wheeler, pucca house, land, cooking fuel). Asset-based measures are usually more reliable and less intrusive than a single income question. Method note

Writing Questions That Work

Bad questions produce bad data. Here are the most common survey-killers — and how to identify them.

Spot the bias. Below are survey questions modelled on real development projects. For each, identify the main problem. These items are teaching examples. Illustrative items

Build Your Survey — Water Insecurity

You are designing a survey for a drought-affected village in Maharashtra. Draft a 5-question instrument using what you have learned. Illustrative scenario

Question 1 · Screening / demographic

Question 2 · Core indicator

Question 3 · Follow-up / consequence

Question 4 · Perception / satisfaction

Very Dissatisfied

Dissatisfied

Neutral

Satisfied

Very Satisfied

Question 5 · Open-ended (use sparingly)

Sampling, Bias & Ethics

A perfect questionnaire sent to the wrong people is still wrong data — and the wrong questions can cause real harm.

Sampling bias vs response bias: These are different problems. Sampling bias is about who ends up in your sample (a flawed frame or non-random selection). Response bias is about how people answer once selected (social desirability, acquiescence, recall error). A large sample fixes neither.

Sampling methods compared

Method	When to use	Main risk
Simple random	A complete population list (frame) exists	Rare in field settings; frame is often incomplete
Stratified	Need representation across caste / gender / wealth	More complex to design and implement
Cluster	Geographically dispersed population; no full frame	Higher design effect (larger sample needed)
Purposive	Qualitative depth or specific expertise	Not statistically generalisable
Snowball	Hard-to-reach / hidden populations	Network bias — sample mirrors referral chains

Translation matters as much as sampling: If you survey across languages, translation quality shapes your data. Current best practice (WHO and the European Social Survey's TRAPD approach) favours a team-based translation, review and adjudication process with cognitive pre-testing in each language. Simple "back-translation" (translate out, translate back, compare) is a useful check but is no longer considered sufficient on its own — it can miss meaning shifts that back-translation happens to reverse.

Ethics: consent under India's DPDP Act, 2023

If your survey collects personal data digitally in India, the Digital Personal Data Protection (DPDP) Act, 2023 applies. Tick the safeguards you can honestly say your survey meets:

Consent is free, specific, informed, unconditional and unambiguous, with a clear affirmative action (Section 6)
Respondents can withdraw consent as easily as they gave it (Section 6)
Data is used only for the specified purpose stated in the consent notice (Sections 5–6)
Children's data (anyone under 18) is processed only with verifiable parental / guardian consent (Section 9)
No tracking, behavioural monitoring or targeted advertising directed at children (Section 9)
Data is not retained longer than necessary and is erased once the purpose is met (Section 8 duties)
Respondents know how to exercise their rights — access, correction, erasure and grievance redress (Sections 11–13)

Common myth to unlearn: Unlike the GDPR (and India's older 2011 SPDI Rules), the DPDP Act, 2023 does not create a separate legal category of "sensitive personal data" with graded safeguards. It regulates all digital personal data under one framework, with extra protection specifically for children and persons with disabilities (Section 9). Ethically you should still handle caste, health and biometric data with special care — but do not cite a "sensitive data" tier that the Act does not contain. Not legal advice

Survey design includes survey conditions. A poorly protected interview can put respondents at risk. For example, questions on domestic violence asked without a genuinely private setting can be overheard by household members and lead to retaliation. Sensitive modules need a private space, trained enumerators, and a clear protocol for referral and safety. Illustrative example

Pilot, Iterate, Deploy

Never deploy a survey you have not tested. A good pilot catches problems that only look obvious in hindsight.

The pilot checklist

Language & translation: Team-translated and reviewed (not back-translation alone)? Local idioms checked with native speakers?
Cognitive interview: A handful of respondents think aloud while answering — what confuses them?
Timing: Does the survey stay short enough to avoid fatigue? (Long instruments degrade data quality.)
Skip logic: All conditional flows tested? No dead ends or orphaned questions?
Device test: Works on the actual tablets / phones the field team will use?
Offline sync: Data saves locally and uploads when connectivity returns?
Enumerator training: Team practised with role-play? Inter-rater consistency checked?
Privacy protocol: Private space secured? Consent (and any recording consent) obtained?

Quick readiness score

Check the boxes above, then calculate your pilot readiness.

Lab complete

You can now design survey instruments that produce clean, actionable data — and protect the people who answer them.

Start with the decision, not the question — every item must earn its place
Spot leading, double-barrelled, jargon-laden and loaded questions
Choose scales deliberately: 5- vs 7-point, neutral vs forced choice, and guard against acquiescence
Distinguish sampling bias from response bias, and translate with a team-based, cognitively tested process
Treat ethics and consent (DPDP Act, 2023) as part of design, not an afterthought
Pilot, iterate, and never deploy an untested instrument

Survey Design Questionnaire Bias Likert Scales Research Ethics South Asia

Recommended next steps

Explore more Labs → Explore more on ImpactMojo