Bad surveys produce bad data. This pack walks you from research question to construct map to item writing to pilot protocol -- so you end up with an instrument that actually measures what you think it measures.
4 modules~100 minInteractiveIndia-context
Your progress
0% complete
Your Capstone
Drafted 30-Item Instrument with Pilot Plan
By the end you will have a construct map, a drafted instrument section, and a complete pilot protocol. The capstone compiles everything from your module work.
Module 1 -- ~25 min
From research question to construct map
The single most common survey design mistake: jumping straight to writing questions. Before a single item is drafted, you need a construct map -- a structured table that traces every research question to the constructs you need to measure, and every construct to the dimensions and indicators that will become your items.
What is a construct?
A construct is the abstract concept you are trying to measure. "Livelihood resilience" is a construct. "How many income sources does your household have?" is an item that operationalises one dimension of that construct. The gap between the construct and the item is where measurement error lives.
Building the construct map
Research Question
Construct
Dimensions
Sample Indicators
# Items
Has household food security improved?
Food security
Availability, access, utilisation, stability
Meals/day, dietary diversity, coping strategies
8-10
Do women report greater agency?
Women's agency
Decision-making, mobility, economic control
Who decides on health expenditure? Can you visit the market alone?
6-8
Has programme awareness increased?
Programme awareness
Knowledge, recall, attitude
Can you name the scheme? Have you received benefits?
4-5
Worked example
Jeevika (Bihar) wants to survey 500 SHG members on livelihood outcomes after 3 years. Research question: "To what extent have livelihood outcomes improved for SHG members in target blocks?"
Map your research questions to constructs. These save automatically.
One sentence. What do you most need to know?
Who are the respondents? Language, literacy level, setting (household/school/clinic)
Abstract concepts, not specific questions
Rule of thumb: 1 item = 30-45 seconds. 30 items = 15-22 min. Above 40 items, fatigue degrades data quality.
Saved
Self-check
Your colleague drafts a 60-item survey for rural women with low literacy, administered by enumerators. What is the most likely problem?
The items are probably too complex
Respondent fatigue -- answers degrade sharply after 25-30 minutes, especially in field settings
Enumerators cannot handle 60 items
60 items is too few for reliable measurement
Correct. Research consistently shows that response quality drops after ~25 minutes in field-administered surveys, especially with populations unaccustomed to formal questioning. A 60-item survey takes 30-45 min. Cut to 30-35 items or split across two visits.
Module 2 -- ~25 min
Item writing that works in the field
An item (survey question) must do four things: be understood the same way by every respondent, measure only one thing, offer response options that cover the full range, and not lead the respondent toward a particular answer. In Indian field contexts, two additional constraints matter: literacy levels vary enormously, and social desirability bias is strong.
Item writing rules
One concept per item. "Do you save money and plan for emergencies?" is two questions. Split them.
Simple language. Write at the reading level of your least-literate respondent. For enumerator-read surveys, use conversational phrasing.
Avoid leading questions. "Don't you think the programme has helped?" is leading. "How has your situation changed since joining?" is neutral.
Concrete, not abstract. "How empowered do you feel?" is unmeasurable. "In the last month, did you make a decision about a major household purchase on your own?" is concrete.
Time-bound. "In the last 30 days..." or "Since joining the SHG..." -- anchor responses to a specific period.
Balanced response scales. 5-point Likert (Strongly disagree to Strongly agree) or frequency (Never / Rarely / Sometimes / Often / Always). Avoid asymmetric scales.
Worked example -- Bad vs. good items
Bad: "Has the SHG programme empowered you economically and socially?" (double-barrelled, abstract, leading)
Good: "In the last 3 months, how often did you make a decision about spending INR 500 or more without needing permission from another household member?" (Never / Once or twice / Several times / Often / Always)
Draft Your Items
Write 3-5 draft items for your primary construct. Apply the rules above.
Write the full question + response options for each
Saved
Self-check
"Do you feel that the government's nutrition programme has been beneficial for your family's health?" -- what is wrong?
Nothing -- it is a valid question
It should use a Likert scale
It is leading (presupposes benefit), abstract ("feel"), and conflates programme assessment with health outcome
It is too short
Correct. Three problems: "beneficial" leads toward a positive answer, "feel" is abstract and unmeasurable, and it conflates the programme's perceived value with actual health change. Better: "In the last month, how many days did your child eat dal or eggs?" -- concrete, time-bound, one concept.
Module 3 -- ~25 min
Cognitive pretesting and pilot protocol
You have drafted items. Now test whether they work -- before you go to the field with 500 respondents and discover that "decision-making" means something entirely different in Bundelkhand than in your Delhi office.
Cognitive pretesting (5-10 respondents)
Sit with 5-10 people from your target population. Read each item aloud. After they answer, ask:
"What did you think this question was asking?" -- tests comprehension.
"How did you arrive at your answer?" -- tests the cognitive process.
"Was there an answer you wanted to give but couldn't find in the options?" -- tests response coverage.
"Was anything confusing or uncomfortable?" -- tests acceptability.
Pilot protocol (30-50 respondents)
After revising from cognitive pretesting, run a full pilot with 30-50 respondents from the actual target population (but not from the final sample). Track:
Administration time -- average, range, items where respondents hesitate
Skip patterns -- do they work? Do enumerators get confused?
Response distributions -- if 95% give the same answer, the item has no variance and is useless
Missing data -- which items get skipped or refused? Why?
Enumerator feedback -- which items are hard to read, hard to explain, or produce awkward silences?
India-specific pretesting issues
In Indian field surveys, three problems surface repeatedly in pretesting: (1) income questions are resisted or underreported -- use asset proxies and expenditure instead; (2) caste and religion questions require careful placement (never first) and phrasing; (3) women's autonomy items must be pre-tested separately with women to check if they feel safe answering honestly with other household members present.
Your Pretesting Plan
Design your cognitive pretesting and pilot protocol.
How many? Who? Where? Language?
Income, caste, domestic violence, autonomy -- how will you handle them?
Saved
Self-check
Your pilot reveals that 92% of respondents answer "Always" to "Do you feel safe in your neighbourhood?" What should you do?
Keep it -- high safety is a positive finding
Drop or rewrite -- a 92% ceiling means no variance, so the item cannot detect differences or change
Move it to the end of the survey
Add more response options
Correct. An item where 92% give the same answer has a ceiling effect and zero discriminating power. Either rewrite it to probe a more specific dimension (e.g., "In the last month, how often did you avoid going out after dark?") or drop it entirely.
Module 4 -- ~25 min
Revision and finalisation
The pilot data is in. Now revise systematically. Instrument finalisation is not "fix the typos" -- it is a structured process of dropping weak items, rewriting ambiguous ones, reordering for flow, and producing a clean instrument with an enumerator manual.
Revision checklist
Drop zero-variance items -- if 90%+ give the same answer, the item adds noise, not signal.
Rewrite misunderstood items -- cognitive pretest flagged them. Use the respondent's own language as a guide.
Check administration time -- if the pilot average was 28 min and your target is 20, you need to cut 8-10 items.
Order matters -- start with easy, non-threatening items (demographics, programme participation). Place sensitive items (income, autonomy, violence) in the middle, after rapport is built. End with satisfaction or open-ended items.
Enumerator manual -- for every item, write a one-line instruction: when to read options aloud, when to probe, when to accept "don't know," skip logic.
Back-translation check -- if the instrument was translated, have someone who has not seen the English version translate the Hindi/Odia version back into English. Compare.
The enumerator manual is not optional
In India, many survey errors are enumerator errors, not instrument errors. If item 14 says "How many meals did your household eat yesterday?" and the enumerator interprets "household" differently from you, you have 500 inconsistent data points. Write a manual. Train on it. Test enumerators on it before fieldwork.
Your Finalisation Decisions
Document your revision decisions and final instrument structure.
Saved
Self-check
Your final instrument has 33 items. Pilot average was 26 min. Your target population is rural women with low literacy. What should you do?
Accept 26 min -- it is close enough
Cut 5-8 items to reach 20-22 min -- fatigue will be worse in full fieldwork than in pilot
Add items to make it comprehensive
Split into two surveys administered on two different days
Correct. Pilot conditions are typically better than full fieldwork (smaller team, more supervision, novelty effect). Expect the real administration time to be 10-15% longer. Cut to reach 20-22 min to keep data quality high.
Capstone
Your Survey Instrument Brief
Click Build my brief to compile your construct map, draft items, pretesting plan, and finalisation decisions into one document.
Survey Instrument Design Document
Your module answers will be pulled into the artefact below.
Your instrument design document will appear here when you click "Build my brief".