Method Pack · M2 · Interactive

Designing a Survey Instrument

Bad surveys produce bad data. This pack walks you from research question to construct map to item writing to pilot protocol -- so you end up with an instrument that actually measures what you think it measures.

4 modules~100 minInteractiveIndia-context

Your progress
0% complete
Your Capstone

Drafted 30-Item Instrument with Pilot Plan

By the end you will have a construct map, a drafted instrument section, and a complete pilot protocol. The capstone compiles everything from your module work.

Module 1 -- ~25 min

From research question to construct map

The single most common survey design mistake: jumping straight to writing questions. Before a single item is drafted, you need a construct map -- a structured table that traces every research question to the constructs you need to measure, and every construct to the dimensions and indicators that will become your items.

What is a construct?

A construct is the abstract concept you are trying to measure. "Livelihood resilience" is a construct. "How many income sources does your household have?" is an item that operationalises one dimension of that construct. The gap between the construct and the item is where measurement error lives.

Building the construct map

Research QuestionConstructDimensionsSample Indicators# Items
Has household food security improved?Food securityAvailability, access, utilisation, stabilityMeals/day, dietary diversity, coping strategies8-10
Do women report greater agency?Women's agencyDecision-making, mobility, economic controlWho decides on health expenditure? Can you visit the market alone?6-8
Has programme awareness increased?Programme awarenessKnowledge, recall, attitudeCan you name the scheme? Have you received benefits?4-5
Worked example (fictional organisation, hypothetical figures)

Illustrative case: Bhoomijan Livelihoods Society (Bihar), a fictional SHG-promoting organisation, wants to survey 500 SHG members on livelihood outcomes after 3 years. Research question: "To what extent have livelihood outcomes improved for SHG members in target blocks?"

Construct map: Income (3 items), savings behaviour (3 items), asset ownership (4 items), food security (FIES-adapted, 8 items), self-efficacy (4 items), programme satisfaction (3 items), demographics (5 items). Total: ~30 items, estimated 20-25 min administration time.

Your Construct Map

Map your research questions to constructs. These save automatically.

One sentence. What do you most need to know?
Who are the respondents? Language, literacy level, setting (household/school/clinic)
Abstract concepts, not specific questions
Illustrative rule of thumb, to replace with your own pilot timings: 1 item = 30-45 seconds. 30 items = 15-22 min. Above 40 items, fatigue degrades data quality.
Saved
Self-check
Your colleague drafts a 60-item survey for rural women with low literacy, administered by enumerators. What is the most likely problem?
The items are probably too complex
Respondent fatigue -- answers degrade sharply after 25-30 minutes, especially in field settings
Enumerators cannot handle 60 items
60 items is too few for reliable measurement
Correct. Treat about 25 minutes as a practical ceiling for field-administered surveys, since answer quality tends to fall as interviews lengthen, especially with populations unaccustomed to formal questioning. A 60-item survey takes 30-45 min. Cut to 30-35 items or split across two visits.
Module 2 -- ~25 min

Item writing that works in the field

An item (survey question) must do four things: be understood the same way by every respondent, measure only one thing, offer response options that cover the full range, and not lead the respondent toward a particular answer. In Indian field contexts, two additional constraints matter: literacy levels vary enormously, and social desirability bias is strong.

Item writing rules

  1. One concept per item. "Do you save money and plan for emergencies?" is two questions. Split them.
  2. Simple language. Write at the reading level of your least-literate respondent. For enumerator-read surveys, use conversational phrasing.
  3. Avoid leading questions. "Don't you think the programme has helped?" is leading. "How has your situation changed since joining?" is neutral.
  4. Concrete, not abstract. "How empowered do you feel?" is unmeasurable. "In the last month, did you make a decision about a major household purchase on your own?" is concrete.
  5. Time-bound. "In the last 30 days..." or "Since joining the SHG..." -- anchor responses to a specific period.
  6. Balanced response scales. 5-point Likert (Strongly disagree to Strongly agree) or frequency (Never / Rarely / Sometimes / Often / Always). Avoid asymmetric scales.
Worked example -- Bad vs. good items

Bad: "Has the SHG programme empowered you economically and socially?" (double-barrelled, abstract, leading)

Good: "In the last 3 months, how often did you make a decision about spending INR 500 or more without needing permission from another household member?" (Never / Once or twice / Several times / Often / Always)

Draft Your Items

Write 3-5 draft items for your primary construct. Apply the rules above.

Write the full question + response options for each
Saved
Self-check
"Do you feel that the government's nutrition programme has been beneficial for your family's health?" -- what is wrong?
Nothing -- it is a valid question
It should use a Likert scale
It is leading (presupposes benefit), abstract ("feel"), and conflates programme assessment with health outcome
It is too short
Correct. Three problems: "beneficial" leads toward a positive answer, "feel" is abstract and unmeasurable, and it conflates the programme's perceived value with actual health change. Better: "In the last month, how many days did your child eat dal or eggs?" -- concrete, time-bound, one concept.