Subject Pack · S4 · Interactive

Education Programme Evaluation

What "learning outcomes" actually means in the Indian context, how teacher practice connects to student outcomes, NEP 2020 evaluation opportunities, and the TaRL evidence pattern. Walk out with an education evaluation design brief.

4 modules~3 hoursInteractiveIndia-context

Your progress
0% complete
Your Capstone

Education Evaluation Design Brief

Walk in with an education programme. Walk out with an evaluation design brief -- learning outcome measurement, teacher-practice linkage, data plan, and analysis approach. Built automatically from your module answers.

Module 1 . ~25 min

What "learning outcomes" actually means

India has at least four different systems for measuring learning outcomes, and they do not agree with each other. Before designing an education evaluation, you must decide which definition of "learning" you are measuring, and why.

The four measurement systems

SystemWhat it measuresGradesStrengthsLimitations
ASERFoundational literacy and numeracy (can the child read a paragraph? do division?)Ages 5-16 (household-based)Simple, comparable across states/years, household-based avoids school-selection biasFloor-level measure; cannot differentiate above-basic proficiency
NASCurriculum-linked competencies across subjects3, 5, 8, 10Government-administered, nationally representative, aligned to NCFSchool-based (misses out-of-school children), item quality varies
Board examsSubject mastery at secondary/higher secondary10, 12High-stakes, institutionally embeddedRote-focused, not comparable across boards
PISAApplication of knowledge in reading, math, science15-year-oldsInternational comparabilityIndia has taken part only once, in the PISA 2009+ round (Himachal Pradesh and Tamil Nadu)

Choosing your outcome measure

Real-world example

Pratham's Read India uses ASER-type assessments because their programme explicitly targets foundational literacy. The assessment takes a few minutes per child, is administered one-on-one, and produces a clear categorical outcome (cannot read, letter level, word level, paragraph level, story level). This simplicity is a feature, not a limitation -- it matches the programme's theory of change precisely.

Your Learning Outcome Definition

Fill these for your programme. Your answers flow into the capstone.

e.g., "FLN remedial programme, grades 1-3, 80 government schools, Madhya Pradesh"
How will you know the learning gain is due to your programme, not normal schooling?
Saved
Self-check
An ed-tech company reports: "Our app improved learning -- students completed 200% more modules." Why is this NOT a learning outcome finding?
The sample size is too small
Module completion measures engagement/usage, not whether the child actually learned the content
The comparison group is missing
The effect size is implausibly large
Correct. Platform usage metrics (time-on-app, modules completed, levels cleared) are process/engagement measures, not learning outcomes. Learning outcomes require an assessment of what the child can actually do, administered independently of the platform.
Module 2 . ~30 min

Teacher practice vs student outcomes

Most education programmes in India work through teachers -- training them, providing materials, or restructuring classrooms. The evaluation question is whether changes in teacher practice lead to changes in student learning. This link is harder to establish than it appears.

The measurement gap

Teacher training programmes routinely measure: (a) training attendance, (b) teacher knowledge post-training, and (c) teacher satisfaction. None of these predict classroom behaviour change. The critical measure is what teachers actually do differently in the classroom after the training.

Classroom observation protocols

ProtocolWhat it capturesIndia useCost per observation (illustrative)
Stallings SnapshotTime-on-task, activity type (10 snapshots at even intervals across a lesson, each a 15-second scan)Used in many World Bank studies; suits large samplesRs 3,000-4,000
CLASSEmotional support, classroom organisation, instructional supportDeveloped and validated mainly in US classrooms; needs extensive observer trainingRs 5,000-7,000
Teach (World Bank)Time on learning plus 9 elements of teaching practice in 3 areasGrowing use; free World Bank toolRs 3,500-5,000
Custom structured checklistProgramme-specific behaviours (e.g., "uses TLM," "groups students by level")Most common in Indian NGO evaluationsRs 2,000-3,000

The attribution chain

Teacher training changes teacher knowledge (maybe). Teacher knowledge changes teacher practice (sometimes). Teacher practice changes student learning (under specific conditions). Each link requires separate evidence. Most evaluations measure the first link and claim the third.

The Hawthorne problem

Teachers teach differently when observed. A single announced observation captures performance behaviour, not typical practice. Budget for at least two observations per teacher, one unannounced. Fixed-interval coding such as the Stallings snapshot reduces subjective judgement, though it does not remove the observer effect.

Your Teacher Practice Measurement Plan

Design how you will measure teacher practice change. These flow into your capstone.

Saved
Self-check
After a teacher training programme, 95% of teachers report "high satisfaction" and post-test scores show a 30% knowledge gain. Can you conclude that classroom practice has changed?
Yes -- knowledge gain predicts practice change
No -- satisfaction and knowledge are necessary but not sufficient; classroom observation is needed to confirm practice change
Yes, if the sample size is large enough
Only if the training was more than 5 days
Correct. Knowledge gain without classroom observation evidence is the most common over-claim in teacher training evaluations. Teachers may know the content but not change their practice.