Monitoring, Evaluation & Learning

Why Study MEL?

MEL in development differs from academic research in crucial ways: it must balance rigor with practicality, serve both accountability and learning purposes, and operate within complex institutional environments where power dynamics shape what gets measured and how.

Evidence-Based Practice

Move beyond anecdotes to systematic evidence. Learn to design indicator systems, collect quality data, and analyze results that drive real decisions.

Methodological Range

Master the full spectrum—from RCTs to contribution analysis, from household surveys to Most Significant Change. Know when to use which approach.

South Asian Context

Deep focus on India, Bangladesh, and the region—with case studies from MGNREGA, Pratham, BRAC, and J-PAL South Asia that show MEL in action.

"What gets measured gets managed—but what gets measured is often what's easy to measure, not what matters most." — Carol Weiss, Evaluation Theorist

Module 1: MEL—History, Variations & Foundations

A Brief History of Development MEL

Results-based thinking in development emerged from surprisingly military origins. Understanding this history illuminates many of today's tensions.

The Logframe's Military Origins: The Logical Framework was developed in 1969 by Practical Concepts Inc. for USAID, adapting systems engineering approaches from the U.S. Department of Defense. Its linear, hierarchical structure reflects Cold War-era faith in rational planning—an approach that has proven both durable and deeply contested.

"The logframe has been compared to the curate's egg—good in parts. It can be a useful thinking and planning tool, but its rigid application has done real damage to development effectiveness." — Robert Chambers, IDS Sussex

The Alphabet Soup: MEL, MEAL, MLE, and More

Different organizations use different acronyms, reflecting distinct institutional histories and priorities. Understanding these variations is essential for navigating the field:

Vandana

Don't get too hung up on acronyms—they often reflect institutional politics more than substantive differences. What matters is whether your evidence system serves both accountability and learning, whoever your stakeholders are.

Book 1:1 Coaching

Humanitarian vs. Development MEL

One of the most important distinctions in the field is between humanitarian and development contexts. Each has distinct evidence needs, timeframes, and ethical considerations:

South Asia Case: The Nexus Challenge

In contexts like Bangladesh (Rohingya response), Nepal (post-earthquake), and India (disaster-prone regions), organizations increasingly work across the humanitarian-development "nexus." This requires hybrid MEL approaches: rapid humanitarian monitoring that can transition into longer-term development evaluation.

BRAC's response to the Rohingya crisis exemplifies this: initial MEAL focused on immediate service delivery, gradually transitioning to development-oriented indicators around livelihoods and social cohesion.

The Logframe: Enduring Tool, Enduring Critiques

The Logical Framework Approach (LFA) remains the most widely used planning and monitoring tool in development, despite decades of criticism. Understanding both its utility and limitations is essential:

The Rise of Theory of Change

Theory of Change (ToC) emerged partly as a response to logframe limitations. Pioneered by Carol Weiss and further developed through the Aspen Institute's work on community initiatives in the 1990s, ToC emphasizes understanding how and why change happens, not just what changes.

Logframe vs. Theory of Change:

Logframe asks: What will we do? What will we deliver? How will we know?

Theory of Change asks: Why do we believe this will work? What must be true for success? What could invalidate our assumptions?

In practice, most organizations now use both: ToC for strategic thinking and program design, logframes for operational tracking and reporting.

The Accountability-Learning Tension

Every MEL system faces a fundamental challenge: the same evidence that demonstrates success to donors might also reveal failures that could threaten funding. This creates powerful incentives that can distort what gets measured, how it's measured, and what gets reported.

"Evaluation serves many masters... The same data that an agency uses to improve its programs may be used by others to criticize and cut the agency's budget. The tension between accountability and improvement is fundamental, and there is no easy resolution." — Carol Weiss, Evaluation: Methods for Studying Programs and Policies (1998), p. 29

Results-Based Management (RBM)

The dominant paradigm in development MEL is Results-Based Management, which organizes interventions around a "results chain" connecting resources to impact:

The Results Chain: Inputs → Activities → Outputs → Outcomes → Impact

• Inputs: Resources invested (funding, staff, materials)
• Activities: What the program does (training, service delivery)
• Outputs: Direct products (people trained, services delivered)
• Outcomes: Changes in behavior, knowledge, practice
• Impact: Long-term changes in well-being

Major Donor MEL Frameworks

Different development agencies have evolved distinct approaches to MEL, reflecting their institutional priorities and cultures:

Case Study: MGNREGA Social Audits

South Asia Case: India's MGNREGA (Mahatma Gandhi National Rural Employment Guarantee Act) includes a legally mandated social audit system. In Andhra Pradesh, these community-led audits have recovered over ₹350 crore (~$42 million) in misappropriated funds since 2006.

The Andhra Pradesh model shows how MEL can be redesigned for accountability to beneficiaries rather than just upward to donors—with village assemblies publicly reviewing records, workers verifying payments, and independent audit teams facilitating the process.

Varna

The tension between donor-centric and community-centric MEL is one I've navigated throughout my career. Want to discuss how to build MEL systems that genuinely serve beneficiaries while meeting funder requirements? Let's talk.

Book 1:1 Coaching Try Interactive Labs

Module 2: Theory of Change & Logic Models

"A theory of change approach to evaluation examines the theoretical assumptions underlying the program and tests them against reality. It's not enough to know whether a program works; we need to know how and why it works, and under what conditions." — Carol Weiss, "Nothing as Practical as Good Theory," American Journal of Evaluation (1995), p. 69

Components of a Theory of Change

A robust Theory of Change includes several interconnected elements that together tell the story of how change happens:

The IF-THEN-BECAUSE Framework:

IF we provide [activities/interventions], THEN we will see [outcomes], BECAUSE [causal mechanism/assumption].

Example: IF we train community health workers in newborn care, THEN neonatal mortality will decrease, BECAUSE trained CHWs will identify danger signs early and refer appropriately.

Logframe vs. Theory of Change

The Logical Framework (logframe) and Theory of Change are related but distinct tools. Understanding their differences helps you use each appropriately:

The "Missing Middle" Problem

Many Theories of Change suffer from a "missing middle"—a gap between activities and outcomes where the mechanisms of change are assumed rather than articulated. This makes it impossible to understand why a program works or doesn't work.

Theory Failure vs. Implementation Failure

When a program doesn't achieve expected outcomes, there are two fundamentally different explanations—and the distinction matters enormously for what you do next:

Why This Matters: Many programs are abandoned prematurely because evaluators couldn't distinguish theory failure from implementation failure. A well-designed ToC with indicators at each link in the causal chain allows you to diagnose where the breakdown occurred—and respond appropriately.

Alternative Frameworks

Case Study: Pratham's Teaching at the Right Level

South Asia Case: Pratham's Teaching at the Right Level (TaRL) program evolved through explicit Theory of Change iteration. The original theory—"build more schools → more children in school → better learning"—failed when data showed enrollment increasing but learning stagnating.

The revised theory: "Assess children's actual level → Group by learning level (not grade) → Targeted instruction → Foundational skills acquisition → Improved learning outcomes."

This ToC revision, informed by ASER data, led to the TaRL methodology now replicated in 14 countries through the PAL Network, with RCT evidence showing learning gains of 0.6-0.7 standard deviations.

Vandana

Theory of Change should be a living hypothesis, not a static document filed away after proposal submission. I've seen teams transform their programs by treating ToC as a learning tool. Let me help you develop one that actually drives decisions.

Book 1:1 Coaching

Try it in the Lab

Build your Theory of Change interactively with our MLE Framework Lab. Map causal pathways, surface assumptions, and generate a visual ToC diagram you can share with stakeholders.

Open MLE Framework Lab

Module 3: Indicator Design & Frameworks

"The choice of indicators is a political act. Indicators determine what gets attention, what gets resources, and ultimately what gets done. The seemingly technical question of 'what should we measure?' is actually a normative question about 'what do we value?'" — Robert Chambers, Whose Reality Counts? (1997), p. 58

Five Indicator Frameworks

Different frameworks offer different criteria for what makes a good indicator. The choice depends on your context, values, and methodological approach:

When to Use Which Framework

Types of Indicators

Indicators can be classified along several dimensions:

• By results level: Input, output, outcome, impact indicators
• By type: Process vs. performance indicators
• By timing: Leading (predictive) vs. lagging (confirmatory) indicators
• By directness: Direct vs. proxy indicators

Disaggregation

Disaggregation breaks down aggregate data to reveal who is (or isn't) benefiting. Common disaggregation dimensions include:

• Sex/Gender: Male, female, other (as culturally appropriate)
• Age: Age bands relevant to intervention (0-5, 6-14, 15-24, etc.)
• Geography: Urban/rural, district, block, village
• Wealth: Quintiles, BPL status, asset indices
• Disability: Washington Group questions
• Caste/Ethnicity: SC, ST, OBC, minority status (context-specific)

Common Indicator Pitfalls

Case Study: ASER's Simple Indicators

South Asia Case: The Annual Status of Education Report (ASER) in India uses remarkably simple indicators: "Can a child read a short story?" and "Can a child do two-digit subtraction?" These binary, foundational measures have proven more powerful for driving policy than complex standardized tests.

By 2023, the ASER methodology had been replicated in 14 countries through the PAL Network, demonstrating that simple, well-chosen indicators can have more impact than sophisticated but opaque measures.

Varna

The most powerful indicators are often the simplest. I've seen organizations drown in 50-indicator logframes while missing the 3-4 measures that actually matter. Let's work together to identify what really counts in your context.

Book 1:1 Coaching

Try it in the Lab

Design your indicator framework with the MEL Design Lab. Apply SMART/CREAM criteria interactively, map indicators to results levels, and export a ready-to-use indicator matrix.

Open MEL Design Lab

Module 4: Baselines, Targets & Benchmarks

Why Baselines Matter

Baselines serve four critical functions in MEL systems:

1. Establish starting conditions before intervention begins
2. Enable change measurement by comparison with later data
3. Inform program design by revealing actual (vs. assumed) conditions
4. Set realistic targets based on evidence rather than aspiration

Target-Setting Approaches

The Problem with Over-Ambitious Targets: CGD research on World Bank projects found that projects with more ambitious targets were more likely to be rated as failures—not because they achieved less, but because they promised more. Set targets based on evidence of what's achievable, not donor expectations.

Module 5: Data Collection Methods

Quantitative Methods

Qualitative Methods

Digital Data Collection

Mobile data collection tools like KoBoToolbox, ODK, and SurveyCTO have transformed field data collection:

• Skip logic: Questions appear based on previous answers
• Validation: Built-in range checks and consistency rules
• GPS capture: Automatic location tagging
• Real-time monitoring: Daily data quality dashboards

Varna

Match your methods to your actual decision needs. A quick qualitative study that informs real decisions is more valuable than a rigorous survey whose results arrive too late to matter.

Book 1:1 Coaching

Try it in the Lab

Plan your data collection strategy with the MEL Design Lab. Define data sources, sampling approaches, and collection methods for each indicator—with built-in guidance on method selection.

Open MEL Design Lab

Module 6: Data Quality Assurance

USAID's Five DQA Dimensions

The 10% Back-Check Rule

Standard practice: re-contact 10% of respondents (randomly selected) to verify key responses. This deters fabrication and identifies systematic errors. Back-checks should occur within 48 hours of original data collection.

Module 7: Quantitative Analysis

Descriptive Statistics

Start with the basics: means, medians, percentages, and distributions. These simple measures often provide 80% of the insight.

When to Use Median vs. Mean: For income, wealth, and other skewed distributions, the median (middle value) better represents the "typical" case than the mean (average), which is pulled by outliers.

Comparing Groups

• Percentage point change: From 40% to 60% = 20 percentage point increase
• Percent change: From 40% to 60% = 50% increase
• Difference-in-differences: Change in treatment minus change in comparison
• Effect size: Standardized difference (Cohen's d, standard deviations)

Common Misunderstandings in MEL Data

Even experienced practitioners fall into these conceptual traps. Understanding these distinctions will save you from costly misinterpretations:

1. Internal Validity vs. External Validity

Practitioner Warning: A study with high internal validity showing a program "works" in one context may not replicate elsewhere. Before scaling, ask: What was special about the study context? Were implementers unusually motivated? Was the population selected for likely success?

2. Accuracy vs. Precision

MEL Implication: Small sample sizes give you imprecision (wide confidence intervals). Systematic measurement errors give you inaccuracy (bias). Both can coexist—you can be both imprecise AND inaccurate.

3. Statistical Significance vs. Effect Size vs. Practical Magnitude

The Three Questions You Must Ask:

• Statistical significance (p-value): Could this result have occurred by chance? (p < 0.05 means <5% chance)
• Effect size (Cohen's d): How large is the effect in standardized units? (d=0.2 small, 0.5 medium, 0.8 large)
• Practical magnitude: Does this effect matter for real-world decisions?

"Statistical significance is not the same as practical importance. A treatment effect can be highly significant statistically but trivial in magnitude, or large in magnitude but statistically insignificant due to small sample size." — Gertler et al., Impact Evaluation in Practice (2016), p. 89

4. Why Averages Can Mislead

The mean (average) is heavily influenced by outliers and skewed distributions. Consider this example:

Example: Village Income Data

10 households with incomes: ₹8K, ₹10K, ₹12K, ₹12K, ₹15K, ₹15K, ₹18K, ₹20K, ₹25K, ₹500K

• Mean: ₹63,500 (pulled up by one wealthy household)
• Median: ₹15,000 (middle value—better represents "typical" household)
• Standard Deviation: ₹153,000 (huge spread indicates outliers)

Reporting "average income is ₹63,500" would grossly misrepresent village welfare.

Rule of Thumb: For income, wealth, land holdings, and other variables with right-skewed distributions, always report the median alongside the mean, and consider the standard deviation to understand spread.

5. Cumulative vs. Current Year Numbers

This is one of the most common sources of confusion in program reporting:

Watch For: Programs may report cumulative numbers to inflate apparent reach, or fail to deduplicate beneficiaries who receive multiple services. Always ask: "Is this cumulative or current year?" and "Are these unique individuals or service contacts?"

Varna

Prioritize actionable insight over statistical sophistication. A clear bar chart that shows the problem is more useful than a regression table that no one reads.

Book 1:1 Coaching

Module 8: Qualitative Analysis

Major Approaches

The Coding Process

1. Familiarization: Read through all data multiple times
2. Initial coding: Label meaningful segments
3. Code consolidation: Group similar codes
4. Theme development: Identify higher-level patterns
5. Theme review: Check themes against data
6. Interpretation: Explain what themes mean

Vandana

Good qualitative analysis is systematic, not just "picking good quotes." Let me show you how to build credible interpretations that stand up to scrutiny.

Book 1:1 Coaching

Module 9: Evaluation Design & the OECD-DAC Framework

The OECD-DAC Evaluation Criteria: A Deep Dive

Established in 1991 and revised in 2019, the OECD Development Assistance Committee (DAC) criteria provide the most widely used framework for structuring development evaluations. The 2019 revision added Coherence as a sixth criterion and refined definitions to emphasize equity and human rights.

Why These Criteria Matter: The DAC criteria shape how billions of dollars in development spending are evaluated. Understanding them isn't just academic—they structure donor expectations, evaluation ToRs, and funding decisions. Mastering these criteria is essential for any MEL professional.

Operationalizing Each Criterion

Vandana

The DAC criteria are a starting point, not a straitjacket. Good evaluators adapt and prioritize based on the evaluation purpose and stakeholder needs. Don't try to answer every sub-question for every criterion—focus on what matters most for decision-making.

Book 1:1 Coaching

Beyond DAC: Emerging Evaluation Principles

The 2019 DAC revision also emphasized cross-cutting principles that should inform all evaluation work:

• Equity and inclusion: Who benefits and who is left behind? Are results disaggregated?
• Gender equality: How does the intervention affect gender relations and women's empowerment?
• Human rights: Does the intervention respect and advance human rights?
• Environmental sustainability: What are the environmental implications?
• Leave no one behind: Are the most marginalized populations reached?

Evaluation Design Choices

Experimental Designs

Randomized Controlled Trials (RCTs) randomly assign units to treatment and control groups. When well-implemented, they provide the most credible evidence of causal impact.

Quasi-Experimental Designs

Theory-Based Evaluation

Choosing an Evaluation Design

Design Selection Framework:

Ask these questions in sequence:

1. What decisions will this evaluation inform? (Determines rigor needed)
2. Is random assignment ethically and practically feasible? (If yes, consider RCT)
3. Is there a natural comparison group or cutoff? (If yes, consider quasi-experimental)
4. Do we need to understand mechanisms and context? (If yes, add theory-based elements)
5. What's the budget and timeline? (Constrains options)

Case Study: J-PAL South Asia

South Asia Case: J-PAL South Asia (based at IFMR) has conducted over 200 RCTs in the region since 2007. Their work on Teaching at the Right Level showed learning gains of 0.6-0.7 standard deviations—evidence strong enough to influence education policy across India, Africa, and beyond.

But J-PAL also demonstrates the limits of RCTs: their evaluations require 2-3 years, limiting their usefulness for rapid program adaptation. They've increasingly complemented RCTs with process evaluations and cost-effectiveness analyses.

Varna

Don't let the perfect be the enemy of the good. A well-designed quasi-experimental study that answers your actual questions is more useful than an RCT that answers questions you didn't need to ask. Match your design to your decision needs.

Book 1:1 Coaching

Module 10: Learning Systems & Adaptive Management

USAID CLA Framework

USAID's Collaborating, Learning, and Adapting framework emphasizes three interconnected components:

Adaptive Management Practices

• Pause-and-reflect sessions: Regular (quarterly) team sessions to review data and discuss implications
• Decision logs: Document what was decided, why, and based on what evidence
• Fail-fast mechanisms: Early warning indicators that trigger review
• Stakeholder feedback loops: Regular input from beneficiaries and partners
• Flexible funding: Budget mechanisms that allow reallocation based on learning

Case Study: BRAC's Research & Evaluation Division

South Asia Case: BRAC established its Research and Evaluation Division (RED) in 1975—one of the earliest examples of an NGO building in-house research capacity. RED operates with significant independence, enabling honest assessment rather than just validation of programs.

This institutional design has enabled genuine learning: RED's research on the Ultra-Poor Graduation program led to iterative refinements that increased effectiveness. The program now operates in over 50 countries.

Vandana

Learning systems require more than good intentions—they need institutional design. Let me help you identify the structural changes needed to make learning happen in your organization.

Book 1:1 Coaching

Module 11: MEL Technology & Data Systems

Mobile Data Collection Tools

Data Visualization & Dashboards

Good visualization makes data accessible to decision-makers. Key principles:

• Start with decisions: What actions might the dashboard inform?
• Less is more: Focus on 5-7 key metrics, not 50
• Update frequency: Match refresh rate to decision cycles
• Mobile-first: Many users will access on phones

Try it in the Lab

Build your data management plan with the MEL Plan Lab. Define data flows, storage systems, quality protocols, and reporting rhythms—with templates you can customize for your context.

Open MEL Plan Lab

Module 12: MEL Leadership & Sustainability

Building MEL Culture

Technical systems alone don't create learning organizations. Building MEL culture requires:

• Leadership commitment: Senior leaders must model evidence use
• Psychological safety: People must feel safe surfacing problems
• Time and space: Reflection requires protected time
• Incentives alignment: Reward learning, not just success stories
• Capacity building: Invest in people, not just systems

Sustainability Considerations

Module 13: Capstone Project

Module 14: Building Your Own MEL System

"A good MEL system is like good plumbing—when it works, no one notices it. When it fails, everyone suffers. The goal is invisible infrastructure that makes evidence-based decision-making the path of least resistance." — adapted from USAID Learning Lab

When to Build vs. Adapt

Before building a new MEL system, ask whether you need to build from scratch or can adapt existing infrastructure:

The Six-Step MEL System Building Framework

1

Stakeholder Mapping & MEL Purpose Definition

Before designing any indicators or data collection, clarify who needs what evidence for which decisions. This shapes everything else.

Stakeholder	Primary Evidence Needs	Decision Types	Frequency
Donors/Funders	Outcome achievement, value for money	Continue/modify funding	Quarterly/Annual
Program Managers	Implementation fidelity, bottlenecks	Tactical adjustments	Weekly/Monthly
Frontline Staff	Individual case progress	Service delivery	Daily/Weekly
Beneficiaries	Service quality, responsiveness	Engagement; feedback	Ongoing

The Accountability-Learning Balance: Every MEL system must serve both accountability (proving value to funders) and learning (improving programs). Explicitly decide your balance—a 70/30 split toward accountability is common, but learning-focused organizations might aim for 50/50.

2

Theory of Change → Logframe Translation

Transform your strategic theory into operational monitoring architecture. The Theory of Change captures why you believe the program will work; the logframe operationalizes what you'll track.

• Articulate the Theory of Change: Map causal pathways, surface assumptions, identify intermediate outcomes
• Identify Critical Assumptions: Which assumptions, if wrong, would invalidate the entire theory?
• Translate to Results Framework: Structure into Impact → Outcomes → Outputs → Activities
• Build the Logframe: Add indicators, means of verification, assumptions column

Try it in the Lab

Use the MLE Framework Lab to build your Theory of Change interactively. Map pathways, add assumptions, and export a visual diagram plus logframe template.

Open MLE Framework Lab

3

Indicator Selection & Operationalization

Select indicators that balance rigor, feasibility, and utility. The goal is the minimum set that enables the decisions identified in Step 1.

The Indicator Selection Funnel

1. Long List — Brainstorm all possible indicators (typically 30-50)
2. Apply CREAM/SMART — Score each indicator on quality criteria
3. Map to Decisions — Which decisions does each indicator inform?
4. Assess Feasibility — Cost, data source availability, staff capacity
5. Final Selection — 8-15 key performance indicators

Try it in the Lab

Build your indicator framework interactively in the MEL Design Lab. Apply quality criteria, define operationalization details, and export a complete indicator reference sheet.

Open MEL Design Lab

4

Data Architecture & Collection Protocols

Design the infrastructure for data to flow from collection points to decision-makers. This includes tools, systems, roles, and quality assurance protocols.

Data Flow Architecture

Your MEL data system should move through four layers:

1. Collection Layer: Field surveys (KoBoToolbox), administrative records, participant feedback, partner reports
2. Storage & Processing Layer: Central database, data cleaning protocols, quality assurance checks
3. Analysis & Visualization Layer: Dashboards, standard analysis templates, report generation
4. Decision Layer: Pause-and-reflect sessions, management reviews, stakeholder reporting

Try it in the Lab

Design your data management system with the MEL Plan Lab. Define data flows, storage requirements, quality protocols, and responsible parties—with exportable templates.

Open MEL Plan Lab

6

Costing & Sustainability Planning

MEL systems require resources to function. Build sustainability into the design from the start.

The 5-10% Rule: MEL should typically represent 5-10% of total program budget. Below 5%, systems tend to be under-resourced and produce poor quality data. Above 10%, you may be over-investing relative to program scale.

MEL Budget Components

• Personnel (50-60%): MEL staff salaries, data collectors, enumerator fees
• Data collection (20-25%): Survey costs, travel, participant incentives
• Technology (10-15%): Software licenses, devices, server costs
• External support (5-15%): Evaluations, technical assistance
• Learning activities (5-10%): Workshops, dissemination, knowledge products

Varna

Building a MEL system is one thing—making it actually work in your organizational context is another. The technical design is often the easy part; navigating the politics, building buy-in, and creating a genuine learning culture requires different skills. Let's discuss your specific context.

Book 1:1 Coaching

Papers & Resources

Key Academic Resources

MEL Lexicon

Sample Terms