MEASURE CHANGE AND IMPACT
WHAT CHANGED, WHY DID IT CHANGE AND DID THE PROGRAM MAKE A DIFFERENCE
Understanding whether a program is working requires more than tracking activities. ClerkMaster’s Measure Change and Impact research helps organisations establish what has changed, why it changed, whether intended results were achieved, and whether those results can be sustained. We use structured evaluation approaches to generate evidence around programme implementation, outcomes, impact and contribution.
MEASURING CHANGE
Knowing what happened is only the beginning.
A program can deliver activities without achieving its intended outcomes. Change may occur without being caused entirely by the program. And positive outcomes may not necessarily be sustained over time. Measurement and evaluation therefore need to examine more than implementation alone. They need to establish what changed, the direction and magnitude of change where appropriate, the factors associated with those changes, and the program’s contribution to observed outcomes.
This page covers the research approaches ClerkMaster uses to assess implementation, outcomes, impact, contribution and sustainability.
WHAT WE CAN HELP YOU MEASURE
Different stages of a program require different kinds of evidence.
PROGRAM IMPLEMENTATION
Is the program being implemented as intended?
Monitoring & Evaluation (M&E)
Measure programme implementation and performance against:
- Objectives
- Indicators
- Targets
- Intended results
BASELINE CONDITIONS
Where are things before the intervention?
Baseline Studies
Establish pre-intervention conditions and benchmark indicators against which future change can be measured.
CHANGE OVER TIME
What changed between before and after the intervention?
Pre / Post Studies
Compare indicators before and after an intervention to establish the:
- Direction of change
- Magnitude of change
PROGRAM PROGRESS
How is the program performing while it is being delivered?
Mid-Term Evaluations
Assess:
- Implementation progress
- Emerging outcomes
- Challenges
- Corrective actions required during
- delivery
OVERALL RESULTS
What has the program achieved?
End-Term Evaluations
Assess:
Overall achievements
Outcomes
Impact
Sustainability
Lessons learned
Future programme implications
IMPACT AND OUTCOMES
What changed for beneficiaries, communities, institutions or markets?
Impact & Outcome Assessments
Determine what changed and examine the programme’s contribution to those changes.
THE PROGRAM JOURNEY
Measure change across the programme journey.
01
BASELINE
Establish starting conditions.
02
IMPLEMENTATION
Monitor delivery and performance.
03
MID-TERM
Assess progress and emerging outcomes.
04
END-TERM
Assess achievements, outcomes and impact.
05
SUSTAINABILITY
Understand future implications and whether results can be sustained.
UNDERSTANDING CHANGE
A change in an indicator does not automatically explain why it changed.
Program evaluation can require multiple forms of evidence to understand not only whether an outcome changed, but also the factors associated with that change and the plausible contribution of the intervention.
WHAT CHANGED?
Identify changes in relevant indicators or outcomes.
HOW MUCH DID IT CHANGE?
Where appropriate, establish the direction and magnitude of change.
WHY DID IT CHANGE?
Examine the factors and pathways associated with observed outcomes.
WHAT WAS THE PROGRAMS CONTRIBUTION
Assess the programme's plausible contribution, particularly where direct attribution is difficult.
EVALUATING PROGRAM EFFECTS
When the question is about program effects, the design matters.
Where feasible and appropriate, evaluation designs can be used to examine programme effects using intervention and comparison groups.
Three approaches;
EXPERIMENTAL / QUASI-EXPERIMENTAL DESIGNS
Use intervention and comparison groups to estimate program effects where feasible.
CONTRIBUTION ANALYSIS
Determine the plausible contribution of an intervention to observed outcomes where direct attribution is difficult.
THEORY OF CHANGE ASSESSMENT
Test whether expected pathways from activities through outputs and outcomes to impact occurred as intended.
LEARNING FROM UNEXPECTED OUTCOMES
Not every important outcome is planned in advance.
OUTCOME HARVESTING
Start with significant outcomes and work backwards.
Identify significant outcomes and work backwards to establish programme contribution.
MOST SIGNIFICANT CHANGE
Learn from the changes people consider important.
Use systematically collected beneficiary and stakeholder stories to understand important programme changes.
DATA TRIANGULATION
One source rarely tells the whole story.
Data Triangulation cross-validates findings across:
- Methods
- Stakeholder groups
- Data sources
The aim is to examine evidence from different perspectives and strengthen understanding of the issue being evaluated.
WHAT THIS RESEARCH CAN REVEAL
What do you need to know about your programme?
IMPLEMENTATION
Is the program being implemented as intended?
PROGRESS
Are we making progress toward the intended results?
CHANGE
What has changed since the intervention began?
OUTCOMES
What outcomes have emerged?
IMPACT
What changed for beneficiaries, communities, institutions or markets?
CONTRIBUTION
What has the program plausibly contributed to those changes?
SUSTAINABILITY
Are results likely to be sustained?
LEARNING
What lessons should inform future programme decisions?
MEASURE WHAT CHANGED
Understand why. Learn what to do next.
ClerkMaster helps organisations build evidence around program implementation, outcomes, impact, contribution and sustainability.
What do you need to know about the change your program is creating?
Let’s determine the evidence and evaluation approach needed to understand what changed, why it changed and what the findings mean for future decisions.