The core difference between impact and implementation lies in what they measure: impact focuses on the results or effects of an intervention, while implementation focuses on how the intervention is delivered.
Based on research, impact evaluations measure the attributable effects of an intervention, determining whether it achieved its intended outcomes. In contrast, implementation research examines how the intervention is being implemented in different contexts, exploring the processes, challenges, and facilitators involved in its delivery.
Understanding this distinction is crucial for evaluating programs and planning for their success.
Understanding Impact
Impact evaluation asks: Did the intervention work?
- Focus: Measures the changes or outcomes directly caused by a specific program, policy, or intervention. It aims to isolate the effect of the intervention from other factors.
- Goal: To determine causality – whether the intervention led to the observed results.
- Questions asked: Did the program reduce poverty? Did the new teaching method improve test scores? Did the health campaign decrease disease incidence?
- Key metric: The attributable effect of the intervention.
Understanding Implementation
Implementation research and evaluation ask: How was the intervention delivered?
- Focus: Examines the process of putting an intervention into practice. It looks at fidelity (was it delivered as intended?), dose (how much was delivered?), reach (who received it?), and the context in which it was implemented.
- Goal: To understand how and why an intervention worked or didn't work in a specific setting.
- Questions asked: Was the training delivered according to the manual? Were there barriers to people accessing the service? How did local conditions affect delivery? What adaptations were made?
- Key insight: Provides context for understanding impact results and identifies factors that help or hinder success.
Key Differences at a Glance
Here’s a quick comparison:
| Feature | Impact Evaluation | Implementation Research |
|---|---|---|
| Primary Question | Did it work (what were the effects)? | How was it delivered (the process)? |
| Measures | Attributable effects / Outcomes | Delivery process, fidelity, context |
| Goal | Determine causality / Effectiveness | Understand how and why it worked/didn't |
| Focus | Results | Process |
The Relationship Between Impact and Implementation
While distinct, impact and implementation are closely related and often studied together. As the reference highlights:
- Implementation research informs analysis of program effects (impact): Understanding how an intervention was implemented helps explain why it had the impact it did (or didn't). For example, a program might show little impact not because the intervention design was flawed, but because it wasn't implemented correctly or reached enough people.
- Implementation research is needed to inform scale up of the intervention: If an intervention is successful (positive impact), understanding the nuances of its implementation in different contexts is essential for replicating that success elsewhere and scaling it up effectively.
Example:
Imagine a project introducing new water filters in rural villages (the intervention).
- Impact question: Did introducing the filters reduce the incidence of waterborne diseases (the outcome/effect)?
- Implementation questions: Were the filters distributed to all households? Were people trained on how to use and maintain them correctly (fidelity/dose)? Were there local customs or logistical issues that affected filter usage (context)?
Studying implementation helps explain the impact results. If disease incidence didn't drop, implementation research might reveal that many filters weren't used because people didn't understand the instructions or couldn't get replacement parts.
In summary, impact tells you if something happened as a result of an intervention, while implementation tells you how the intervention was put into practice, providing vital context for the results and guidance for future efforts.
