Your evaluation plan should address questions related to both process (i.e., program operations, implementation, and service delivery) and outcomes (the ultimate impact of your intervention).
An outcome evaluation looks at the intervention’s effect on the environmental conditions, events, or behaviors it aimed to change (whether to increase, decrease, or sustain). Usually, an intervention seeks to influence one or more particular behaviors or conditions (such as risk or protective factors), assuming that this will then lead to a longer-term change, such as a decrease in the use of a particular drug among youth.
You may have followed your plan completely and still had no impact on the conditions you were targeting, or you may have ended up making multiple changes and still reached your desired outcomes. The process evaluation will tell how closely your plan was followed, and the outcome evaluation will show whether your strategy made the changes or results you had intended.
At a minimum, your community should strive to put measures in place that allow you to track the problem of interest over time.
- Example: Comparing the percentage of high school students in the community who report past 30-day misuse of pain relievers prior to the implementation of any strategies (baseline) and again at the end of the project (and ideally at multiple points during the project) will help you identify whether the issue is getting better, getting worse, or remaining the same over time.
Other strategies to enhance the quality of your evaluation:
- Measure changes in the intervening variables over time—this will help demonstrate if any changes in the long-term outcomes are related to the intervening variables targeted by your strategies
- Measure changes in the short-term outcomes that are the expected antecedents of changes in your intervening variables—this will help you determine whether your strategies are having their desired effect
- Examine whether there is a dose-response relationship between your short-, intermediate-, and long-term measures and variations in the amount (dose) of prevention services received by different individuals
- Compare differences in short-, intermediate-, and long-term measures between individuals who were exposed to the intervention and those who were not