Guidance

STEP 5: Evaluation

Evaluation is the systematic collection and analysis of information about intervention activities, characteristics, and outcomes. Evaluation activities help groups describe what they plan to do, monitor what they are doing, and identify needed improvements.

The results of an evaluation can be used to assist in sustainability planning, including determining what efforts are going well and should be sustained, and showing sponsors that resources are being used wisely.

Five Functions of Evaluation

Program evaluations are often conducted in response to a grant or other funding requirement. As a result, reporting may be structured only to address the requirement rather than to provide a functional flow of information among partners and supporters.

A comprehensive and well-rounded evaluation process gathers information to accomplish five key functions:58

  • Improvement. This is the most important function of an evaluation—improving the efficiency and effectiveness of your chosen strategies and how they are implemented.
  • Coordination. The evaluation process assesses the functioning of your group, allowing partners to know what the others are doing, how this work fits with their own actions and goals, and what opportunities exist for working together in the future.
  • Accountability. Are the identified outcomes being reached? A good evaluation allows your group to describe its contribution to important population-level change.
  • Celebration. This function is all too often ignored. The path to reducing drug use at the community level is not easy, so a stated aim of any evaluation process should be to collect information that allows your group to celebrate its accomplishments.
  • Sustainability. A thorough evaluation can help you provide important information to the community and to various funders, which promotes the sustainability of both your group and its strategies.

To accomplish these five functions, you need to provide information to the appropriate stakeholders so that they make better choices (improvement), work more closely with your partners (coordination), demonstrate that commitments have been met (accountability), honor your team’s work (celebration), and show community leaders why they should remain invested in the coalition process (sustainability).

Engaging Stakeholders

Evaluation cannot be done in isolation. Almost everything done in community health and development work involves partnerships—alliances among different organizations, board members, those affected by the problem, and others who each bring unique perspectives. When stakeholders are not appropriately involved, evaluation findings are likely to be ignored, criticized, or resisted. People who are included in the process are more likely to feel a good deal of ownership for the evaluation plan and results. They will probably want to develop it, defend it, and make sure that the evaluation really works.

Therefore, any serious effort to evaluate a program must consider the viewpoints of the partners who will be involved in planning and delivering activities, your target audience(s), and the primary users of the evaluation data.

Engaging stakeholders who represent and reflect the populations you hope to reach greatly increases the chance that evaluation efforts will be successful. Stakeholder involvement helps to ensure that the evaluation design, including the methods and instruments used, is consistent with the cultural norms of the people you serve. Stakeholders can also influence how or even whether evaluation results are used.

All partners in your substance misuse and abuse prevention or reduction efforts should be involved in developing and implementing your evaluation plan. To facilitate this process, you may consider forming a committee focused on evaluation. The committee would work in collaboration with an evaluator to collect the data, analyze results, and share findings with partners, the community, the media, and others. Having more people trained in data collection and analysis and able to spread the word about the group’s successes contributes to sustainability.

A strong evaluation system can provide monthly data about activities and accomplishments that can be used for planning and better coordination among partners. In addition, sharing evaluation data can give the group a needed boost during the long process of facilitating changes in community programs, policies, or practices.

Cultural Competence in Evaluation

Culture can influence many elements of the evaluation process, including data collection, implementation of the evaluation plan, and interpretation of results. Tools used to collect data (such as surveys and interviews) need to be sensitive to differences in culture—in terms of both the language used and the concepts being measured.

When selecting evaluation methods and designing evaluation instruments, you should keep in mind the cultural contexts of the communities in which the intervention will be conducted. Here are some guiding questions to consider:

  • Are data collection methods relevant and culturally sensitive to the population being evaluated?
  • Have you considered how different methods may or may not work in various cultures?
  • Have you explored how different groups prefer to share information (for example, orally, in writing, one on one, in groups, through the arts)?
  • Do the instruments consider potential language barriers that may inhibit some people from understanding the evaluation questions?
  • Do the instruments consider the cultural context of the respondents?

Evaluation and Sustainability

Evaluation plays a central role in sustaining your group’s work. It enables you to take key pieces of data and analyze and organize them so that you have accurate, usable information. This process facilitates the development of the best plan possible for the community and allows your group to accurately share its story and results with key stakeholders. It can also help you track and understand community trends that may have an impact on your group’s ability to sustain its work.

A good evaluation monitors progress and provides regular feedback so that your strategic plan can be adjusted and improved. Your group may implement a variety of activities aimed at changing community systems and environments. By tracking information related to these activities and their effectiveness, as well as stakeholder feedback, community changes, and substance misuse and abuse outcomes, you can build a regular feedback loop for monitoring your progress and results. With this information, you can quickly see which strategies and activities have a greater impact than others, determine areas of overlap, and find ways to improve your group’s functioning. Using information from the evaluation, your group can adjust its strategic plan and continually improve its ability not only to sustain its work, but also to achieve community-wide reductions in substance misuse and abuse and its consequences.