1) Understand the Importance of Program Evaluation
How administrators THINK a program works may be
entirely different than HOW it works in practice.
If new programs are designed appropriately and incorporate the right feedback channels and data collection processes, evaluations becomes significantly easier. By gathering rich information across a variety of indicators, analysts will be able to identify and describe trends in the data and better understand how (or even if) the program is actually improving the lives of vulnerable populations on the ground. This may be more difficult for existing programs as the level of detail and data may not have been collected over time. However, the time and resources needed to build the groundwork for effective evaluations will prove fruitful for several reasons:
- Stewardship: Evaluations help nonprofits identify areas in which ROIs are lower than expected as well as areas in which significant client growth could be realized with additional investment.
- Development: Even if evaluations uncover areas of improvement within a particular project or initiative, funders are likely to appreciate and honor your team’s willingness to foster a culture of accountability and implement reforms.
- Diversity and Inclusion: Over the course of an exhaustive and comprehensive evaluation, several voices should have the opportunity to speak about the program and voice their praises/concerns to top decision-makers. Staff members, beneficiaries, volunteers, donors, and regulators typically view the program through their unique worldviews and lenses – gathering all of these perspectives helps you capture the fullest picture possible.
- Public Support: By demonstrating humility and bringing a wide range of individuals and community groups into the conversation, it is likely that public opinion of your organization will improve and a greater number of organizations will partner with you in the future. Because no nonprofit is capable of delivering every service needed to marginalized populations, such collaboration is critical to delivering comprehensive care.
- Impact Multiplication: One of the most prominent reasons nonprofits conduct evaluations is to determine what works, what doesn’t, and what can be done on a larger scale to reach more people. If the proper metric data is collected, statistical models have the ability to determine (to an extent) the expected efficacy of the program in other contexts and among other people groups.
The Handbook of Practical Program Evaluation by Wholey, Harry, and Newcomer is a free textbook that discusses both the various types of evaluations available to nonprofit leaders as well as the ways in which these evaluations can be performed. Topics include:
- Determining information needs
- Engaging stakeholders
- Stress-testing logic models
- Qualitative and quantitative evaluations
- Building surveys and retaining participants over time
- Controlling the costs of evaluations
2) Explore Various Evaluation Methods
The nature of your nonprofit and the people it serves will fundamentally shape the way that you are able to collect data and assess the success (or failure) of your organization to reach its goals. Entities with less tangible goals (e.g., greater appreciation for the arts, spiritual growth, etc.) may be more difficult to quantify and will require greater reliance on qualitative methods such as surveys, interviews, and focus groups. Nonprofits that deliver services that produce measurable change (e.g. dental and medical clinics, food pantries, etc.) may get better results by using both qualitative and quantitative evaluations.
- Typically takes the form of numbers
- Sources include administrative records and structured surveys
- Typically takes the form of words or visual images
- Sources include interviews and open-ended responses
Most of the time, evaluations are not able to undeniably prove that one particular factor (or a set of particular factors) is directly responsible for changes in the population. However, many analysts are able to show with high probabilities which characteristics and traits of the program are likely having the most profound effects. The resources below will help you think through the survey design process as well as the best techniques for conducting focus groups.