Evaluation: Why it's better than the alternative
The thought of evaluation tends to strike fear in the hearts of the most seasoned and confident professionals. The process itself is often met with such reluctance that I often felt that some of my clients would rather be doused with gasoline and set aflame than meet with me to discuss how to monitor their work or discuss the results of their evaluation (and I don’t think it had anything to do with me personally…at least I hope not!) I had a supervisor who dealt with this anxiety and uncertainty in a very direct way; she would do trainings on outcome evaluation to large groups wearing a grim reaper costume complete with scythe. Suddenly, evaluation was no longer the greatest thing to fear! This humorous approach helped to ease uncertainties and suspicions and allowed all those present to begin the work from a place of common understanding. We know how you might see evaluation and we understand why, but it could be worse.
Perhaps it’s the fault of the term itself which Merriam-Webster defines with the following statement: “to determine the significance, worth, or condition of usually by careful appraised and study” (m-w.com). Our (second) greatest fear is that our hard work is not paying off in the ways we had hoped. Often the prevailing notion is that any whiff of failure will have grave consequences – reprimands, funding cuts, fewer clients… (add your favorite worst evaluation-related nightmare here). This is acutely true for evaluations that occur after a program has ended. This autopsy versus preventive care approach only heightens feelings of dread and hopelessness. If we could just come up with a better term to describe what can be a very useful and (relatively) painless exercise.
Also, the whole practice needs to be reframed as a collaborative process to facilitate and inform the development of, implementation and management of and possibly, improvements to a particular approach, strategy, model, or set of activities and services. To do so effectively, the evaluation should be part and parcel of the program design and budget. (Yes, ten percent of the total project budget is a decent rule of thumb.) From the beginning, all involved should agree on the answers to the following questions:
What do we hope to accomplish?
How should we define success?
What amount of change will be considered good enough?
Can we really accomplish what we have proposed?
Can we do this with the resources (defined, broadly, personnel, space, money, collaborators) to which we currently have access?
Is it realistic to expect that the program strategies and activities will result in the desired outcomes?
Do we have the necessary information and if not, can we get access to it, to allow us to track the change we seek in a timely and useful fashion?
How will we know and what’s the timeframe for determining whether we would continue with the approach, model before we program ends?
There are many different types of evaluation that will help you address specific questions. Are the services, activities, and strategies you outlined really being implemented and delivered as you had hoped? A Process Evaluation (also known as formative evaluation) could help you decide. Do you want to have a better sense of what you need to do? Then a Need Assessment is the right way to go. Want to know what changes in knowledge, attitudes, beliefs and behaviors have resulted or been influenced by your work? An Outcome Evaluation (or summative evaluation) is for you. Finally, if you are wondering the price tag for offering a successful program, one that results in the desired outcomes, or what’s the return on your investment, consider a Cost Benefit Analysis.
Overall, keep in mind that the anticipation of the evaluation is much more frightening than the act itself. It’s like going to the dentist. We realize we should do it, but we often worry about what the doctor will find. (Not another cavity?!) However, wouldn’t you want to gauge your general health over time, take the appropriate medicine or change your diet as needed, or start that workout routine before it is too late (flashback to grim reaper image here)? An effective evaluation will help you do the following: implement your program or strategy in a way that is informed by the overall outcomes, monitor your work over time so that you can make timely and important changes to address unanticipated challenges and limited progress, highlight the return (or lack there of) on your investment of time, money and other resources for the participants you assist and the funders that support you, and make the change you seek. Now wouldn’t that feel good?