Evaluation often plays second fiddle to project implementation yet good project management should start with evaluation in mind because without knowing whether you have made a positive difference in moving from A to B you cannot possible know what step C should be.
Evaluating organisational wide change projects is probably one of the more difficult evaluation undertakings as there are so many moving parts, so many differing expectations and competing objectives. Nevertheless, the same pitfalls and good practices that are used in traditional project evaluation still apply.
- Not including evaluation planning as part of project planning
- An evaluator’s nightmare is when you get the call to undertake an evaluation and you find out that project implementation started two months ago. Any opportunity for baseline data (except for administrative data) is seriously comprised as is any ability to assist develop measurable project objectives.
- Project managers can set themselves up for failure by committing to objectives that are out of scope or unachievable. An evaluation is designed to hang off the project objectives so these must be developed carefully.
- Evaluation is not proportional to the project
- Spending disproportional time, money and energy on measuring a relatively low-budget project OR allocating a small effort towards a large project is surprisingly common.
- Reporting interesting yet irrelevant findings to pad out a report
- See below …. Less is more.
- Assuming causality where none exists.
- Maybe it is not the evaluators that are assuming causality because they should after all, know better than that, but rather it is poor articulation of findings that allows politicians or senior executive to make causative assumptions.
- Be realistic on how long it will take to embed new practices, change behaviour and deliver cultural shifts.
- Bosses want quick answers and politicians driven by the short election cycle will want even quicker ones. As the evaluator, you must push back on creating measures or agreeing to delivering findings on things that you know cannot be accurately measured in the timeframe provided. Ultimately you may not be the decision maker but make sure your concerns are on the record.
- Focus on what matters / Less is more
- It can be tempting to measure and report on everything. Triangulation of data is important to provide extra validity to your findings but measuring everything just makes the report confusing and wastes a lot of your time. In fact, rather that providing extra credibility to your findings it will work to obscure the really important findings.
- Set up data collection mechanisms prior to commencement of the project
- In reality, this rarely occurs. Ideally evaluators should be sitting at the project table at the instigation of the project so that data collection requirements are considered as the project is being built. This will only happen if evaluation experts build relationships with leaders and demonstrate their role as adding value and not just as adding unnecessary work to a project.
- Identify responsible data collectors
- Someone needs to be responsible for each type of data requirement. If no one is the go-to person for data then issues will slip through the cracks, quality of the data will suffer and by the time anyone realises that information has not been collected or inadequately collected it will be too late.
- Formative vs summative assessment
- Consider opportunities to contribute meaningfully to project implementation through formative assessment processes that provide time-sensitive feedback to the project lead and senior executive. One of the biggest complaints about evaluation is that it takes too long to provide an answer.
- Build processes that allow you to report on activities, behaviours or attitudes that might be de-railing an organisational change project. In this way not only do you give the best chance of successful change management but you demonstrate the value of evaluation to the organisation.
FACILITATING THE PEOPLE SIDE OF ORGANISATIONAL CHANGE
Kim Adams is an experienced organisational performance improvement and program evaluation professional with a distinctive ability to manage change, negotiate partnerships, and communicate sensitively with diverse groups and levels.
She is passionate about connecting leaders with their unique voice so they lead with confidence, influence and success. She combines 15 years in police program evaluation with her qualifications in psychology, management and yoga to provide a unique and pragmatic approach to change management and leadership development.
Kim is the principal of Kim Adams Consulting specialising in Organisational Change, Developing Leaders, Coaching and Program Evaluation.