Did I make a difference? Measuring our contribution to change

It is my last day at WaterAid after seven years of working for this incredible organisation. When you leave a job, you go through all sorts of emotions, but as a campaigner the one question I will be asking myself as I walk out of the door will be 'What impact did I make on the world while I was here?'

I could list a dozen things that the water sector, WaterAid, my team and I have achieved (or, more likely, contributed to achieving – interesting view on this here) in the past seven years. The highlight would be the big increase in international awareness and attention on the importance of water and sanitation, signified by the agreement of a goal on water and sanitation in the Sustainable Development Goals (SDGs).

I can say with some certainty what our contribution to this achievement was because of two evaluations we did of this work with our partner organisations in the water sector. But this was possible because although our SDG advocacy work was complex, the change we wanted to effect was a relatively 'simple' change in policy (rather than, for example, implementation, norms or behaviour). Also, the work was managed centrally, by colleagues interested in and able to be reasonably systematic about our planning, monitoring, evaluation and reporting (PMER).

PMER in a complex environment

The most effective advocacy work we do at WaterAid is political and systems-based, complex, decentralised, unpredictable, led by thousands of partners and staff across multiple countries. How do you go about systematically PMER-ing in this environment – and in a way that encourages this best type of advocacy, rather than stifling it?

Image: Sidney Harris

This month, WaterAid will begin implementing a new approach to PMER of our advocacy work. I blogged about an earlier stage of this just a few months ago. We have agreed a set of basic commitments to doing PMER of our advocacy work (what we are calling a ‘common approach’) – which are simple on the surface but surprisingly difficult to deliver in practice.

For example, one of the areas we have recognised that we could be better at is regularly taking time to reflect and learn from our advocacy work. By this I don’t mean a quick coffee between meetings to work out key messages for the next lobby meeting. I mean a systematic, consciously facilitated, regular and short meeting to reflect on and learn from our work of the previous months.

Campaigners – myself included – often love to talk but loathe to spend time stuck in the office writing about advocacy. We frequently find it hard to record our reflections so we can later remember and evaluate what we have done, and build on previous work. We are encouraging WaterAid’s staff to do this during or after every ‘reflect and review’ meeting. This information could be tagged and collated over time and used as ‘data’ for future evaluations, or aggregated and collected thematically at regional and global levels, or for specific donor reports.

Taking 'reflect and review' to a global level

Now comes the hard work – implementing this across the organisation, and learning and adapting it as we go. There’s loads of information out there about 'adaptive management' and this approach to PMER of advocacy – some of it more accessible than others. But what I’ve found to be often missing is practical examples of how to actually implement it at any organisation, let alone one as complex as WaterAid. I’ve seen the start of this in the documenting of USAID’s approach. But what do you need to do to implement the approach in your own organisation or team?

Two of the many questions I’ve been thinking about are:

• How do you ask open questions when facilitating reflective sessions to draw out the best ideas? In a recent session I genuinely felt scared as I started the meeting, not knowing how people would respond. I had to fight against all my instincts to 'facipulate' – to control, summarise and direct. Genuinely good facilitation is an incredible and uncommon skill, but it’s absolutely central to supporting teams to learn and adapt.
• How, in a hierarchical culture and organisation, do you change culture enough that people can reflect on what’s gone badly in front of the 'boss'? Recent research from Google on this shows that successful teams are able to create a space that allows them to admit to vulnerability and take risks. As a leader of an organisation or a manager, what can you do to create that space? Leading by example must be a start.

And the list goes on – how do we work with traditional donors using this approach? Will we truly be able to 'aggregate' our impact globally? How can we get people to write stuff down? How can we unblock challenges and get some quick wins going at the start? How does culture change actually work ('Leading change' by John Kotter is great on that).

We are promoting four 'tools' to support staff with all of this. Do try them out and let us know in the comments below what you think!

Kate Norgrove tweets as @katenorgrove