Measuring the immeasurable: monitoring and evaluation in advocacy

4 min read

To continue to grow our impact, it is important that we measure what we do. We can add up toilets, and record how many people have gained safe water, but how do we count the effect of the advocacy that forms half of our work? Kate Norgrove, WaterAid’s Global Head of Campaigns, finds some clues in Malawi.

Gladys sits in a circle of women in a school hall with her small son, Blessings. "I bring him here so he learns to appreciate what women can do,” she says.

I’m here in Malawi, ‘The Warm Heart of Africa’, this week, looking with colleagues at how we plan, monitor and evaluate (PME) our advocacy work. Gladys is part of a savings group that I’ve come to see which gives loans to women who want to build a toilet. The same group of women has also lobbied to get affordable water to the community, started multiple (and profitable), small businesses, and paid for children’s uniforms and books. No wonder Gladys brings her son to be part of it.

Kate Norgrove, WaterAid's Global Head of Campaigns, (left) with Gladys and Blessings at savings group in Malawi.

WaterAid/ Dennis Lupenga

Kate Norgrove, WaterAid's Global Head of Campaigns, (left) with Gladys (second right) and Blessings (front) at a savings group in Malawi.

WaterAid’s partner, Training Support for Partners, has simply supported the community to organise themselves, but now this community feels one generation away from transformation. As my colleague Chilufya said: "Grow up there, get a good education and you can do anything.”

How can we measure the impact of advocacy?

Much of the impact of WaterAid’s work feels immeasurable. But is it?

I’ve been working on this project to improve our PME processes for advocacy for a while now. Conversations have seemed to rotate around the same familiar refrains. How can we attribute advocacy success to WaterAid? How can a world of complex change be translated in to a linear log-frame?

The trouble is that half of our work quite literally can’t be counted. How would you ‘count’ the strength of WESNET – the network of civil society organisations I met this morning, that has become a brilliant national lobbying outfit currently working without WaterAid funds?

The tensions between our fundraising colleagues, PME specialists, and our advocates are palpable at times like these. The creative campaigner doesn’t want to be tied down by process; the lobbyist wants to be pounding the streets, influencing. The PME specialist wants some evidence; the fundraiser feels pressure from donors to know precisely what impact we will have, years in advance.

So, this week we have been trying to find some common ground, and we think we may have the beginnings of it.

Our colleagues in Cambodia have, for a while now, been trialling ‘adaptive management’ as a process – a buzz phrase which many others have written about (hereherehere and, brilliantly, a request to move from SMART to STUPID objectives here) – which is enabling them to chart impact and change in complex territory.

At the heart of the approach is a process of frequent reflection and adaptation, which is, importantly, documented and used as evidence for monitoring and evaluation. Projects might be run on six week or three month cycles of activity, with clear plans that are reflected on and adapted at the end of the agreed period.

This instinctive process, common to many advocacy projects already, is then documented in a way that makes the most sense for that project. Over time, this documentation builds up and can be tagged and used for both learning and accountability.

We plan to combine this approach with smart political analysis and some more traditional tools, to try to build up a picture of our advocacy successes – and failures.

Key changes

All of this will take some change, though. WaterAid is an organisation founded by engineers, and numbers are a core part of our business. In order to plan, monitor and evaluate our advocacy work more effectively in this context, we think we are going to have to make some key organisational shifts:

  • Towards ‘good enough’ (away from perfectionism)  – doing good enough PME to effectively inform next steps.
  • Towards our processes being more adaptable to frequent change.
  • Towards doing more monitoring, linked to more planning throughout, and good enough (much less) planning up-front.
  • Towards more documenting of activity and progress at the time it happens.
  • Towards reflection – using reflection for learning, valuing reflective skills, and using reflection as data for monitoring.
  • Towards analysis – valuing political and analytical skills, and recruiting for them.
  • Towards learning – putting the emphasis on PMER for learning as much as accountability.

We won’t ever be able to ‘count’ the influence that Gladys has had on her son’s upbringing. But we could get better at measuring the impact of what we do, and have some evidence to show next time a donor asks us what we’ve done to strengthen that national network.

Kate Norgrove tweets as @katenorgrove

For more information see: