Beyond Inclusion: Realising gender transformational change and sustainable WASH systems

Terms of Reference

About the role

We are seeking an individual consultant or small, multi-disciplinary team to complete a short-term contract leading a final project evaluation of our Australian Government-funded Water for Women project in Timor-Leste.  As well as undertaking the evaluation, the consultant(s) will also prepare 5 succinct stories of change to highlight project achievements and update a WASH system strengthening building block assessment.

Background

WaterAid’s vision is a world where everyone, everywhere has safe water, sanitation and hygiene.  Extreme poverty cannot be eradicated without universal access to safe water, sanitation and hygiene (WASH). These basic services are human rights: everyone on our planet needs them to live healthy, dignified and productive lives. 

Water for Women

Water for Women is the Australian Government’s flagship WASH program and is being delivered as part of Australia's aid program, investing $110.6 Million over five years from 2018 to 2022.

Water for Women is partnering with 10 Civil Society Organisations (CSOs) to deliver 18 Projects in 15 Countries in South Asia, Southeast Asia and the Pacific. Over the course of the Fund, Water for Women aims to support an estimated 2.95 million people including the most marginalised within these communities.

WaterAid has four projects funded through Water for Women, in Myanmar, Papua New Guinea and Timor-Leste. The focus of this evaluation will be our project in Timor-

Leste.

Beyond Inclusion, WaterAid Timor-Leste’s Water for Women project

Beyond Inclusion is a five-year, AUD7.6 million project, which aims to improve WASH services in Liquica and Manufahi Municipalities of Timor-Leste and influence social norms for greater gender equality.

The project builds on and extends much of our previous work in Timor-Leste. It continues our work with national and municipal governments to strengthen the WASH sector and improve coordination across related sectors (for example nutrition), all while ensuring gender and social inclusion approaches are mainstreamed.

At the local level, the project continues to support communities with gaining access to, and maintenance of, inclusive WASH services, including to continue to use our Community Dialogue toolkit to create more equal power dynamics between men and women. It also continues to employ our Community Scorecard (CSC) process, enabling people to provide feedback to government on the quality of services they’re receiving. In 2021, to respond to COVID-19, additional activities were also integrated specifically around improving WASH and hygiene awareness in schools in Liquica and Manufahi.

Across all these activities, the project is also focused on documenting and sharing lessons and emerging models of good practice within Timor-Leste, so that others can adopt them.

The four project objectives are:

1. National WASH sector systems are strengthened to effectively integrate gender equality and social inclusion.
2. Women and men share roles and responsibilities in decision making in the household and at the community level, with a particular emphasis on WASH.
3. Strengthened WASH sector systems in supported Municipalities enable significant progress towards achieving and sustaining hygienic status.
4. National WASH sector knowledge management and learning systems strengthened, including exchange between relevant sectors.

Purpose of the evaluation

The purpose of the evaluation is to assess and document what the project has achieved and identify what lessons can be drawn from the project to inform WaterAid’s future WASH programming. Within this broad purpose, we are particularly interested in:

• understanding the aspects of our system strengthening in WASH and Gender Equity, Disability and Social Inclusion (GEDSI) approaches that contributed to achieving the programming objectives and how these could be further refined, and
• understanding how the outcomes can be built on to support enhanced climate resilience.

The findings of the evaluation will be used by WaterAid in our final project reporting for Water for Women, and as an important reference for future project design and delivery in Timor-Leste.

Key evaluation questions

The following key questions will be reviewed and finalised with the evaluation team as part of the inception process: 

1. How well has the project responded to the needs and priorities of local actors in relation to improving access to WASH?
2. To what extent did the project design enable progress towards program objectives?
   1. Was the project doing the right things to affect change?
   2. How well has the project adapted to the changing context over time (including COVID-19) and/or responded to learning and evidence?
3. To what extent have the project objectives been achieved?
   1. What factors have supported or hindered the achievement of objectives?
   2. What other changes (intended or unintended) has the project contributed to? 
4. To what extent was the project appropriately resourced (including human and financial resources) to meet the objectives?
   1. What were the strengths of key project partnerships and what could have been improved?
5. How well were GEDSI considerations integrated across project implementation? Were these effective?
6. What evidence is there that the results achieved will be sustained over time?
7. What key lessons can be drawn from this project to inform WaterAid’s future WASH programming in Timor-Leste? Including in the areas of:
   1. System strengthening
   2. GEDSI
   3. Climate resilience
   4. Locally-led programming and influencing

Evaluation annexes

In addition to preparing the main evaluation report addressing the evaluation questions, the evaluation consultant(s) will also prepare two annexes, as outlined below.

Stories of change

The evaluation consultant will prepare five stories of change, highlighting different aspects of the project and its achievements. These stories will form an annex to the evaluation. These stories should be 2-3 pages in length drawing on evidence collected during the evaluation. The topics should be negotiated and agreed with WaterAid during the evaluation, but will include at least one story related to GEDSI and one related to system strengthening.

Update to system strengthening building block assessment

A participatory system strengthening building block assessment (based on WaterAid’s building block assessment rubric) has been completed each year during implementation at a district level and was last undertaken in November 2021. The consultant should use the data gathered and analysed during the evaluation to update the building block assessment. Findings from this assessment should be included as an annex to the evaluation report.

Methodology

The evaluation consultant(s) is encouraged to propose an appropriate methodology to respond to the evaluation questions and prepare the annexes. This will be more fully developed in a detailed evaluation plan prepared during the inception phase, which will set out how data will be collected and analysed for each evaluation question.

Context permitting, it is anticipated that the evaluation will involve a desk review of existing monitoring data and reporting, and primary data collection in Timor-Leste. The consultant should propose contingencies if travel is not considered possible due to COVID-19 or other reasons.

Existing data sources

A range of existing data sources will be available for the evaluation team to analyse as part of the evaluation. This should be taken into consideration when proposing the methodology. These include (but are not limited to):

• Project design document
• Baseline data, including:
  • WFW TL Baseline report 2018
  • Hygienic suco baseline report 2021
• Mid-term review
• Annual reports
• Disability inclusion review 2019
• Gender module implementation review
• Association of GMFs review (estimated completion by Sep 2022)
• District-level annual sector building block assessments (completed at baseline and in 2019, 2020 and 2021).
• Endline quantitative data on WASH service levels (including comparison at baseline) for target communities and schools.

Roles and Responsibilities

WaterAid Australia’s Head of Learning and Evidence and WaterAid Timor-Leste Program Effectiveness and Advocacy Unit Manager, will jointly coordinate the review, with support from relevant technical leads and specialists.  WaterAid will provide support throughout the process and sign-off on the final report. 

Deliverables and timelines:

We expect to appoint the consultant(s) by mid-late June, with the evaluation commencing in July and being completed by mid-November 2020. The following are indicative milestones for the evaluation.

Deliverable	Indicative timeframe
Evaluation Plan	15 July 2022
Fieldwork	TBC – between late July – end September 2022.
Draft evaluation report (inc. stories of change)	15 October 2022
Validation workshop	End October 2022
Final report	15 November 2022

 

The final report must clearly set out the methodology, limitations, findings against each evaluation question and conclusions, along with the two annexes specified above. 

Resourcing

The budget for this evaluation is up to AUD $55,000 (inclusive of GST and reimbursable expenses). 

Evaluation team qualifications:

We seek an individual or small, multi-disciplinary team for this assignment who would bring specific experience with and knowledge of:

• Evaluation – particularly methods for qualitative data collection, analysis and interpretation
• Theory of change and adaptive management
• WASH systems strengthening
• Gender equality and social inclusion – particularly as it relates to transforming systemic and cultural norms
• Experience and/or knowledge of the Timor-Leste context

Request for Proposals:

Interested consultants are requested to submit a proposal with the following:

1. An EOI of up to five pages including:
   1. statement of the expertise you or your team will bring to this evaluation
   2. description of the proposed approach and methodology for the evaluation, including an indicative workplan.
2. CV for each proposed evaluation team member
3. Financial proposal with daily rates for team members (inclusive of GST) and anticipated reimbursable expenses
4. Please provide 2-3 executive summaries from recent evaluation reports written by the lead evaluator(s).

The deadline for submitting Expressions of Interest is 9am AEST Tuesday 14 June 2022. When submitting EOIs, please include reference to Water for Women Timor-Leste Final Evaluation in your email subject title.  Submissions should be made to WaterAid Australia via [email protected].  Please note that only shortlisted applicants will be contacted.

If you have any questions about this evaluation please contact Clare Hanley, Head of Learning and Evidence, WaterAid Australia at [email protected]

WaterAid is a child safe organisation. We recognise that we must meet community expectations and the trust placed in our personnel to maintain the highest standards of conduct with children. Therefore, all positions within WaterAid are required to comply with our Child Safeguarding Policy and Code of Conduct.

Annex A: WaterAid System Strengthening Building Block Assessment Rubric

 

	Emergency WASH	Fragile but strengthening	Transitional	Desired – all actors using District systems – WaterAid exit?
Coordination	Humanitarian actors coordinated through WASH Cluster by UNICEF – no one working on long term development	No coordination of agencies	Geographical coordination of agencies within the district	Agencies aligned behind comprehensive district level strategy/policy
Strategic planning	OCHA Humanitarian Action Plan or government relief plans, no development plan	Plan responding to donor priorities – sustainability not addressed	Plan in place to extend services but not to sustain them	Credible plan to deliver sustained universal access.
Financing	Emergency spending directly through NGOs and UN Agencies	No fiscal decentralized spending. Donor spending on District Plan (“On Plan”) – not covering lifecycle costs	Fiscal decentralization and donor spending on capital costs. “On Budget” but not covering lifecycle costs.	Full fiscal decentralization, external support agencies channel funds through District accounts (“On Treasury”). Lifecycle costs matched to sources of finance.
Institutional Arrangements	Focus on saving lives by providing access rather than on building life-saving institutions	Institutions exist on paper but not functional or accountable to citizens. Overlapping, unclear roles and responsibilities	Partially functional institutions with weak capacity and accountability to citizens. Roles and responsibilities not fully clear	All necessary institutions and capacities are in place with clear roles and responsibilities eg regulator, health, education, HR, IT systems. Full accountability to citizens
Service delivery	Ad hoc emergency interventions	Fragmented project interventions, multiple missions, and reporting systems. No post implementation support.	District authorities and agencies mainly focused on extending coverage. Weak post implementation support.	Both coverage and post implementation support to all users fully addressed by duty bearer.
Monitoring	Through WASH cluster and to donors	No common monitoring or review process	Common sector targets and multi-stakeholder monitoring but no aggregated reporting	Government owned, regularly updated monitoring process feeding into strategic planning.

Note – this is an abbreviated version of the building block assessment rubric. A full version, contextualised to the Timorese context will be provided to the evaluation team during the inception phase.