Final Evaluation of WASH and Protection Response to the Syria Crisis in Lebanon

Terms of Reference for Final Evaluation of WASH and Protection Response to the Syria Crisis in Lebanon

Project Title

WASH and Protection Response to the Protracted Syrian Refugee Crisis in North Bekaa, Lebanon 2020-2021

Donors

Denmark Ministry of Foreign Affairs (DANIDA)

Project Location

Bekaa (North Bekaa- Saaide and Bouday)

Project Duration

January 1st, 2021 -  March 31st, 2022

Evaluation to be conducted in

February - April 2022 (for approximately 35 days)

 

About Oxfam

Oxfam in Lebanon works to ensure that women and men are protected and empowered to enjoy their basic rights and access services that support a dignified life within a more equitable society. To achieve our mission, Oxfam implements activities under two programmatic pillars: 1) Humanitarian and 2) Good Governance and Economic Justice.

Projects under our Humanitarian programme often employ cash interventions with social protection strategies to provide more holistic, sustainable support to households to weather economic shocks and prevent and mitigate protection threats.

Project Overview

Oxfam seeks an external consultant for a final evaluation of our integrated WASH and Protection project implemented across 263 informal settlements and host communities in North Bekaa. The project is funded by DANIDA and builds on programming in place since 2019 funded by both DANIDA and Global Affairs Canada (GAC). 

 

Specific Objective 1: Girls, boys, women, and men have increased access to WASH and protection services

Specific Objective 2: Syrian and Lebanese civil society are capacitated to participate and strategically influence the rights and needs of conflict-affected populations

 

Implementation

Oxfam implements the project in two ways 1) through Lebanese partners NABAD for Development, Lebanese Centre for Human Rights (Centre Libanais des Droits Humains-CLDH), KAFA and ALEF, and 2) directly with Oxfam staff teams. This evaluation applies to the delivery of the project through both modalities, which feed data into indicator tracking covering both logframes. 

 

The project responds to immediate WASH needs in a gender-sensitive, inclusive and participatory way that adheres to humanitarian principles and international standards (Sphere, Core Humanitarian Standard - CHS, Charter for Change) as follows:

 

• Oxfam will provide access to gender-sensitive latrines, adhering to a maximum of 15 persons/latrine. For every latrine, Oxfam provides a one-off cleaning kit. All construction works abide by Oxfam's Safe Programming principles, ensuring access is cleared of obstruction and that there are door handles, locks, lighting (solar) and rope handles for support. Location, design and positioning are selected in consultation with refugees, with a focus on women and girls, and persons with disabilities (PwD).

 

• For facility (latrines) rehabilitation, Oxfam shifted maintenance from implementation via contractors to a community-led modality. Oxfam selects Community Technical Volunteers (CTVs) and provides them with (on-the-job) training, a WASH facility toolkit and spare parts. This market-based approach greatly reduces maintenance costs compared with private contractors, empowers beneficiaries to engage with local markets, and prevents overcharging of refugees.

 

• Ongoing desludging services will be maintained for existing and new latrines and septic tanks via Oxfam-contracted desludgers to ensure functionality, mitigate community tensions within the ISs that arise from sharing latrines, reduce open defecation, and prevent proliferation of hand-dug pits that expose people to health and protection risks. Desludging is conducted every 2 months, as per latrine pit design requirements; households coordinate with the desludgers and pay with an e-voucher.

 

• Oxfam supports all HHs with water-trucking services through the e-voucher modality, complemented with a distribution of water tanks. Oxfam will also support rehabilitation, fees provision and water quality monitoring of water points to ensure the water is safe. 

 

• Public Health and Hygiene Promotion (PHP) activities are prioritized to reduce public health risks and environmental impacts. Oxfam has shifted from directly-implemented to community-based sessions facilitated by Community Hygiene Volunteers (CHVs), children’s groups and women’s groups – to favor community ownership and sustainability. Oxfam and Nabad, along with refugees, will celebrate World Water Day for the two years to raise awareness on water as a fundamental human right, leveraging the occasion to advocate for greater donor support of the WASH Sector in Lebanon.

 

• Oxfam identified the need for Menstrual health management (MHM) support to women and girls, who consistently report challenges. Research on MHM amongst Syrian refugee women/girls in the Bekaa is ongoing by Oxfam, to explore women’s experiences, perceptions and practices and present solutions. The project will support all menstruating-aged women and girls in the 101 ISs with sanitary product distribution and wider educational/awareness activities to address harmful myths, communal perceptions and stigmatization.

 

• Oxfam introduced new activities and approaches to further mitigate adverse public health and environmental impacts. Referrals and field assessments revealed that lack of and/or open grey water systems lead to health and protection issues. Oxfam built grey water systems for water disposal to mitigate risks from ad-hoc pit construction. Oxfam introduced septic tanks as per Water Sector and Ministry of Energy and Water (MoEW) guidance to minimize sewage pollution of underground systems. The septic tanks were also part of an eventual exit strategy to minimize the burden on refugee communities of covering the costs of desludging in the future. In line with the MoE’s request that humanitarian actors support solid waste management in ISs, Oxfam provided solid waste sorting bins as well as heavy solid waste sorting collection containers to HHs, plus awareness sessions on sorting and recycling.

 

The project also scaled up Protection activities by further empowering communities to identify and respond to shared threats and concerns, as follows:

 

• Oxfam and partners provide case management for GBV survivors, those at-risk and children experiencing or at-risk of child protection (CP) violations, among others. Legal assistance regarding administrative procedures, intervention in courts and legal representation is provided for cases of childbirth and marriage registration, lawyer and detention fees.

 

• Capacity-building will continue to be provided on legal issues, GBV, basic services, and safe referrals for women and girls community leaders, FPs, and local actors, as well as information sessions on legal services and GBV among other topics to all HHs. In parallel, referrals by Oxfam, partners and FPs to specialized services available will take place as required.

 

• Oxfam works with refugees to identify community initiatives, where community groups identify protection threats and mitigation measures, and organically identify community projects to benefit the entire community benefits.  

 

• Oxfam and KAFA work with women’s groups in a collective process of support, solidarity and capacity-building that includes facilitated discussions, psychosocial and emotional support, training to deliver GBV peer-to-peer sessions, and support for groups to engage in collective action through innovative community initiatives that respond to shared threats and concerns.

 

• Oxfam provides Cash-for-protection (CFP) to (a) prevent the materialisation of a Protection risk, (b) respond to the consequences of a Protection incident and (c) reduce exposure to a threat. The amount received can be used to access legal documentation and Protection services (e.g. service providers not present in the immediate proximity), address housing emergencies (e.g. immediate threat of eviction, fires). Oxfam couples CFP with other services (e.g., case management or longer-term cash assistance) and refers cases to specialized service providers as needed.

 

To date, our WASH activities have benefited 875 households, 5,153 individuals (412 girl infant, 567 boy infant, 464 girl children, 515 boy children, 360 girl teen, 309 boy teen, 1,237 adult women, 1,031 adult men, 206 elderly women, 52 elderly men) in 103 ISs. Protection activities have benefited 3,733 people (1,668 women and girls; 2,065 men and boys) in 30 ISs (those with 7 tents or more).

 

Under Specific Objective 2, Oxfam and partner ALEF facilitate monthly meetings with Nabad, CLDH and Kafa to develop their policy and advocacy work individually and collectively. Partners collect and analyse data mapped to policy gaps around refugee protection, then produce policy products and identify advocacy opportunities.

 

Purpose and Scope of Evaluation

 The specific objectives of the final evaluation are to:

• Explore if and how the project has impacted upon protection of the target and affected population and contributed to a reduction of vulnerability at the individual, household and community level;
• Explore if and how the project has impacted the WASH environment at the household, community and municipal level;
• Identify and assess key internal and external factors that have supported, affected, or impeded project outcomes and potential impact, and how Oxfam and partners have managed these factors;
• Assess the design, planning, delivery and management of the project by Oxfam and its partners in accordance with Oxfam’s Protection and WASH Minimum Standards in Emergencies, Oxfam Minimum Standards for Gender in Emergencies, and Oxfam’s updated Community-based Protection guidance;
• Assess the project in relation to Oxfam’s guidance on local humanitarian leadership;
• Draw key lessons and learning from the project at multiple levels and make recommendations to inform future Oxfam humanitarian programming in Lebanon.

 

Target audiences

Primary users of the evaluation findings will be Oxfam and partner management and staff, donor staff, and sector actors in Lebanon.

 

Evaluation Criteria and Sample Questions

The evaluation must adhere to OECD/DAC criteria for development evaluation. The following are sample evaluation questions organized by criterion. The evaluation consultant will finalize evaluation questions jointly with Oxfam and partners:

 

Impact

• Has the project contributed to higher-level effects (positive or negative, intended or unintended) in the WASH and/or Protection environments of the target areas?
  • Is there potential differential impact by gender, nationality, location, or other factors?
• Has the project had higher-level effects on Covid-related behaviours in the community and/or among partners?

 

Relevance

• Have the WASH and protection interventions continued to be relevant for households and communities as economic, Covid, and subsidy changes impact the area? Why or why not?
  • Has the project been relevant when considering gender differences in both WASH and protection work, across host and refugee communities?
• To what extent did key contextual changes, threats and opportunities during implementation influence project implementation? Were Oxfam and partners able to meet sector standards amidst contextual changes?

 

Effectiveness

• How effective were the selection criteria over time in reaching the most vulnerable populations, especially amidst changing conditions?
• What unintended consequences (if any), whether positive or negative, has the intervention had on women, men, boys and girls so far (refugees and the host community)?

• What systems are in place (at Oxfam and partners) to ensure accountability to community members and how well are the systems working?
• Has the program contributed to strengthening the partnerships between Oxfam and the four local partners (Kafa, Nabad, CLDH, ALEF) and enabled local humanitarian leadership among local NGOs, community groups and other local structures aligned with Oxfam’s 7 Partnership Principles and Local Humanitarian Leadership Tips?

 

Coherence

• Are the WASH and protection interventions complementary to related work in the target areas? Specifically, how do Oxfam and partners’ WASH approach and activities interact with the work of the municipalities and Bekaa Water Establishment (BWE)?
• Are policy and advocacy efforts supported by the project aligned with policy and advocacy at the sector level?

 

Efficiency

• Cost-effectiveness and cost-savings of the approach, particularly addition of e-voucher mechanism and shift to community-led work.
• How and to what extent are M&E findings used to inform decision-making and the improvement of project implementation by Oxfam and partners?

 

Sustainability

• Do shifts to community capacity building and community-driven planning and implementation, in both WASH and Protection, support longer-term sustainability of project impact?
• Has the project design and/or implementation seeded any long-term shifts in gender inequitable attitudes and behaviours in relation to WASH and protection issues? 

 

Methodology

The evaluation must be conducted in line with evaluation best practices, drawing upon both primary and secondary data collection. The evaluator is expected to develop a detailed methodology that will be reviewed and approved by Oxfam and partners. The evaluation methodology must make use of existing project baseline and endline data and other available monitoring and evaluation data, including:

• Project baseline and endline (in process) with 290 households
• Extensive monitoring data including PDMs, spot checks and qualitative data
• Previous DANIDA project evaluation completed in June 2020
• A comprehensive qualitative mid-term review completed in July 2021 that covered work during 2020, plus a corresponding management and adaptation plan across all project outcomes and objectives
• A 2021 independent evaluation of Oxfam’s community-based protection approach
• An August 2021 needs assessment with 448 men and women across project locations
• An MHM research report
• A gender analysis with data collected across intervention communities
• An evaluation of the pilot period of the Red Rose e-voucher system for WASH services (Oxfam began using this e-voucher system in mid-2021 to replace paper vouchers and multi-step monitoring) 

 

The methodology requires that the consultant(s) have experience with participatory approaches to learning and inquiry, particularly in seeking the views of key stakeholders such as:

• Targeted community participants
• The implementing partners Nabad, CLDH, Kafa and ALEF
• Sector representatives
• Local authorities and community leaders
• Oxfam staff involved the implementation of the project (e.g., PM, coordinators and field team)

 

Timeframe and Expected Level of Effort

Timeframe

The evaluation is to be carried out from February – April 2022, with the final report submitted to Oxfam as per the timeline below. The exact dates will be confirmed with the selected consultant(s).

 

Total expected level of effort: 35 working days

• Review essential documents of the project, including but not limited to the original project proposal, previously mentioned baseline/endline and other data reports, and related internal and external research reports. Propose evaluation questions based on the review (all documents will be shared by Oxfam via the online platform Box);
• Develop a detailed inception report and evaluation plan (to be reviewed by Oxfam and partners);
• Primary data collection (a mix of in person and remote);
• Data analysis and preparation of draft evaluation report;
• Online Validation workshop;
• Finalize the final evaluation report and send it to Oxfam. Oxfam will then prepare a management response to be annexed to the evaluation report.

 

Reporting

The consultant will report directly to Oxfam’s Humanitarian Program Coordinator and work closely with Oxfam’s Planning, Monitoring, Evaluation, Accountability and Learning (PMEAL) Coordinator and other Oxfam staff in Lebanon.

 

Consultant Qualifications

• Minimum of 5 years of experience in evaluation of humanitarian programs; WASH and protection experience are requirements;
• Proven experience collecting and analysing gender data;
• Experience working in the geographic region required; experience in Lebanon preferred;
• Strong English writing and facilitation skills;
• Professional fluency in Arabic required.

 

Expected Outputs

1. Develop a detailed inception report and evaluation plan (to be submitted after the document review but before the data collection), outlining the proposed methodology;
2. Draft evaluation report (English) to be presented to Oxfam and partners (2 rounds of comments); 
3. Online validation workshop and slide deck (English and Arabic) to present and discuss draft findings with Oxfam and partners;
4. Final report incorporating Oxfam and partner comments (2 rounds of comments).
5. A 2-3 pages executive summary/communications document (template to be determined)

 

Please note that Oxfam will review all outputs and provide comments for the consultant to consider and incorporate. Partner comments must also be taken into account before the final report is submitted and approved.

 

Instructions for interested consultants

Payment will be 100% in fresh USD upon Oxfam’s approval of the final evaluation report.

 

Costs to include in the offer: Consultants should include the following costs in their proposed budget: daily rate, cost of travel (in the case of an international consultant living outside Lebanon), per diems, accommodation in the Zahle area, communication costs, miscellaneous costs (e.g. visa costs, depending on nationality of consultant).

 

Note that payment will be made based on the budget in the offer (not based on actual expenses incurred by the consultant). No receipts will be requested from the consultant towards the end of the evaluation.

 

Codes of Behaviour

The evaluation process must adhere to Oxfam’s guidelines for the ethical conduct of evaluations and research. The consultant and team are expected to consider key ethical implications at every stage of the evaluation.

 

Sharing and Using Findings

Oxfam International’s Policy on Program Evaluation requires Confederation members to act on the commitment to transparency by making public the Executive Summary and a Management Response to all final evaluations.

 

Disclosure

Although free to discuss with the authorities on anything relevant to the assignment, under the terms of reference, the consultant is not authorized to make any commitments on behalf of Oxfam. All data collected as part of this consultancy belongs to Oxfam and public dissemination of the data and evaluation products can only be done with the written consent of Oxfam.