RFQ-LBQZQ-23-0143/ Final Program evaluation in the Bekaa

Programme

WASH and Protection Interventions in the Bekaa (covering two projects funded by different donors)

Projects Titles

DANIDA:

Towards more protected, resilient, and self-reliant crisis-affected communities in the Baalbek District of Lebanon

GAC:

Lifesaving WASH and Protection Response to the Multifaceted and Protracted Crisis in Refugee-Hosting Beqaa Region, Lebanon

Donors

Danish International Development Agency (DANIDA)

Global Affairs Canada (GAC)

Project Location

DANIDA: Bekaa (Baalbek district)

GAC: Baalbek-El Hermel Governorate and wider Beqaa region, Lebanon

Project Duration

DANIDA: April 1st, 2022 – October 31st, 2023

GAC: April 1^st, 2022 – January 31st, 2024

Evaluation to be conducted in

October – December 2023 (for approximately 35 days)

 

Introduction

The purpose of this document is to outline the terms of reference (ToR) for a comprehensive programme evaluation, encompassing two integrated projects implemented in Bekaa, Lebanon. The first project, titled “Towards more protected, resilient and self-reliant crisis-affected communities in the Baalbek District of Lebanon”, is funded by the Danish International Development Agency (DANIDA). The second project, titled “Lifesaving WASH and Protection Response to the Multifaceted and Protracted Crisis in Refugee-Hosting Beqaa Region, Lebanon”, is funded by Global Affairs Canada (GAC). These two projects, sharing similar and complementary scopes and activities, have been instrumental in addressing the pressing WASH (water, sanitation, and hygiene) and protection needs of the local communities. Following the OECD DAC criteria, the programme evaluation will assess the effectiveness, efficiency, relevance, sustainability, and impact of the programme, with the aim of providing actionable recommendations for improvement.

 

Oxfam in Lebanon

Oxfam has been operating in Lebanon since 1993 on the provision of rapid emergency humanitarian assistance to vulnerable communities, as well as long-term development support to Civil Society Organizations (CSOs) and local authorities. Through partnerships with local organizations, Oxfam in Lebanon has responded to several conflicts and crises in Lebanon, including the July War in 2006, the clashes in Nahr el Bared camp (NBC) in 2007, the Syria Crisis since 2013, the Beirut Port Blast, the Economic Collapse, Covid19 and the Cholera Outbreak.

Oxfam in Lebanon’s work is guided by a 5-year vision, which aims “to protect and empower marginalized women and men (Lebanese, Palestinian and Syrian populations including refugees) to enjoy their basic rights, and to enable them to live in dignity within a more equitable society.” Such a vision heavily relies on rights-based, citizen-based, intersectional, and partnership approaches in the development of the programs and projects. To implement this vision, Oxfam’s work focuses on two key programmatic work streams, namely Humanitarian and Governance and Economic Justice with influencing and gender justice as cross cutting themes.

Programme Overview

Oxfam seeks an external consultant/consulting firm for a final evaluation of its integrated WASH and Protection programme implemented in the Bekaa, funded by both DANIDA and GAC. Both projects started in 2022 but served as a continuation to previous DANIDA and GAC funds that have been in place since 2019. The two projects were guided by the strategic objectives set for the Humanitarian programme in Oxfam’s Country Strategy (OCS), developed in 2017, which shaped the country office’s programming for a five-year period. The following programmatic objectives, listed in the OCS, influenced the logic and implementation of these two projects:

• Marginalised women and men, particularly refugees from Syria, are able to have safe access to WASH, including more equitable, efficient and sustainable use of water resources
• Marginalised women and men attain their rights to life, security, protection and assistance, in a manner that retains and respects their dignity
• Community-based structures and relevant stakeholders are better equipped to contribute to an improved protective environment for women and men at the local level in conflict-affected areas
• Community-based structures and relevant stakeholders, and partner organizations have the tools, capacity, and network to fight SGBV (sexual and gender-based violence) among all communities in Lebanon, especially among refugees, challenging social and cultural stereotypes.

This programme integrates WASH with cash and protection programming, using WASH and cash as entry points to deliver community-based protection services and community information sessions. The programme also adheres to safe programming principles, integrating do no harm guidelines. By including community groups, such as women and youth groups, and ensuring their equal and safe access and participation, Oxfam and partners aim to prevent exclusion or increased protection risks for beneficiaries.

The intervention strategy of the programme focused on providing direct basic services while also building the capacities of local actors, including communities, to assume these roles. Oxfam’s programme was guided by the following principles:

• Strategic partnerships to ensure basic services – including cash, water and sanitation, and protection – are accessible to marginalised women and men, particularly refugees
• Strengthening national and local-level capacities for the provision of WASH, Cash/safety nets, and protection assistance to any segment of the population through supporting collaboration among national or local specialised protection agencies, and in promoting community-based protection approaches
• Strengthening accountability to beneficiaries, referral processes and quality of the wider humanitarian response

DANIDA funds:

Impact: Strengthening protection, resilience, and self-reliance of the most vulnerable crisis-affected population, in particular women and youth, in the Baalbek district, Lebanon

Specific Objective 1: Most vulnerable, crisis-affected women, men, girls, and boys have improved access to protection and WASH services

Specific Objective 2: Improved access to food, income, and livelihoods opportunities for most vulnerable households, in particular women and youth

Specific Objective 3: Local actors (local authorities, local NGOs, and community-based groups) are strengthened to take leadership and respond to needs of crisis-affected populations

The project brough together the technical expertise of two local Lebanese organizations – the Lebanese Centre for Human Rights (Centre Libanais des Droits Humains-CLDH) and Nabad for Development. CLDH took charge of the legal aspect of the project, while Nabad for Development handled the Public Health Promotion (PHP) component of the project. Oxfam directly managed the protection component.

With the support of DANIDA funds, Oxfam provided WaSH and Protection services in ten municipalities in North Bekaa, namely Chlifa, Deir El Ahmar, Houch tel Safieyeh, Houch Barada, Haour Taala, Jebaa, Btedaai, Talia, Bouday and Saaide. The project included various community-based protection activities, including Protection Focal Points (PFP) meetings, referrals facilitated by protection focal points, protection information sessions, psychosocial support sessions and assistance in accessing services. Additionally, cash for protection, legal consultations and legal representation were provided. The public health promotion component involved the distribution of menstrual hygiene management (MHM) kits, raising awareness about MHM, conducting public health information sessions, and training Community Health Volunteers (CHVs) to promote personal and environmental hygiene awareness.

Another aspect of the project focused on supporting the rehabilitation of WASH facilities led by trained and equipped women-led youth groups. Additionally, support was provided to the Bekaa Water Establishment (BWE) at both the strategic development level to strengthen local governance and the technical level through the placement of a specialized engineer to implement and monitor infrastructure projects. The project also involved training youth on solar systems, providing vocational training to young members of women’s groups and rehabilitating solar systems.

Moreover, the project highlighted the importance of involving the community, particularly youth and women, in training and capacity building initiatives. The overarching goal was to ensure the active participation of community members in decision-making processes that directly affect their lives and future.

GAC funds:

Ultimate outcome: Improved health, safety, and exercise of equitable human rights for crises-affected women, men, girls, and boys in the wider Beqaa region of Lebanon

Immediate Outcome 1110: Improved access to safe, gender-sensitive, environmentally-friendly WASH facilities, with a focus on the most vulnerable and People with Disabilities (PWD)

Immediate Outcome 1120: Strengthened community-based inclusive response to public health emergencies and disease outbreaks, especially Covid-19 and waterborne diseases

Immediate Outcome 1130: Increased community capacity to adopt and promote gender-equitable WASH practices

Immediate Outcome 1210: Increased access to inclusive, lifesaving protection information and services

Immediate Outcome 1220: Increased community capacity to prevent and mitigate protection threats with a focus on refugee women's protection and empowerment

Through funding from the current and previous GAC, Oxfam covers basic WASH needs, mitigates, and responds to imminent protection threats, such as SGBV, strengthens and builds the capacity of community structures, and establishes trust and strategic partnerships with local CSOs and stakeholders.

This project is implemented in partnership with three Lebanese organizations: NABAD for Development, CLDH, and Kafa. Nabad for Development is responsible for implementing the Public Health Promotion (PHP) aspect of the project, while Kafa oversees the Gender-based violence (GBV) component, and CLDH handles the legal component. Oxfam directly manages the remaining protection component, including community-based protection and emergency cash assistance.

The project responds to immediate WASH needs in Saaide and Bouday by providing water trucking and desludging services, constructing emergency grey water systems, building gender-sensitive and culturally-appropriate latrines and handwashing facilities, selecting and training community technical volunteers (CTVs), rehabilitating WASH facilities with the help of contractors and CTVs, as well as identifying and training community hygiene volunteers (CHVs) to promote public health information and conduct hygiene promotion activities. Additionally, the project distributes MHM and hygiene kits and carries out awareness activities to address harmful myths, communal perceptions, and stigmatization.

Protection activities are also expanded through the project by empowering communities to identify and respond to protection threats and shared concerns, providing case management for GBV survivors, offering legal assistance and representation, facilitating information sessions on accessing services, referring individuals to specialized services, and providing emergency cash assistance. Furthermore, the project collaborates with women’s groups to create safe spaces for receiving psychosocial and emotional support, delivering GBV peer-to-peer sessions through trainings, and supporting their involvement in collective action through innovative community initiatives that address protection threats and shared concerns.

Additionally, the project delivers WASH services to vulnerable Syrian refugee populations living in informal tented settlements (ITS) in North Bekaa. This includes the provision of water tanks, the facilitation of water provision through e-vouchers, and the construction of water points.

Objectives

The main objectives of the evaluation are as follows:

1. Assess the overall impact of the programme based on evidence-based outcomes (intended and unintended, positive or negative) and determine Oxfam’s contribution to identified changes
2. Assess the degree to which the programme aligns with and contributes to the Lebanon OCS strategic objectives and the degree to which the two projects were coherent in producing effects as envisioned in the programme’s Theory of Change
3. Assess the programme’s overall relevance, efficiency, effectiveness, impact, coherence and sustainability as per the OECD DAC criteria and the degree to which the design and delivery of the programme by Oxfam and its partners adhered to Oxfam’s Protection, WASH and Gender Minimum Standards in Emergencies
4.  Capture good practices and compile lessons learned and draw specific, forward-looking recommendations to inform future programming in similar context

Target audiences

Primary users of the evaluation findings will be Oxfam and partner management and staff, donor staff, and sector actors in Lebanon.

Evaluation Criteria and Sample Questions

The evaluation must adhere to OECD/DAC criteria for development evaluation. The following are sample evaluation questions organized by criterion. The evaluation consultant will finalize evaluation questions jointly with Oxfam and partners:

 

 

Relevance

• To what extent were the program's interventions driven by the needs of the community in Bekaa?
• Have the WASH and protection interventions within this programme continued to be relevant to the needs and priorities of households and communities in Bekaa? Why or why not?
  • How well does the programme integrate gender-responsive approaches and take into account the specific needs and vulnerabilities of different groups, such as women, children and refugee communities?
• To what extent did key contextual changes, threats, opportunities and learning during implementation influence implementation? Were Oxfam and partners able to adapt and meet sector standards amidst contextual changes?
• To what extent does the programme promote community ownership and participation in the planning, implementation and monitoring of WASH and protection activities?

 

 

Effectiveness

• How effective were the selection criteria over time in reaching the most vulnerable populations, especially amidst changing conditions?
• To what extent have the WASH and Protection activities achieved their intended outcomes and objectives?
• What factors (positive or negative, internal or external) have had the greatest influence on the achievement/ progress towards the desired changes/objectives? What systems are in place (at Oxfam and partners) to ensure accountability to community members and how well are the systems working?

• How well has the programme contributed to strengthening the partnerships between Oxfam and local partners (Kafa, Nabad, CLDH) and building the capacities of local NGOs, community groups and other local structures as part of Local Humanitarian Leadership in alignment with Oxfam’s Partnership Principles and Local Humanitarian Leadership Tips? To what extent have the partnerships contributed to reaching the project’s objectives?

 

Coherence and Complementarity

• Are the WASH and protection interventions complementary to each other and complementary to other ongoing interventions in the target areas? Specifically, how do Oxfam and partners’ WASH approach and activities complement the work of the municipalities, the Bekaa Water Establishment (BWE), and other INGOs contributing to the sector? How effectively does the programme align with national policies, strategies and frameworks related to WASH and protection?

 

Efficiency

• To what extent has the programme been cost-effective in terms of the approaches used, and the utilization of resources (funding, equipment, personnel)?
• How well did the programme coordinate and collaborate with relevant stakeholders to maximize efficiency in delivering WASH and protection services?
• How and to what extent are M&E findings used to inform decision-making and the improvement of project implementation by Oxfam and partners?

 

Sustainability

• To what extent were the programme’s interventions designed and implemented with a focus on long-term sustainability?
• Do shifts to community capacity building and community-driven planning and implementation, in both WASH and Protection, support longer-term sustainability of project impact?

 

Impact

• Has the programme contributed to higher-level effects (positive or negative, intended, or unintended) in the WASH and/or Protection environments of the target areas?
  • Is there potential differential impact by gender, nationality, location, or other factors?
  • Was there an increase in leadership and decision making, especially among women and marginalized groups?
  • Did any specific program intervention demonstrate positive effects and acceptance  that make them feasible for scale up?
• Were there any notable improvements in the overall living conditions and quality of life for the target communities as a result of the programme’s interventions?

 

Learning and cross-cutting themes:

 

• To what extent was the learning process embedded and demonstrated throughout the programme? (Including lessons learnt from previous programs and documentation)
• Has the programme led to any long-term shifts in gender inequitable attitudes and behaviours in relation to WASH and protection issues in the targeted communities? What mechanisms have been put in place to ensure the meaningful participation and representation of women and girls in decision-making processes related to WASH and protection activities?
• How has the program addressed and mitigated potential risks and threats to the safety and well-being of program participants in relation to WASH and protection interventions?
• How has the program promoted community ownership and mobilization in the identification and resolution of WASH and protection challenges?

 

Scope and Methodology

The evaluation must be conducted in line with evaluation best practices, drawing upon both primary and secondary data collection. The evaluator is expected to develop a detailed methodology that will be reviewed and approved by Oxfam and partners. Both qualitative and quantitative data collection methods will be employed, including document review. The evaluation methodology must make use of existing baseline and endline data and other available monitoring and evaluation data. The methodology should also consider the integration of outcome harvesting or other appropriate methods to identify unintended outcomes. The evaluator is encouraged to explore the use of these methods to capture and document the broader impacts and unintended positive or negative outcomes of the programme.

 

The methodology requires that the consultant(s) have experience with participatory approaches to learning and inquiry, particularly in seeking the views of key stakeholders such as:

• Targeted community participants (around 10,000 households and 40,000 household members, Lebanese and Syrian)
• The implementing partners Nabad (7 individuals), CLDH (2 individuals), and Kafa (2 individuals)
• Sector representatives (5 individuals)
• Local authorities and community leaders (Up to 18 municipality representatives and 3 individuals from the Bekaa Water Establishment)
• Oxfam staff involved the implementation of the two projects (e.g., PM, coordinators and field team) (Up to 20 individuals)

 

The evaluation should be conducted in an independent, impartial, and transparent manner adhering to ethical guidelines and ensuring the confidentiality of information.

The evaluator is also expected to establish a clear quality control plan to ensure the accuracy and credibility of the data collected. Data triangulation should be employed to enhance the reliability and validity of evaluation results. An online validation workshop should be conducted to present the evaluation findings to Oxfam and partner staff. This workshop will serve as one layer of data triangulation by allowing stakeholders to provide feedback, clarification, and additional insights on the findings.

By incorporating quality control measures and data triangulation techniques into the methodology, the evaluation can ensure the accuracy, credibility, and robustness of the findings. The use of an online validation workshop will also enable the evaluation team to obtain additional perspectives and insights, contributing to a more comprehensive understanding of the program's achievements and challenges.

Timeframe and Expected Level of Effort

Timeframe

The evaluation is to be carried out from October – December 2023, with the final report submitted to Oxfam as per the timeline below. The exact dates will be confirmed with the selected consultant(s).

 

Total expected level of effort: 35 working days

• Review essential documents of DANIDA and GAC projects, including but not limited to the original project proposals, previously mentioned baseline/endline and other data reports, and related internal and external research reports. Propose evaluation questions based on the review (all documents will be shared by Oxfam via the online platform Box);
• Develop a detailed inception report and evaluation plan (to be reviewed by Oxfam and partners);
• Primary data collection (a mix of in person and remote);
• Data analysis and preparation of draft evaluation report;
• Online Validation workshop;
• Finalize the final evaluation report and send it to Oxfam. Oxfam will then prepare a management response to be annexed to the evaluation report.

 

 

Reporting

The consultant will report directly to Oxfam’s Planning, Monitoring, Evaluation, Accountability and Learning (PMEAL) Advisor and other Oxfam staff in Lebanon (particularly from the Humanitarian team).

 

Expected Outputs

1. Develop a detailed inception report and evaluation plan (to be submitted after the document review but before the data collection), outlining the proposed methodology;
2. Draft evaluation report (English) to be presented to Oxfam and partners; 
3. Online validation workshop and slide deck (English and Arabic) to present and discuss draft findings with Oxfam and partners;
4. Final report incorporating Oxfam and partner comments.

 

In addition to the mentioned outputs, a daily follow-up and regular quick meetings with the programme team focal points are required. These meetings will serve as a verbal data validation process to ensure the accuracy and reliability of the collected information. These follow-ups and meetings are necessary to address any concerns or questions that may arise during the evaluation process and to ensure that the evaluation findings align with the expectations of Oxfam and its partners.

 

Please note that Oxfam will review all outputs and provide comments for the consultant to consider and incorporate. Partner comments must also be taken into account before the final report is submitted and approved.

 

Consultant Qualifications

• Demonstrated experience (minimum 5 years) in evaluation of humanitarian programs; WASH and protection experience are requirements;
• Proven experience collecting and analysing gender data;
• Excellent skills in research methods, data collection and analysis;
• Experience working in the geographic region required; experience in Lebanon preferred;
• Strong English writing and facilitation skills;
• Professional fluency in Arabic required.

 

Instructions for interested consultants

Payment will be done in two instalments: 25% upon receiving the final version of inception report, and 75% upon Oxfam’s approval of the final evaluation report.

 

Costs to include in the offer: Consultants should include the following costs in their proposed budget: daily rate, cost of travel (in the case of an international consultant living outside Lebanon), per diems, accommodation in the Zahle area, communication costs, miscellaneous costs (e.g. visa costs, depending on nationality of consultant).

 

Note that payment will be made based on the budget in the offer (not based on actual expenses incurred by the consultant). No receipts will be requested from the consultant towards the end of the evaluation.

 

Codes of Behaviour

The evaluation process must adhere to Oxfam’s guidelines for the ethical conduct of evaluations and research. The consultant and team are expected to consider key ethical implications at every stage of the evaluation.

Sharing and Using Findings

Oxfam International’s Policy on Program Evaluation requires Confederation members to act on the commitment to transparency by making public the Executive Summary and a Management Response to all final evaluations.

 

Disclosure

Although free to discuss with the authorities on anything relevant to the assignment, under the terms of reference, the consultant is not authorized to make any commitments on behalf of Oxfam. All data collected as part of this consultancy belongs to Oxfam and public dissemination of the data and evaluation products can only be done with the written consent of Oxfam.

 

Expressions of Interest (EoI)

Oxfam invites EOI from organisations, or individuals, with the experience and skills described above. The EOI must include:

1. A cover letter of no more than 2 pages introducing the evaluator/organisation and how the skills and competencies described above are met, with concrete examples. Please also use this cover letter to indicate the consultants’ availability for the proposed period;
2. An outline of no more than 5 pages of the proposed process including:
   1. Key considerations for this evaluation;
   2. Outline of proposed methodology, including proposed sampling among project target groups;
3. A CV for the evaluator(s), including 2 referees (with phone number and email address);
4. A one-page budget of the offer, covering all major anticipated costs (see above on what costs should be included);
5. A sample evaluation report that includes a WASH component
6. A sample evaluation report that includes a Protection component