External Evaluator Consultancy

Project Title

Reducing vulnerabilities for refugees and Lebanese host communities and strengthening Lebanese civil society

Donors

DANIDA

Project Location

Tripoli, North Lebanon: Jabal Mohsen, Tebbaneh, Zehriyeh, Hay Il Tanak (Mina), Qobbeh, Abou Samra neighbourhoods and the Beddawi Palestinian Camp

Project Duration

01/01/2021 – 31/12/2022

Evaluation to be conducted in

January – February 2023 (approximately 35 days)

Oxfam in Lebanon

Oxfam has been working in Lebanon since 1993 on the provision of rapid emergency humanitarian assistance to vulnerable communities, as well as long-term development support to Civil Society Organizations (CSOs) and local authorities. Through partnerships with local organizations, Oxfam in Lebanon has responded to several conflicts in Lebanon, including the July War in 2006, and the clashes in Nahr el Bared camp (NBC) in 2007, the Syria Crisis since 2013, and most recently the Beirut Port Blast.  

Oxfam in Lebanon’s work is guided by a 5-year vision, which aims “to protect and empower marginalized women and men (Lebanese, Palestinian and Syrian populations including refugees) to enjoy their basic rights, and to enable them to live in dignity within a more equitable society.” Such a vision heavily relies on rights-based, citizen-based, intersectional, and partnership approaches in the development of the programs and projects. To implement this vision, Oxfam’s work focuses on two key programmatic work streams, namely Humanitarian and Governance and Economic Justice with policy and gender justice as cross cutting themes.

Project Overview

Impact Objective: Increased Resilience and Social Stability of 12,600 Lebanese and refugee women, men, girls, and boys affected by the multi-faceted crises in Lebanon

Outcome 1: Crisis-affected populations in Tripoli and the Beddawi Camp, particularly women and youth, have strengthened agency and improved access to protection and basic services

Outcome 2: Crisis-affected populations and local actors in Tripoli and the Beddawi Camp are active agents in the promotion of protection, resilience, and social stability

The project convened the technical expertise of two local Lebanese organizations - Utopia and ABAAD – and Palestinian organization, Najdeh, alongside Oxfam to contribute to the well-being and resilience of vulnerable communities in Tripoli and the Beddawi Camp. 

The project employed a humanitarian-development – peace nexus approach utilizing emergency response activities and initiatives with intermediate impacts. The emergency response supported the most vulnerable women, men, girls, and boys by providing 1) direct and facilitated access to protection services including GBV case management and PSS with specific focus on 6 neighbourhoods in Tripoli and the Beddawi Camp, and 2) Temporary Cash Assistance (TCA) as a safety net for vulnerable families that experienced one or more shocks due to multiple crises, all to help households overcome shocks and decrease the use of negative coping mechanisms. The modality of cash, with its injection of cash into the local economy, also supported local businesses severely affected by the crises. A more indirect outcome of TCA was the reduction of social tensions because assistance benefited the broader community and not only the targeted population.  

The intermediate initiatives were closely linked to the emergency response, but with the aim of creating more sustainable solutions by 1) supporting priority community initiatives with a particular focus on women and youth, and 2) strengthening the capacities and structures of local actors including Lebanese and Syrian people and groups and public local structures including Social Development Centers (SDCs).  

The community groups included both Lebanese and refugees who jointly assessed local needs through participatory approaches, identified needs, and developed community projects for full local ownership. The collaborative approach combined with capacity development on conflict sensitivity and conflict prevention was meant to reinforce the notion of social cohesion and strengthen communal ties.  

The project aimed to advance the triple nexus approach by grounding engagements through community-based structures, recognizing that such actors were best placed to respond to emergency needs and address longer-term structural causes of instability. Activities were planned to contribute to more conducive civic space by facilitating interactions and activities with local authorities and diverse civil society actors, but they were not designed to push broader civic change or address political reform. Youth were involved as drivers of activities and as agents of change in their communities. Gender inequalities were addressed in the project design in part via GBV awareness, emergency cash, case management, plus improved access to protection and other services.  

Purpose of Final External Evaluation

The goal of this external final evaluation is to assess the overall performance of the project against OECD-DAC evaluation criteria, with a focus on contribution of the project to household and community resilience and social stability in the target areas of Tripoli. It should provide an objective analysis of the project’s nexus approach considering criteria of relevance, effectiveness, efficiency, impact, and sustainability. The evaluation should present results, conclusions, lessons learned, and recommendations that, when taken together, can help Oxfam design more impactful, evidence-driven programs in Lebanon.

External Evaluation Objectives:

• To gauge the relevance and contribution to impact of the intervention mix and nexus approach in supporting resilience capacities and social stability in the target areas;
• To identify key internal and external factors that contributed to or impeded the project’s effectiveness and efficiency, and how Oxfam and partners managed those factors;
• To understand if the nexus approach and particular interventions supported more sustainable outcomes and solutions for target communities in Tripoli; and
• To draw lessons learned from the project and make recommendations that will help inform the design of Oxfam’s future nexus programming.

Sample Evaluation Questions

• To what extent did the intervention mix (Cash plus intervention + community-based protection + capacity building with women’s and youth groups, community-based organizations, and SDCs) contribute to household and community-level resilience and social stability in Jabal Mohsen, Tebbaneh, Zehriyeh, Hay Il Tanak (Mina), Qobbeh, Abou Samra, and the Beddawi Palestinian Camp? (Impact, Effectiveness)
  • Were there differential impacts for men and women, and for different age and nationality groups, in the target areas (beyond just those involved in the project)? What were the experience and impacts among youth in particular?  

• Agency and leadership: To what extent did the intervention mix improve the agency of women and youth engaged in the project to manage change and lead initiatives in their communities? (Effectiveness)

• Community-based protection: How relevant was community-based protection as part of the overall nexus approach? Did community-based protection support wider impacts in the target areas? (Relevance, Impact, Sustainability)

• Youth: Were youth effectively engaged throughout the project? Was the intervention mix relevant for young men and women dealing with multiple shocks and stressors in the target areas? (Relevance, Effectiveness)

• Civil Society: Did Oxfam support relevant linkages across civil society actors, including supported community groups, the partner organizations, and SDC and other municipal bodies? How relevant was this approach to different target groups?  (Relevance, Effectiveness)
  • Were the necessary measures taken to build on local capacities of partners?
  • What concrete steps were/are being taken to enhance the sustainability of the project’s achievements? What is the likelihood of benefits to continue over the medium and long-term?

• To what extent did the project management and operations respond to changing context, shocks, and stressors, including economic, fiscal, and protection-related? (Efficiency, Effectiveness) 

Target Audiences

The primary audiences for and users of the evaluation findings will be Oxfam staff (Lebanon, regional, and global technical units), Utopia, Najdeh, and Abaad staff, community group and community structure members, and the project funder.

Methodology

The evaluation must be conducted in line with evaluation best practices, use quantitative and qualitative data, and draw upon both primary and secondary data collection. Any proposed sampling strategy must be adequate and representative of all target groups. The evaluator is expected to develop a detailed methodology that will be reviewed and approved by Oxfam.

The evaluation methodology must make use of, but not be limited to, the following existing data sources:

• 681 baseline/endline household surveys with recipients of temporary cash assistance (TCA) and referral information 
• Referral monitoring
• Emergency cash assistance monitoring data
• Post-distribution monitoring data
• Community protection focal points’ thematic reports and meeting notes
• 76 baseline/midline/endline with members of women’s groups, youth groups, and SDC employees who received multi-session capacity building and support
• Partner self-assessments

The methodology requires that the consultant(s) have experience with participatory approaches to learning and inquiry, particularly in seeking the views of key stakeholders such as:

• Supported households, individuals, and civil society actors
• Management, project, and administrative staff of Oxfam and implementing partners Utopia, Najdeh, and Abaad 

Timeframe, Key Activities, and Expected Level of Effort

Timeframe

The evaluation is to be carried out in January and February 2022, with the final report submitted to Oxfam by 10 March 2023. The exact dates will be confirmed with the selected consultant(s).

Total expected level of effort: 35 working days

• Inception Phase:

• Desk review of project documents, including but not limited to the original project proposal, interim and final narrative reports, partner reports and existing internal and external reports (all documents will be shared by Oxfam via the online platform Box)
• Preparation of an Inception Report, including evaluation questions and a detailed evaluation plan (to be endorsed by Oxfam)

• Primary data collection
• Data analysis and preparation of draft evaluation report
• 2 rounds of review by Oxfam and partners
• Online Validation workshop
• Finalize the final evaluation report and send it to Oxfam. Oxfam will then prepare a management response to be annexed to the evaluation report

Expected Outputs

1. Develop a detailed inception report and evaluation plan (to be submitted after the document review but before the data collection - to be endorsed by Oxfam), outlining the proposed methodology including budget, timeline, and data collection tools
2. Draft evaluation report (English), with up to 3 rounds of comments/reviews before validation, to be presented to Oxfam and partners
3. Online validation workshop and slide deck (English and Arabic) to present and discuss draft findings with Oxfam and partners
4. Final report (English) incorporating Oxfam and partner comments.

Please note that Oxfam will review all outputs and provide comments for the consultant to consider and incorporate. Partner comments must also be considered before the final report is submitted and approved.

Consultant Qualifications

• Demonstrated experience (minimum 5 years) working with NGOs in complex crisis environments
• Demonstrated experience in conducting project evaluations with all of the following:
  • Humanitarian cash projects
  • Resilience programming (can be agriculture based)
  • Social stability or social cohesion projects
• Excellent skills in research methods, data collection, and analysis
• Experience working in Lebanon and documented experience in the city of Tripoli
• Knowledgeable about the social, political, economic, and cultural context in Lebanon especially since 2019
• Strong verbal communication skills in Arabic and English and facilitation skills
• Excellent report-writing and presentation skills in English and Arabic

Instructions for interested consultants

Payment will be done in two instalments: 25% upon contract signature, and 75% upon Oxfam’s approval of the final evaluation report.

Costs to include in the offer: Consultants should include the following costs in their proposed budget: daily rate, cost of travel (in the case of an international consultant living outside Lebanon), per diems, communication costs, miscellaneous costs (e.g., visa costs, depending on nationality of consultant).

Note that payment will be made based on the budget in the offer (not based on actual expenses incurred by the consultant). No receipts will be requested from the consultant towards the end of the evaluation.

Codes of Behaviour

The evaluation process must adhere to Oxfam’s guidelines for the ethical conduct of evaluations and research in addition to the Non-Staff Code of Conduct. The consultant and team are expected to consider key ethical implications at every stage of the evaluation.

Sharing and Using Findings

Oxfam International’s Policy on Program Evaluation requires Confederation members to act on the commitment to transparency by making public the Executive Summary and a Management Response to all final evaluations.

Disclosure

Although free to discuss with the authorities on anything relevant to the assignment, under the terms of reference, the consultant is not authorized to make any commitments on behalf of Oxfam. All data collected as part of this consultancy belongs to Oxfam and public dissemination of the data and evaluation products can only be done with the written consent of Oxfam.

Expressions of Interest (EoI)

Oxfam invites EOI from organisations, or individuals, with the experience and skills described above. The EOI must include:

1. A cover letter of no more than 2 pages introducing the evaluator/organisation and how the skills and competencies described above are met, with concrete examples. Please also use this cover letter to indicate the consultants’ availability for the proposed period
2. An outline of no more than 5 pages of the proposed process including:
   1. Key considerations for this evaluation
   2. Outline of proposed methodology for this evaluation
3. A CV for the evaluator(s), including 2 referees (with phone number and email address)
4. A one-page budget of the offer, covering all major anticipated costs (see above on what costs should be included)
5. Two sample reports for an evaluation conducted for humanitarian and/or humanitarian-development projects.