Evaluator in Livelihood - Lebanon

Call for Evaluator in Livelihood

Caritas Germany is a catholic relief organization dedicated to provide humanitarian assistance to people affected by man-made and natural disasters worldwide. From our headquarter in Freiburg, Germany, we support approximately 680 projects each year in both sudden-onset and protracted crises across 78 countries. In our humanitarian assistance, we trust in the experience and expertise of our local partners as a matter of principle. For decades, we have invested in capacity-building of local partners. Our localized approach enables us to deliver sustainable humanitarian assistance and preparedness work that contributes towards promoting equality and reconciliation.

In cooperation with our local partner Caritas Lebanon we are looking for a qualified Evaluator in Livelihood.

Terms of Reference

Final Evaluation, “Contribution to improving the living conditions of Syrian refugees and vulnerable Lebanese in the Bekaa, Akkar and Lebanon Mountains regions”

1. Project Information

Project Title: “Contribution to improving the living conditions of Syrian refugees and vulnerable Lebanese in the Bekaa, Akkar and Lebanon Mountains regions.”

Project Number: P. 311-2017-007; BMZ 2017.4067.9

Project Period: 01.12.2017 – 31.12.2020

Project Evaluation Period: 01.12.2017 – 01.11.2020

Overall Budget: 2 Mio. EUR

Project Donors: German Federal Ministry of Economic Cooperation
and Development (BMZ); Caritas Germany

Project Partner: Caritas Lebanon

2. Background and Context

For more than nine years the continuous refugee crisis in Lebanon is putting enormous pressure on the refugee and host communities and their resources. Many refugees still live in very poor conditions; many Lebanese feel the pressure and competition in the labour market as well as the over-use of public and local services due to 1,5 Mio. additional refugees in the country.

The additional schooling needs of Syrian refugee children put the Lebanese education system under massive pressure. Children go to school in shifts. And schools in districts with large refugee populations (e.g. Bekaa) are overcrowded which creates a negative learning environment.

The situation for refugees remains dire. Additional income and a valid residence permit are essential. The increased pressure and competition on the labour market decreases wages and creates unfavourable working conditions for Lebanese and Syrian workers. For immediate effects, labour market policy is not sufficient. Therefore, a combination of (temporary) job opportunities and humanitarian protection for extremely vulnerable individuals seems appropriate to achieve immediate effects and strengthen the institutional structure which is necessary to provide the much- needed services to people in need.

The project was jointly designed by Caritas Lebanon (CL) and Caritas Germany (CG). The project is implemented by CL with technical support by CG. The project activities are taking place in the 10 Primary health care centers operated by CL, which are located all over Lebanon.

Since December 2017 Caritas Germany and Caritas Lebanon are implementing a livelihood project for Syrian refugees, vulnerable Lebanese as well as migrants. Until December 2020, the project will reach 1.675 persons, 50% refugees (including Syrians, Iraqis, Palestinians) and 50% vulnerable Lebanese and migrants.

The project has the following objective:

Objective 1: Syrian refugees and vulnerable Lebanese have improved their access to essential goods
and services

Output 1.1: The beneficiaries of the school rehabilitation component have earned income under
adequate working conditions and (where necessary) have received legal advice from
Caritas Lebanon.

Output 1.2: The beneficiaries of the two women's shelters Reyfoun and Bhershaf have earned
income and learned new skills by working in the socio-economic greenhouse project.

Output 1.3: The beneficiaries have a permanent employment in the social enterprise "Catering",
which contributes to the preservation of the soup kitchen.

In order to achieve this objective the following activities were implemented:

• Rehabilitation of schools: In order to rehabilitate 17 semi-private schools and provide income opportunities though CfW, schools have been identified, technically assessed and suitable construction companies have been selected. The construction contractors were trained on decent work conditions and modalities of CfW.
• Socio-Economic project: vegetable gardens: As a first step the gardens and greenhouses were set-up. The CfW-beneficiaries received on-the-job training through skilled staff. In order to secure the continuation and sustainability of the vegetable gardens a business concept for the sale of the harvest was developed.
• Social Enterprise (U-Feed): The kitchen facilities were rehabilitated in order to meet the requirements of a social catering company. At the same time a business plan was developed. The newly recruited staff was trained. In order to consolidate the social catering company business plan was revised and marketing strategies are currently adapted.

Current State:

By mid of July 2020, the school rehabilitation activities are ongoing. So far, the rehabilitation of six schools is completed, in four schools the rehabilitation works are ongoing and expected to be finalized by end of July. Schools for the last phase are selected and the technical assessments are on- going. Regarding the socio-economic project (vegetable gardens), the gardens and greenhouses have been successfully established and CfW activities as well as on-the-job training are ongoing. A draft guideline to ensure the continuation of the activities beyond the project period was elaborated. For the social enterprise U-Feed, the rehabilitation of the kitchen facilities was completed in 2018. Staff was recruited and trained and a first business plan was developed. The process of revising and adapting the business plan is ongoing.

3. Objective and Scope of the Evaluation

This final evaluation has a distinct focus on the social and socio-economic dimensions of the project. Technical aspects such as quality of rehabilitation works are not within the scope of this evaluation.

The objective of the evaluation is to obtain a systematic and objective assessment of the Cash for Activities as well the set-up and sustainability of the socio-economic projects, since both are relatively new concepts to Caritas Lebanon.

The main purpose of the evaluation is learning. The results are to be used by Caritas Lebanon and Caritas Germany to strengthen future programming and will also be presented to the German Federal Ministry for Economic Cooperation and Development (BMZ). It is therefore expected that the evaluation report provides clear recommendations on the design and implementation of the activities.

The evaluation will cover the project period from 1.12.2017-01.11.2020.

4. Evaluation Criteria and Key Evaluation Questions

The specific objectives to be addressed by this evaluation are expected to be further refined in consultation with the evaluator during the inception phase of the evaluation process. The following key questions are thus indicative.

The evaluation is to focus on the following DAC criteria:

Relevance/Appropriateness:

• To what extent are the activities in line with needs and priorities identified?
• What parts of the affected population benefited from the project? Is this population corresponding to the most vulnerable community members?
• Was the design and implementation of the intervention gender sensitive?

Specific questions:

What proportions of the affected population could be assisted? Who was excluded, and what were the key barriers to full access?

• What critical factors (i.e. security events, infrastructures, procedures, access, enabling environment, etc.) help explain why the project was or was not delivered in an adequate and timely
• manner?
• How far has the project been tailored to meet local needs and ensure ownership by, and accountability to the targeted populations? What measures are in place to ensure transparency?
• What did the beneficiaries appreciate most/ less about the activities?
• In what way were adopted modalities the best way of meeting recipient’s livelihood needs?

Effectiveness

• To what extent does the intervention achieve the expected outcome?
• What are factors (internal and external) hindering the achievement of the outcome, what are factors supporting the achievement of the outcome?
• Is the activity designed and implemented in an adequate way to achieve the intended outcome? What steps can be taken to increase the effectiveness of the activity, both in terms of design and implementation?

Specific questions:

• Are we reaching the right people? Is the process of beneficiary selection designed in an adequate way to identify beneficiaries according to the eligibility criteria? Are the selection criteria applied adequately? How can the process of selection be improved both in terms of optimal targeting and efficiency?
• Are the monitoring tools and processes adequate to measure the effectiveness of the intervention?

Efficiency:

• To what extent are the activities implemented in a timely manner?
• How economically were resources converted into results? Was the project implemented in the most efficient way (in terms of time and resource allocation, planning, coordination, and communication)?
• In which areas are potentials to increase efficiency?

Specific questions:

• Was the set timeframe sufficient for the implementation of the set activities?
• To what extent were the activities implemented as planned?
• Have considerable unforeseen costs arisen?
• Did project activities overlap in any way with other actors’ intervention in the target area?

Impact:

• Which effects did the activity have on the beneficiaries (intended/un-intended, positive/negative)?
• What were the gender-specific impacts? Did the intervention influence the gender context?

Specific questions:

• Is the project contributing to systemic changes?

Sustainability:

• What are the external factors that will help to sustain the positive effects of the project?
• What are the internal factors that support to sustain the positive effects of the project?
• To what extent has the project established processes and systems that are likely to support the continued implementation of the project?

5. Approach and Methodology

The evaluation will adopt a participatory and transparent approach using internal and external stakeholders. The evaluation process, as well as the final results should be gender-sensitive and human-rights-based. The evaluator will use mixed methods that are able to appropriately address the primary evaluation questions, as well as properly support its derivative conclusion and recommendation.

The evaluation methods to be used may include, but are not limited to:

• Review of project documentation, reports and literature (Desk Research)
• Consultative meetings with staff, management and key stakeholders
• Review and analysis of existing Monitoring Data; additional collection of quantitative data if necessary in order to answer key evaluation questions
• Key informant interviews with beneficiaries, staff members and other stakeholders.
• (Semi-)structured Interviews with beneficiaries are a must and should be supported by interview protocols and the list of respondents. The exact sample size should be proposed by the evaluator and discussed together with Caritas Lebanon and Caritas Germany before the start of the evaluation.

6. Deliverables:

i. Inception Report: An inception report will be submitted by the evaluating consultant in order to demonstrate his understanding and planning of the evaluation, which will be reviewed and discussed in cooperation with Caritas Lebanon and Caritas Germany. The inception report should

• an evaluation matrix (including the final evaluation questions and indicators);
• the overall evaluation design and methodology with a detailed description of the data collection methods and data analysis techniques,
• a proposed timeframe for the activities and deliverables. The evaluator is free to suggest additional methodological framework for the evaluation.
• highlight any reservations regarding the feasibility

The inception report of the evaluation should not exceed 10-15 pages and follow a predefined structure.

If the inception report contains proposed changes regarding the methodology or the TORs of the evaluation, these are subject to the written agreement of Caritas Germany and Caritas Lebanon.

ii. Validation Workshop: A workshop should be conducted in the field to present the findings of the evaluation to key staff.

iii. Draft Report: The Draft Report should be presented to Caritas Germany’s desk officer after the field work has been concluded, and should incorporate comments supplied by Caritas Germany and its partner organizations.

iv. Final Report: The final report serves to illustrate the relevant evidence corresponding with the evaluation issues, questions and criteria listed in the Terms of Reference. The report should take all aspects reviewed in the validation workshop and Caritas’ comments on the draft report into consideration and is subject to approval of Caritas. The report should include an executive summary in English.

All documents are to be written in English language. The final report should not exceed 30 pages (excluding annexes).

7. Timeframe for Deliverables

The evaluation is to be carried out in October/ November (exact timeframe and dates to be discussed). The timeline for the activities consists of the following phases:

Preparation Phase: 4 days

• Analysis of relevant project documents, as well as further research
• Exchange with Caritas Lebanon and Caritas Germany
• Preparation of inception report

Field Phase: 7 days

• Briefing with Caritas Lebanon, Caritas Germany and other relevant actors in Beirut
• Data collection
• Validation meeting involving Caritas Lebanon and Caritas Germany

Synthesis Phase: 4 days

• Analysis of findings and preparation of draft evaluation report
• Revision of draft report to integrate comments and feedback from Caritas Lebanon and Caritas Germany.

The evaluator should present a detailed work plan and timeframe of all activities including the relevant resource allocation.

The deadlines for each of the deliverables will be set before the start of the evaluation.

8. Roles and Responsibilities

Caritas Lebanon is responsible for organizing and facilitating the logistics in Lebanon. Caritas Lebanon and Caritas Germany will provide access to all relevant project documents.

The evaluator will be working under and reporting to the Caritas Germany desk officer and Caritas Germany’s Advisor for Humanitarian Assistance during the evaluation phase.

9. Guiding Principles & Donor concepts

The consultant is to conduct the evaluation in accordance with the principles outlined in the “Caritas Internationalis management standards” document [1], the “Caritas Code of conduct”, Caritas “Safeguarding Policy” as well as the “Guidelines on Combating Fraud and Corruption in the Project Work of Caritas Germany” [2].

The evaluator must take all required steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of the people and the communities of which they are members, as well as to ensure that the evaluation is technically accurate, reliable, and legitimate, and conducted in a transparent and impartial manner. Moreover, the evaluation should ideally contribute to organizational learning and accountability.

10. Professional Qualification and Experience

The evaluation consultant should meet with the following, required skills and competence profile:

• A university degree at post-graduate level or an equivalent combination of education and relevant work experience
• Extensive experience in conducting evaluations and a proven record in delivering professional results (verifiable list of evaluations conducted in the past is required)
• In-depth knowledge and experience in the livelihood sector (espc. the concept of Cash for Work)
• Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and prepare well-written reports in a timely manner
• Extensive Experience in qualitative and/or quantitative data collection and data analysis techniques
• Fluency in written and spoken English, ideally in conjunction with good skills in spoken French and/ or Arabic
• Regional experience in the Middle East
• Good understanding and appropriate sensitivity in regards to different cultures and traditions

11. Terms of Payment

The evaluator is to receive payment from Caritas Germany in different installments, details will be explained bilaterally.