Terms of Reference (ToR) for Mid-Term Internal Project Evaluation “ILO Bouzour Project - Building Inclusive Markets and Decent Work for Syrian Refugees and Lebanese Host Communities”

BACKGROUND INFORMATION

Project Context

Lebanon has been suffering from a severe financial and economic crisis with long-lasting effects on its economy and its people. With the drastic devaluation of the currency, prices of imported items have risen sharply, and households are now increasingly struggling to cover food needs. The combined effects of the crises have resulted in unprecedented levels of unemployment and pushed both the Lebanese population and the approximately 1.5 million Syrian refugees further into poverty. The Lebanese agriculture sector remains one of few sectors where both Syrian refugees and vulnerable Lebanese host communities can legally find employment as well as vulnerable Lebanese host communities but has been particularly hard-hit by the ongoing crises.

The International Labour Organization (ILO) is therefore funding a project aimed at ‘Promoting Decent Jobs for Lebanese host communities and Syrian refugees’ that is funded by the Swedish International Development Agency (Sida). The project focuses on the agriculture and agri-food sector as a key sector for the livelihoods of both Lebanese and Syrian vulnerable population groups and in particular women of both communities and aim to develop specific agricultural and agri-food value chains with potential for decent job creation that benefits both groups.

The project makes use of the ILO’s Approach to Inclusive Market Systems (AIMS) to safeguard jobs and livelihoods of vulnerable Lebanese and Syrian communities threatened by the crises and encourage decent job creation for both groups in sectors with potential. To that end, the project makes use of the market systems development approach to address key sectoral constraints to decent job creation sustainably, in combination with more direct interventions to strengthen capacities of vulnerable Lebanese and Syrian communities and safeguard decent work conditions.

The project seeks to achieve four key outcomes with interlinked components:

1. Enhanced Productivity and Working Conditions in the Horticulture Sector: The project supports farmers in adopting modern greenhouses and sustainable farming practices, improving productivity, crop quality, and working conditions for both Lebanese and Syrian workers.
2. Upgraded Market Opportunities for Farmers and Agri-Businesses: By identifying and capitalizing on local and international market opportunities, the project helps farmers and agri-businesses in selected sectors to access upgraded markets, fostering inclusive growth.
3. Adoption of Solar Energy Systems for Sustainable Electricity Generation: To counter the energy crisis, the project promotes solar energy systems for farmers, reducing costs, ensuring stable electricity supply, and leading to environmental benefits.
4. Capacity Building for Efficient Resource Management: The project empowers farmers and small agri-businesses with essential business management skills, enhancing their ability to plan, calculate profits, and make informed decisions.

PURPOSE AND SCOPE OF THE EVALUATION

Evaluation Background

ILO considers evaluation as an integral part of the implementation of development cooperation activities. Provisions are made in all projects in accordance with ILO evaluation policy and based on the nature of the project and the specific requirements agreed upon at the time of the project design and during the project as per established procedures. The Regional M&E and Knowledge Management Officer at the ILO ROAS supports the evaluation function for all ILO projects.

In accordance with the ILO evaluation policy, two evaluations are required during the project implementation, one at the mid-term and the other at the end of the Action.

The project document states that a mid-term internal evaluation will be conducted, which will be used to enhance learning within the ILO and among key stakeholders and guide the project team in adjusting if necessary the project’s approach and scope for implementation during the project’s remaining period.

Purpose

The purpose of the mid-term internal evaluation is to enhance learning within the ILO and among key stakeholders and guide the project team in adjusting if necessary the project’s approach and scope for implementation during the project’s remaining period.

The mid-term evaluation will be conducted using the OECD/DAC criteria (relevance, coherence, effectiveness, efficiency, impact and sustainability). Specifically, the mid-term evaluation focused on the following aspects:

(1) Assessing the project’s design and activities, its implementation strategy and approaches, the challenges and how these challenges are addressed; and to what extent these contribute to the achievement of project objectives.

(2) Drawing key findings and lessons learned and proposing improvements and adjustments needed to ensure that the project remains relevant to the needs of the target groups and contributes to knowledge development.

The evaluation will comply with the ILO evaluation policy, which is based on the United Nations Evaluation Norms and Standards and the UNEG ethical guidelines.

Scope of the evaluation

The mid-term evaluation will cover the project from inception phase up until June and look at the project activities, outputs and outcomes in Lebanon. The evaluation will take into consideration the project duration, existing resources and political, security and environmental constraints.

The evaluation will take place tentatively from 19/06/2024 until 30/08/2024 through in-person (if the situation allows) and online field work to collect information from different stakeholders. The consultancy shall start with initial briefing with the project team, the Regional Office for Arab States (ROAS). The International Consultant will be working in-person or online and could be supported by a local consultant who supports with the field work; conducting FGDs, interviewing stakeholders, etc.

The evaluation will integrate gender equality and environmental concerns and just transition as a cross-cutting concern throughout its methodology and all deliverables, including the final report.

Clients of Evaluation

The primary clients of this evaluation are ILO ROAS and Lebanon team, and the ILO constituents, and the donor. Secondary users include other project stakeholders and units that may indirectly benefit from the knowledge generated by the evaluation. 

EVALUATION CRITERIA AND QUESTIONS 

The evaluation utilizes the standard OECD’s Development Assistance Committee (DAC) which has 6 main criteria:

• Relevance – the extent to which the objectives are aligned with national and local priorities and needs, the constituents’ priorities and needs, and the donor’s priorities for the country;
• Validity of design – the extent to which the project design, logic, strategy and elements are/remain valid vis-à-vis problems and needs;
• Efficiency - The extent to which the intervention delivers, or is likely to deliver, results in an economic and timely way
• Effectiveness - the extent to which the intervention achieved, or is expected to achieve, its objectives, and its results, including any differential results across groups;
• Impact - the extent to which the intervention has generated or is expected to generate significant positive or negative, intended or unintended, higher-level effects;
• Sustainability – the extent to which the net benefits of the intervention continue, or are likely to continue.

For the purpose of the mid-term evaluation, the focus will be on the following 3 criteria:

Relevance and Coherence:

1. Are the project's objectives aligned with the perceived needs of stakeholders, beneficiaries, etc. in both design and implementation?
2. Do the project's objectives and products complement or align with the ILO’s P&B (2024-25) and the UNSDCF (2023-25) for Lebanon, other initiatives by regional institutions, ILO programs, or development partners? How does the project integrate with the strategies and programs of other partners?
3. How does the project integrate gender considerations into its strategies and programs, aligning with the gender-related goals of the ILO?
4. What is the project's specific contribution, its added value, in relation to the ordinary activities carried out by the national actors concerned?
5. How do the identified conflict dynamics affected the planning, implementation and progress of the project?
6. How coherent is the project's logical framework (e.g. do outputs have a causal link with outcomes, which in turn contribute to the project's broader development objective? Is the results framework realistic?

Impact orientation and sustainability:

1. How is the project expected to influence the dynamics of the market system and its stakeholders?
2. What is the anticipated evolution from initial action research/pilot interventions to more advanced strategies, and what measures are in place to assess this progression during the initial stages of implementation?
3. To what extent are partners already expressing capability and willingness to engage in market systems development, and how is the project fostering early signs of ownership within the context? Does the project have an exit strategy in place, and how does the project ensure that the transition or conclusion of activities aligns with the sustainable engagement and ownership of partners?

Effectiveness and efficiency

1. Will the project be capable of achieving its desired outcomes and outputs?
2. What potential challenges or obstacles might hinder the effectiveness of the project?
3. How effective are resources, including human, financial, and technological, allocated for the project?
4. Considering challenges encountered during implementation, what strategies can be employed to address these challenges in the upcoming year?

METHODOLOGY

The evaluation will be carried out by a hired external consultant. The following is the proposed evaluation methodology. Any changes to the methodology should be discussed with and approved by the Evaluation Manager.

Desk Review

The evaluator will review project background materials before conducting any interviews. These include the project documents, the progress reports (if is available), the intervention plans, the value chain analyses and studies conducted, the TORs and background documents for the ongoing activities, and any other relevant documents including national and regional ones.

Briefing

The evaluator will have an initial consultation with the relevant ILO project staff and specialist in Lebanon and ROAS. The objective of the consultation is to reach a common understanding regarding the status of the project, the priority evaluation questions, available data sources and data collection instruments and an outline of the final evaluation report. The following topics will be covered: status of logistical arrangements, project background and materials, key evaluation questions and priorities, outline of the inception and final report.

Individual Interviews and/or Group Interviews

After the initial consultation with ROAS, and following the initiation briefing with project team and backstopping specialist and the desk review and the inception report, the evaluator will have meetings with stakeholders. Interviews will use a questionnaire designed to solicit feedback on opportunities and constraints to the delivery of project outcomes. The questionnaire will be developed by the evaluator in consultation with project staff.

A list of interviewees shall be developed by the project staff and provided to the evaluator. The evaluator will schedule the meetings.

Debriefing

Upon completion of the data collection & analysis, the evaluator will provide a debriefing to the Project team, ILO DWT and ROAS on the evaluation findings, conclusions and recommendations. The evaluator will also debrief stakeholders to validate results.

Evaluation Management

The evaluator will report to the ILO evaluation manager on technical and methodological matters.

MAIN DELIVERABLES

The main outputs of the evaluation consist of the following:

• Deliverable 1: Inception Report
• Deliverable 2: PowerPoint Presentation on preliminary findings
• Deliverable 3: Final evaluation report with executive summary and templates for lessons learned and good practices dully fille in (as per ILO’s standard procedure)

Inception Report

The evaluator will draft an Inception Report, which should describe, provide reflection and fine-tuning of the following issues:

• • Project background
  • Purpose, scope and beneficiaries of the evaluation
  • Evaluation criteria and questions
  • Methodology and instruments
  • Main deliverables
  • Management arrangements and work plan.

Final Report

The final version of the report will follow the format below and be in a range of 15-20 pages in length, excluding the annexes:

1. Title page
2. Table of Contents, including List of Appendices, Tables
3. List of Acronyms or Abbreviations
4. Executive Summary with key findings, conclusions and recommendations
5. Background and Project Description
6. Purpose of Evaluation
7. Evaluation Methodology and Evaluation Questions
8. Key evaluation findings (organized by evaluation criteria)
9. A table presenting the key results (i.e. figures and qualitative results) achieved per objective (expected and unexpected)
10. Clearly identified conclusions and recommendations
11. Lessons Learned (in prescribed template)
12. Potential good practices (in prescribed template) taking into account gender specificities
13. Annexes (list of interviews, TORs, list of documents consulted, list of meetings and interviews attended, etc.)

The quality of the report will be assessed against the EVAL Checklists 4, 5, and 6.

The deliverables will be submitted in the English language, and structured according to the templates provided by the ILO. 

MANAGEMENT ARRANGEMENTS

ROLES AND RESPONSIBILITIES

The Evaluator- is responsible for conducting the evaluation according to the terms of reference (ToR). He/she will:

• Review the TORs and provide input, propose any refinements to assessment questions, as necessary, during the inception phase;
• Review project background materials (e.g., project document, progress reports).
• Prepare an inception report;
• Develop and implement the evaluation methodology (i.e., conduct interviews, review documents) to answer the evaluation questions;
• Conduct preparatory consultations with the ILO EM and REO.
• Conduct field research, interviews, as appropriate, and collect information according to the suggested format, including leading on logistical arrangements;
• Present preliminary findings to the stakeholders; 
• Prepare an initial draft of the evaluation report with input from ILO specialists and constituents/stakeholders;
• Conduct a briefing on the findings, conclusions and recommendation of the evaluation to ILO project team and ROAS;
• Prepare the final report based on the ILO, constituents’ feedback obtained on the draft report.

The ILO Evaluation Manager is responsible for:

• Drafting the TORs;
• Finalizing the TORs with input from colleagues;
• Hiring the consultant;
• Providing the consultant with the project background materials in consultation with project staff, through uploading in specific space created for this purpose
• Participating in preparatory consultations (briefing);
• Assisting in the implementation of the assessment methodology, as appropriate (i.e., participate in meetings, review documents), in consultation with Project team;
• Reviewing the initial draft report, circulating it for comments and providing consolidated feedback to the External Evaluators (for the inception report and the final report);
• Reviewing the final draft of the report;
• Disseminating the final report to all the stakeholders – in consultation with ILO Lebanon;
• Coordinating follow-up as necessary.

The ILO REO:

• Provides support to the planning of the evaluation;
• Provide technical inputs as needed.

Duration of Contract and Timeline for Delivery

The collaboration between ILO and the Consultant is expected to starts as of 24 June 2024 (or upon signature) until 15 September. The estimated number of required working days for this assignment is as below:

Tasks

Number of Working days

Kick-off meeting and preparations for assignment

1

Desk review of documents related with projects

3

Drafting Inception report

3

Interviews

8

Drafting report

7

Briefing of preliminary findings

1

Integration of comments and finalization of the report

2

Estimated Total number of Working days

25

Supervision

The evaluator will work under the direct supervision of the Evaluation Manager on any technical and methodological matters related to this evaluation. The evaluator will be required to provide continuous updates on the progress of work and revert to the ILO with any challenges or bottlenecks for support. Coordination and follow-up with the evaluator will take place through e-mail or MS Teams or any other digital communication mean.

Payment terms

• 20 per cent of the total fee against deliverable 1 above approved by the evaluation manager.
• 80 per cent of the total fee against deliverables 2 and 3 above, approved by the ILO Regional Evaluation Officer.

LEGAL AND ETHICAL MATTERS  

• This evaluation will comply with ILO evaluation guidelines and UN Norms and Standards.
• The TORs is accompanied by the code of conduct for carrying out the evaluation “Code of conduct for evaluation in the ILO” (See attached documents). The selected consultant will sign the Code of Conduct form along with the contract.
• UNEG ethical guidelines will be followed throughout the evaluation.

REQUIREMENTS  

The evaluator should have:

• Master’s Degree in social sciences, economics, development studies, evaluation or related fields, with demonstrated strong research experience;
• Previous experience in evaluations for UN agencies is required. Evaluation experience with the ILO is an advantage;
• A minimum of 7 years’ experience in conducting projects and programme evaluations, with demonstrated experience in development related programmes.
• Proven expertise on evaluation methods, value chain and market systems development, enterprise development and the ILO approach;
• Experience in the evaluation of market systems development interventions, in the context of the humanitarian-development nexus is an advantage;
• Knowledge of the ILO and its normative mandate, tripartite structure and technical cooperation activities is an advantage
• Knowledge of Lebanon and the regional context is an advantage;
• Full command of the English language (spoken and written) will be required.