Consultancy for conducting an external evaluation

TERMS OF REFERENCE

End of project evaluation under beirut response

Plan International Lebanon

February , 2022

 

1. Background

Beirut Blast  

On the 4^th of August, a massive blast took place in the port of Beirut, Lebanon.  According to the Ministry of Public Health (MoPH), more than 191 people were killed, and 6,500 people injured.  A total of 200,000 housing units were affected in Beirut, impacting around 300,000 people, many who lost their homes and were forced to relocate. An estimated 40,000 buildings, and over 15,000 businesses in Beirut were damaged, around 10,000 in the direct vicinity of the blast destroyed leaving over 70,000 people unemployed and food insecure. According to the World bank needs assessment, damage is reported to 292 health facilities, including private and public hospitals, in addition to primary healthcare centres rendering facilities partially or fully inoperable. An Emergency Medical Teams (EMT) was deployed to Beirut to support the ongoing medical response, as pressure on both public and private hospitals is at an all-level high, with the number of new Covid-19 cases surging rapidly.  Widespread multisector humanitarian needs were identified in affected areas, notably in terms of shelter, food, reconstruction, livelihoods and psychological support.

 

Children specifically are among the most vulnerable groups affected by the blast. Girls’ and boys’ well-being was significantly affected as they suffer from distress that can impact them on the medium and long-term following such a traumatic experience, therefore, the need for psychosocial support was high. Also, as many were displaced from their homes they are at risk of violence, abuse and exploitation. Moreover, according to UNICEF, up to 100,000 children have had their homes damaged and were displaced in Beirut, while 159 schools serving nearly 85,000 children were in various states of damage.  

The World Bank has estimated physical damages at $4.6 billion, and economic losses at $3.5 billion. The blast happened at a time when Lebanon was already facing a severe economic and socio-political crisis, limiting access to livelihoods opportunities and basic services. Lebanon is dependent on imports to satisfy around 85 percent of its food needs. The country was already in a precarious food security situation, as inflation led to skyrocket increase in the cost of goods throughout 2020, and importers have struggled to secure enough dollars in the cash-strapped country to pay for their imports. The devaluation of the Lebanese Pound was up to 80% at the time, and depositors cannot access their deposits.  

Plan International & Giro555

Plan International is an independent development and humanitarian organisation that advances children’s rights and equality for girls. We believe in the power and potential of every child, but know this is often suppressed by poverty, violence, exclusion, and discrimination. Working together with children, young people, supporters and partners, we strive for a just world, tackling the root causes of the challenges girls and vulnerable children face. We support children’s rights from birth until they reach adulthood and enable children to prepare for – and respond to – crises and adversity. We place a specific focus on girls and young women, who are most often left behind. We have been building powerful partnerships for children for more than 80 years, and are now active in more than 70 countries.

Plan International is operating in the Middles East Region: In Lebanon, since 2017, obtaining formal registration in 2019, and has programmes throughout the country. Plan International Lebanon (PIL) works in partnership with local, national and international organizations to respond to the different crises facing the country and support vulnerable Lebanese and refugee children to learn, lead, decide and thrive. Plan has programmes throughout the country (Beirut, Mount-Lebanon, South, North, and Beqaa) and works in sectors of Child Protection (CP) and Gender-based violence (GBV), Sexual and Reproductive Health and Rights (SRHR), Education, Youth economic empowerment and participation.  In addition to its development work, Plan has responded to the Syrian refugee crisis and the COVID-19 crisis, Beirut blast and the on-going economic crisis in Lebanon.

In August 2020, as a response to the devastating Beirut explosion, Giro555 launched a national action to respond to the crisis with Plan International as the chair of that action. Giro555 is a member of the Emergency Appeals Alliance (EAA), an international network of ten National Actions in different countries. The foundation in which the aid organizations behind Giro555 have united is the Stichting Samenwerkende Hulporganisaties ("SHO"). When an exceptional disaster happens, the 11 cooperating aid organizations join forces under the name Giro555. Giro555 works together with Dutch broadcasters, broadcasters and other media, and asks the whole of the Netherlands to join.

Under the Beirut Blast response specifically, Plan International Lebanon and Himaya has signed a partnership agreement on the 26th of August 2020, to respond to this Emergency by improving psychosocial well-being and positive coping mechanisms of girls and boys aged 6-17 affected by the blast and support them in having access to quality and safe education. Throughout this project Himaya conducted: 

1. Child Protection and PSS support to improve psychosocial wellbeing and positive coping mechanisms of girls and boys affected by the blast 
2. Education support to improve access to quality and safe education.

Adding to that, and aside of the program implementation, Plan Lebanon provided Himaya with all the needed support and mentoring throughout the projects’ period. Constant coordination was established through visits, meetings and on-going communication with the project’s deliverables.

Project background

Goal of the project:

The project entitled the provision of response to the Beirut Blast Emergency by improving psychosocial well-being and positive coping mechanisms for girls and boys aged 6-17 years who were affected by the explosion in August 2020, and support them through providing access to quality and safe education.

The project’s activities were implemented by partner Himaya, in several locations; for the focused-psychosocial support and awareness raising sessions, the implementation took place in Achrafieh-Rmeil, and Beshoura through live and remote modalities.

As for the educational activities, the implementation for the activities took place in open spaces (Kasskas, Karem el Aris, and Geitawi parks) and in closed spaces (St. Joseph school – Cornish el Naher, Al Sabil Library, and Erchad w Eslah NGO, Scout Islamic Ras El Nabeh).

Outcomes of the project:

Outcome 1: Girls and boys aged 6-17 affected by the disaster have improved psychosocial well-being and positive coping mechanisms.

Indicators:

• % of identified cases that receive quality child protection case management response. 
• % of affected girls and boys who report an increased sense of safety and well-being after engaging in the project. 

Activities:

• Provision of group-based focused psychosocial support activities to affected girls and boys. Target achieved: 408 (226 F) 
• Distributing psychosocial support kits to affected girls and boys. Target achieved: 200 (110 F)
• Providing individual case management services to child survivors and/or at-risk girls and boys including referrals to multi-sectoral services to girls and boys. Target achieved: 162 (68 F)
• Providing key messages and awareness sessions to 250 parents and caregivers of affected children on how to help children cope with distress. Target achieved: 361 (257 F)

 

 

 

Outcome 2:  Affected girls and boys have access to quality and safe education.

Indicator:

• % of targeted girls and boys who have access to education 

Activities:

• Provision of educational kits with stationary supplies to affected girls and boys. Initial Target Achieved: 286 (150 F). New additional Target: 460 (220)
• Provision of laptops for affected families and information leaflets on remote learning, for online /remote learning. Target: 735 

The full result framework including targets will be shared with the recruited consultant.

1. Goal, objectives and the key questions

Purpose of the evaluation

As the of the SHO Beirut Response is about to close, a final evaluation is planned. This purpose of the final evaluation is summarized below:

• Accountability to the donor, in this case the Dutch Public
• Validating the outcome of the project and deriving lessons learnt and best practices from the project’s implementation, as per improving future humanitarian interventions by Plan International and other actors in humanitarian sector, which ensures continuous learning.
• Ensuring sustainability at its best modalities through identification of gaps and suggesting future interventions as a response.

Evaluation scope

This end of project summative evaluation will cover the implementation period of fourteen month from 26 August 2020 until December 2021 and focus on the two intervention areas of Child Protection & PSS and Education support.

Evaluation Criteria

This evaluation will focus on the following core humanitarian standards:

• Appropriateness & relevance (CHS1)
• Effectiveness & timeliness (CHS2)
• Collaboration and capacity Strengthening of local actors and avoidance of negative effects (CHS3)
• Communication, participation and feedback (CHS4)
• Coordination with and complementarity to other actors (CHS6)

The above evaluation criteria closely link to the DAC evaluation criteria and the underlying questions as per the below:  

• Relevance:  The extent to which the objectives of a development intervention are consistent with beneficiaries’ requirements, country needs, global priorities and partners’ and donors’ policies (aligned to CHS1)

• To what extent was the project relevant to all beneficiaries (ie gender and age specific)?
• To what extent was the project able to reach the most vulnerable?
• To what extent were the chosen project interventions appropriate seen the circumstances? 
• How did the needs assessment inform and affect the intervention?

• How consistent is the project with the Plan International Lebanon, policies principles and strategy (CSP/regional strategy)?
• To what extent have beneficiaries participated in the design, implementation and monitoring of the project?

 

• Effectiveness: The extent to which the development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance (Aligned to CHS2, CHS3 & CHS4).

• To which extent were each of the projects’ objectives achieved?
• How properly was the project executed in terms of quality and timeliness?
• What internal and external factors affected the speed of the response in the start-up phase, including the contingency (preparedness) plan, procurement, HR, financial management systems, pre-positioning of relief items, staff recruitment, staff deployment systems, general external factors?

 

• Coherence: The compatibility of the intervention with other interventions in a country, sector or institution. (aligned to CHS6)

• To what extent has coordination with other humanitarian actors taken place?
• To what extent was Plan’s response complementary to other actors?
• To what extend has the capacity of local actors been strengthened for future humanitarian relief interventions?
• To what extent has working with local actors affected to effectiveness of the Programme?

 

• Impact: Impact looks at the wider effects – positive and negative- of the project - social, economic, technical, environmental - on individuals, gender, age-groups, communities, institutions and their contribution to the disaster – development continuum (aligned to CHS3)

• Which positive/negative and intended/unintended effects can be observed in the project? How do internal and external factors contribute to impact?
• How has the project implemented the “do no harm” principle and avoided negative effects?
• To what extent the community benefited from implemented activities?

 

• Efficiency: The Extent to which financial and human resources were used economically and efficiently.

• The extent to which financial resources were used economically and efficiently, potentially including cost-benefit ratios and alternative programming approaches

 

• Sustainability: To measure whether an activity or an impact is likely to continue after donor funding has been withdrawn.

• Are structures, resources, and processes in place to ensure that benefits generated by the project continue once external support ceases?
• Do the project partners have the technical and financial capacity, and are they committed to maintaining the benefits of the project in the long run?
• What (if any) components and results of the intervention contribute to longer-term development and how?

The crosscutting themes of Gender and Gender equality should be considered for every research question. (aligned to CHS1)

• What did the project achieve in terms of protecting the safety, dignity and rights of affected people, promoting gender equality and addressing barriers to inclusion, including for people with disabilities?
• Were the project activities designed and tailored to meet the needs of different groups of people including age, gender, disability…?

End users of the evaluation

The evaluation will serve the following end users:

• The Dutch Public will, through the evaluation, be informed about the use of the SHO funding and the results achieved with the SHO funding
• Plan International Netherlands and Plan International Lebanon will learn about their performance in humanitarian interventions
• Other actors in the humanitarian sector will also benefit from the lessons learned, as these will be shared amongst partners both in the Netherlands as well as in Lebanon

 

1. Methodology

The evaluation is expected to use a mixed method approach including both quantitative and qualitative approaches. Further to that, below are the four main principles that the evaluation should be built on:

1. Inclusiveness—the methodology should include a wide range of viewpoints, specifically gender and age-sensitivity when applicable.

The consultant/consultancy firm will participate in “project orientation” sessions with relevant Plan and Himaya staff, as part of the development of the Inception Report.  The purpose of the sessions will be to ensure the consultant understands the assignment, the project and how it has evolved.

1. Mixed-method approaches—both qualitative and quantitative methods need to be present in the methodology.
2. Rigor of evidence—gathered information needs to be reliable and transparent
3. Ethics—the methodology needs to consider research ethics and the vulnerability of the target population in order to ensure that the evaluation is fully objective.

The following methods are proposed but the consultant can suggest other or additional methods if applicable.

3.1. Desk Review

A desk review will be conducted by the consultant/s, focusing on available project data (including monitoring data and data from the project’s mid-term review) and external literature.  The desk review will inform the methodology to be used in the evaluation and model development – it will focus on identifying gaps in available data about the project, understanding the project context and identifying and investigating comparable programs.

3.2. Primary Data Collection

Primary data collection for the evaluative component of this assignment will focus on outcome-level changes, using the mixed methods approach, including both quantitative and qualitative data collection methods (Key Informant Interviews and Focus Group Discussions). The consultant will be expected to triangulate data sources to build a robust evidence base around the program’s effectiveness, relevance, impact coherence, sustainability and efficiency and identify ways to improve the program model.

Methods may include:

• Qualitative Methods:

Key Informant Interviews (KIIs) and Focus group Discussions (FGDs) with staff and representatives of key stakeholders including cooperating partner, organizations, communities and individuals (Beneficiaries, teachers and schools’ principles) to gather in-depth information on key questions.

 

• Quantitative Methods:

Surveys: Surveys should be administered to project beneficiaries including boys, girls and others as appropriate, to collect quantitative information related to the study’s main purposes. The consultant will be provided with the list of respondents under the project activities. 

3.3. Sampling

The sample size should be adequate and representative of each target group. Following the sampling methodologies for the quantitative surveys and a clear and logical number for the FGDs and KIIs based on selection criteria.

 

The development of the methodology and tools and the data collection is the responsibility of the consultant, however Plan International will oversee and follow up on the data collection process and the project partners will help in facilitating the communication between them and project stakeholders.

1. Ethics & Child safeguarding

Plan International is committed to ensuring that the rights of those participating in data collection or analysis are respected and protected, in accordance with the Framework for Ethical MERL and our Global Policy on Safeguarding Children and Young People. All applicants should include details in their proposal on how they will ensure ethics and child safeguarding in the data collection process. Specifically, the consultant/s shall explain how appropriate, safe, non-discriminatory participation of all stakeholders will be ensured and how special attention will be paid to the needs of children and other vulnerable groups. The consultant/s shall also explain how confidentiality and anonymity of participants will be guaranteed.

Additionally, all parties involved in the data collection will receive an induction on Plan International’s Safeguarding Children and Young People Policy, referral pathways, Gender Equality and Inclusion Policy and Non-Staff Code of Conduct.

1. Deliverables

The consultant is expected to produce and submit the following deliverables:

• Inception Report (Word Document): Within ten days from signing the contract. The inception report is expected to clearly defines the evaluation methodologies (qualitative and quantitative) with details of sampling (method and sample size), target groups, and evaluation timeline with specific deadlines for each deliverable.
• Final data collection Tools (Word/Excel & KOBO link/any other mobile application): The consultant will be responsible for developing, coding, and digitalizing the data collection tools.
• Cleaned raw data in excel spreadsheets, and FGD and KII transcripts in local language and in English (Excel): All handwritten and electronic transcripts of interviews and KIIs, hard copies of survey questionnaires, photographs taken during the assessment and any equipment received from Plan for the purpose of the study should be submitted to Plan. Furthermore, all information generated during the evaluation will be the sole property of Plan and is subject to submission to Plan, along with the final report, prior to the end of the contract.
• Draft Evaluation Report (Word Document): Draft Evaluation report to be submitted within 10 days of the completion of the data collection. Phase for review and comments from Plan team in Netherlands and Lebanon. The review and feedback of the report could be more than one round depending on the quality of the report and the extent to which the comments and suggestions from first round have been incorporated.
• Validation workshop (Agenda & Power Point Presentation): A validation workshop to present results and agree on recommendations, prior to finalizing the report. (Presentation of key findings and lessons learned needs to be submitted).
• Final Evaluation Report including Executive Summary (Word Document & PDF): A final Evaluation report to be submitted after incorporating the comments of Plan International. The report should be written in English. The consultant is responsible for English editing and proofreading of the final report, which should be well formatted. The report will be credited to the evaluator and potentially placed in the public domain at the decision of Plan International.

The report should include evidence, lessons learned and recommendations.

1. Indicative Timeline

• Interviews, Background Checks and Contracting: February 28 – March 4
• Inception call/workshop: March 11
• Submission of inception report (including tools): March 21
• Revision of Inception Report and Feedback: March 24
• Preparations for data collection (including field piloting and enumerator training: March 25-30
• Data collection: March 31-April 8
• Submission of draft Report: April 18
• Revision of the draft report and feedback: April 21
• Validation of findings with key stakeholders and respondents & feedback incorporation: April 26
• Submission of Final Report: April 29

1. Selection criteria for evaluators

In view of the purpose, scope and focus of the end evaluation, Plan seeks to hire an external institution or consultant with experience in evaluation of humanitarian programs

Qualifications of the lead consultant:

• Minimum Master’s Degree in social studies, International Development or any other related field
• Minimum 7 years’ experiences in undertaking research, baseline and evaluations in Lebanon or similar contexts.
• Relevant experiences in the response to humanitarian crisis, preferably in the areas of protection and gender (gender transformative programmes).
• Demonstrated experience of facilitating research with participatory methods and tools, particularly with children and young people
• Competency in managing, organizing and interpreting quantitative and qualitative data and information
• Experience working with vulnerable groups, including refugees and children
• Report writing skills.
• Excellent written and verbal communication skills in Arabic and English.
• Ability to communicate effectively to a range of different stakeholder

 

1. Indicative budget

The budget is expected to be submitted as part of the application. The consultant shall be responsible for all taxes and applicable governmental fees. Additionally, the consultant should mention in their financial proposal all travel costs, insurance and visa costs, per diems, and translation costs.