External Evaluation Consultant: Education Program in Syria

1- Introduction and background

The Jesuit Refugee Service (JRS) is an international Catholic organization with a mission to accompany, serve and advocate on behalf of refugees and other forcibly displaced persons. Misereor is the German Catholic aid organization for development cooperation. Together with its local partners, Misereor supports human beings of every faith and culture. The overall goal in its work with partners in countries of the global south is to contribute to sustainable development by promoting projects and programs that are directed above all towards the poor. Financial support for these projects is made available by the Federal Ministry for Economic Cooperation and Development (BMZ) through the German Catholic Central Agency for Development Aid (KZE) and by private donors. This evaluation has been initiated by Misereor as the evaluation of the following project is mandatory according to agreements with the Federal Ministry for Economic Cooperation and Development (BMZ).

Goal: Jesuit Refugee Service’s (JRS) holistic education work strengthens child protection and contributes to the well-being and resilience of vulnerable populations (displaced and host communities) in the catchment area of JRS centers in Jaramana, Al-Kafroun, Bab-Sbaa (Homs) and Al-Sakhour (Aleppo).

Objective no.1: Access of 3,750 vulnerable children to quality educational services in the safe spaces of JRS community centers in Jaramana, Al-Kafroun, Bab Sbaa, and Al-Sakhour is ensured.

Indicators:

1.1: By the end of the funding period, 75% of girls and 75% of boys (disaggregated by centers, programs and age) have maintained attendance throughout the school year. (Baseline: 115)

1.2 By the end of the funding period, 75% of children (disaggregated by gender, centers, programs and age) participating in the education program have improved their academic performance.

1.3 By the end of the funding period, 100% of the tutors participated in the JRS training program (e.g. on protective measures, child rearing, case management).

1.4 By the end of the funding period, 80% of the training participants confirm that their attitudes, knowledge and skills in work practice and psychosocial intervention have improved.

Objective no.2: The resilience and psychosocial well-being of 3,750 vulnerable children reached by the project are strengthened.

Indicators:

2.1 By the end of the funding period, 3.750 girls and boys (disaggregated by centers, gender and age) have participated in structured group activities for psychosocial support (including social activities, daily living skills).

2.2 By the end of the funding period, 75% of children (disaggregated by centres, gender and age) report that their psychosocial well-being has improved after participating in the programs.

2.3 By the end of the funding period, 350 girls and boys (disaggregated by centers, gender and age) are supported through individual psychosocial interventions.

2.4 By the end of the funding period, 75% of guardians have participated in awareness-raising events on parenting:

The activities that are provided in the program are:

- Teaching the children is divided by levels.

- Social and emotional learning.

- Case management.

- Individual PSS sessions.

- Daily meal.

- Activities such as outings, summer activities, recreational activities.

- Distributions (food basket – winter kit – hygiene kit -school kit).

- Parenting sessions.

2- Objectives of the evaluation

The main goal of this evaluation is to answer our questions on how the program’s effectiveness can be enhanced. The aim is to obtain an external and independent assessment of the projects. Along with the OECD/DAC criteria for evaluation (relevance, coherence, effectiveness, outcomes and impacts, efficiency, sustainability), it will examine what has been achieved by the projects and in which areas adjustments or a conceptual revision/reorientation might be necessary. These findings and recommendations will provide evidence-based recommendations to inform the next program planning and design. The evaluation result and findings will be shared with JRS staff and beneficiaries, regional JRS team, MIsereor and other relevant stakeholders.

3- Questions to be answered by the evaluation

When drawing up these questions, the DAC criteria for evaluations have been considered: relevance, coherence, effectiveness, efficiency, impact, and sustainability.

a. Relevance: (The extent to which project objectives and design respond to the needs, priorities and policies of the target groups and the organization responsible for the project and its partner organizations and continue to do so if circumstances change.

o To what extent was the project/ program design (in all its components) relevant to the priority needs of the targeted population?

▪ To what extent was the project relevant to these needs in relation to the specific context (needs and barriers) and location?

▪ To what extent is the timeline of intervention per child (two years) aligned with the identified needs?

▪ To what extent are our current distributions relevant to the needs and overall goal and objectives of the project?

▪ To what extent are our programs approaches designed to reach the most vulnerable children (attendance and selection criteria)?

o How participatory is our approach in involving beneficiaries in project design, implementation, and evaluation? How can we further enhance our approach?

b. Coherence:

o How coherent is the logical chain, causal relationship between the project's activities, outputs, outcomes? How well connected are the project components and their contributions to the overall goal?

o To what extent did the project align with the broader sector objectives on a national level? o How effective was JRS's coordination with other actors in ensuring service compatibility and avoiding overlap?

o How effectively do social workers, case workers, teachers and other education and MHPSS staff collaborate inside the project to ensure a cohesive support system for students within the project?

c. Effectiveness: The extent to which a project achieved, or is expected to achieve, its objectives (as laid out in the Project Contract) and outputs, including differential results across target groups.

o To what extent were our program objectives and outcomes, specific, measurable, achievable, relevant, and time-bound?

o To what extent do our indicators and MEAL system accurately measure project performance in achieving its objectives and results and its use for learning and decision-making?

▪ How effective are our technical monitoring tools (placement tests, individuals' evaluation in selecting the children, assessing their progress, and determining their future support needs?

▪ How effective are our technical MEAL tools (guidelines, MEAL plans, reports) in tracking and measuring the intended results?

o To what extent does our internal and external referral mechanism effectively contribute to addressing needs?

o What is the effectiveness of the methodology of our intervention (timeline, number of sessions per subject), given our intervention approach?

▪ To what extent is the current intervention for Retention2 beneficiaries effective in addressing the needs of these students, in terms of timeline and schedule?

o To what extent are the components supporting the education intervention (distributions, parenting sessions, MHPPS interventions) effective in supporting the overall objective of the education program (providing the children who are most in need with educational support)

o To what extent is the recruitment process of teachers and education personnel effective in relevance to the context in Syria?

d. Efficiency: how well are resources being used? The extent to which the project delivers or is likely to deliver results in an economic and timely way.

o How efficiently are we allocating resources to ensure that our educational services reach the maximum number of children while maintaining high quality and minimizing costs?

e. Impacts: what difference does the intervention make? The positive and negative changes produced by a project at a higher level. The evaluation should focus on both intended and unintended outcomes and impacts.

o What is the impact of the education intervention on the local community, considering the area-based approach?

o Has our education intervention contributed to the overall well-being and prospects of the children in the target community?

o Is there any unintended negative or positive impact regarding our intervention in the area?

f. Sustainability: will the benefits last? The extent to which the net benefits of the project continue or are likely to continue. Benefits are intended to be socially, environmentally, economically, and technologically sustainable. The review is also intended to include institutional aspects.

o Are there plans in place to ensure that the project's outcomes are maintained over time, even in the face of changing circumstances?

o What are the potential challenges and opportunities for maintaining longer-term results or/and scaling up the project?

4- Methodology

A set of varied and adapted methods is to be applied (taking a gender-sensitive approach) that focuses on quantitative and qualitative aspects in the following steps of the evaluation:

• Prior to fieldwork: Final revision of ToR with the JRS team, document review, preparation of interview guidelines, assessment of the available regional and technical analyses and data, preparation of data collection tools, kick-off workshop,

• During fieldwork: participant observation, quantitative survey, qualitative interviews (open, semi-structured), focus group discussions, context analyses, interviews with key persons, debriefing workshop ...

• After the fieldwork: report writing, commenting on the report by JRS and Misereor, debriefing with JRS and Misereor.

5. Organisation of the mission

The evaluation shall be carried out by the selected consultant, JRS and Misereor will provide the evaluation team with all necessary information and documents to carry out the evaluation. During the fieldwork, JRS will provide logistical support.

The tentative schedule of the evaluation is:

o Kick-off and inception: Starting Week of November 25th until December 18, 2024

o Fieldwork: January 6, 2025-January 24, 2025

o Submission of initial findings and the first draft report: February 14th, 2025

o Final Report: March 6, 2025

6. Report

Evaluation reports submitted to MISEREOR should meet several requirements. Some are binding; others can be adapted to the corresponding situation. These requirements are listed in the document “Minimum requirements to be met by evaluation reports for projects funded by MISEREOR/German Catholic Agency for Development Aid (KZE)”. The Document will be shared as part of the desk review.

7. Requirements

Applicants must meet the qualifications detailed below.

a. Advanced degree in education, Research or other relevant field, or the equivalent combination of work and education experience in a related area.

b. A minimum of 7 years of experience conducting field-based evaluations using mixed methods in a humanitarian context.

c. Experience conducting evaluations of Education projects is required.

d. Experience working in Syria is required.

e. Must possess strong analytical skills.

f. Strong cultural sensitivity.

g. Fluency in English and Arabic (reading, writing, and speaking) is required.