End of Project Evaluation

Background and Project Description

INITIATE is a non-governmental, non-political, and non-profit organization that works with communities in Lebanon, creating an environment in which women and youth live in dignity and develop joint acting platforms among communities in Lebanon.

With the financial assistance of the European Regional Development and Protection Programme (RDPPII) for Lebanon, Jordan and Iraq, INITIATE is implementing “1001 Nights Life Skills and Civic Education Program in Lebanon to Promote Social Cohesion” project in Lebanon. This edutainment project focuses on increasing positive civic values, decreasing negative civic values, decreasing the notion of vulnerability among children, and enhancing social cohesion. The project is implemented in formal and non-formal education settings in cooperation with the Ministry of Education and Higher Education (MEHE), Catholic Schools Network, and 105 safe spaces across Lebanon. The Program targets 60,000 learners with Lebanese, Palestinian and Syrian backgrounds.  

The program is based upon a very popular and award-winning animated television series called 1001 Nights, which airs as commercial entertainment in over 70 countries to a global audience of over 100 million people in over 25 languages on premier television networks, including Disney, Cartoon Network, Al Jazeera Children’s Channel and Discovery Kids. In 50 countries, including all those in the MENA, it is one of the three most popular family programs on television, reaching up to 25% of the population when it airs. While enormously entertaining, each episode of the series was developed to teach a different civic value, democratic right or concept of the rule of law, with notions of empathy, inclusion and gender equality running through the entire program.

Working with key stakeholders including MEHE/CERD and project partners BBB, TFL, and CSN, the overall objective the project seeks to achieve is Children are agents of change within communities that champion pluralism and promote social justice with the support of MEHE, NGOs and other actors that promote social stability and cohesion.

The intended outcomes are as follows: (1) Children in formal settings are less vulnerable through increased civic values (including gender equality, inclusion, non-violence, the rule of law and civil rights) and critical thinking skills; (2) Improved social cohesion and reduced tension in informal settings within communities that appreciate inclusivity, civic education and human rights (including notions of gender equality); (3) Local NGO has increased capacity to support programming that promotes social cohesion and protection.

Purpose of the evaluation

Scope and focus of the Evaluation

The final evaluation will assess the program’s achievement of objectives lessons gained from its implementation that could inform future programming. This will include assessing impact, relevance, effectiveness, efficiency and sustainability of the project’s observed results under each objective. The specific objectives of the final evaluation are to assess the achievement of observed results, following the impact and outcomes change pathways, measuring success against the planned objectives and related indicators at these levels.

Ultimate Objective: Children are agents of change within communities that champion pluralism and promote social justice with the support of MEHE, NGOs and other actors that promote social stability and cohesion.

Indicator 1: Reduction in-perceived number of disputes between refugee and local communities

Indicator 2: Reduction in perceived sectarian, religious and social tension in vulnerable communities

Indicator 3: Percent of targeted community members (Family members of children) reporting increased positive perception toward other groups in their community.

Indicator 4: Number of community members (Family members of children) finding support useful.

Indicator 5: Percent reduction in negative civic values amongst children (based on attitudinal and behavioral changes)

Indicator 6: Percent reduction in children's acceptability of gender-based violence, exploitation and inequality (attitudinal change)

Outcome 1: Children in formal settings are less vulnerable through increased civic values (including gender equality, inclusion, non-violence, the rule of law and civil rights) and critical thinking skills);

Indicator 1: Percent reduction in negative civic values amongst children attending formal education (attitudinal)

Indicator 2: Percent decrease in children attending formal education deemed to be vulnerable

Outcome 2: Improved social cohesion and reduced tension in informal settings within communities that appreciate inclusivity, civic education and human rights (including notions of gender equality)

                Indicator 1: Percent reduction in negative civic values amongst children in non-formal environments (attitudinal)

                    Indicator 2: Percent decrease in children in non-formal environments deemed to be vulnerable

Outcome 3: Local NGO has increased capacity to support programming that promotes social cohesion and protection

Indicator 1: Capacity building plan developed and implemented

Indicator 2: Development of a referral system

Evaluation Questions

The following are guiding questions for the evaluation and can be expanded upon and/or altered by the selected candidate.

Impact:

• To what extent has the project reduced vulnerability of children in formal and informal educational settings through increased civic values (including gender equality, inclusion, non-violence, the rule of law and civil rights) and critical thinking skills? Describe why or why not.
• To what extent has the project improved social cohesion and reduced tension/increased notions of security within communities (including notions of gender equality)? Describe why or why not.
• How were local NGO (INITIATE) capacities strengthened, at the individual organizational level, to achieve the project outcomes - including contributing factors and constraints?
• What change was observed amongst learners based on defined civic values, internal health, and/or coping strategies to be measured? In what ways did the change in values, internal health and/or coping strategies occur?

• To what extent was there a difference in the achievement of objectives and impact between the formal and informal learning environments, and between male/female/refugee/host community learners? What factors led to these differences?

Effectiveness:

• What were the factors leading to effective implementation? Why?
• What were the lessons learned?
• Were there any unintended consequences of the intervention?
• Which outcome pathways have contributed the most and to contribution at the outcome/impact level?

Efficiency:

• How was the project efficient in delivering desired results?
• Were there more efficient ways and means of delivering more and better results (outcomes) with the available resources? Any examples?
• Where there any constraints to more cost-efficient ways?
• What prevented the team from adopting those ways?
• Could a different approach have produced better results?
• How well were resources used? Cost analysis.

Relevance:

• How were the organizational capacity building components (outcome 3) aligned with INITIATE’s organizational needs?
• To what extent have beneficiaries been satisfied with the results?

Sustainability

• What is the sustainability of project’s outcomes and benefits after completion of the project? In what ways?

Process and Duration for Evaluation

The following provides a guide on the evaluation process and timelines expected for this assessment. The final methodology will be proposed by the selected evaluator in an inception report and presented to the reference group overseeing the assessment for their endorsement.

• Desk review of relevant project documents – Start on November 14, 2022
• Inception report detailing proposal evaluation methodology, including qualitative and quantitative measures, resources and support required (e.g. local enumerators), and timeline  - November 25, 2022
• In-country data collection completed – December 15, 2022
• Draft Final Evaluation Report – February 3, 2023
• Preliminary findings session – Mid February 2023 (exact date TBD)
• Final Evaluation Report (submitted in English) – February 28, 2023

It is expected the evaluation will use both qualitative and quantitative tools (including the existing project pre/post surveys) to collect data and feedback from a range of key stakeholders. This will likely include: survey with beneficiaries; key informant interviews; focus group discussions; and, case studies. The opinions of the beneficiaries, neighborhood committees, local authorities, informal leaders, and INITIATE staff members, should be captured, analyzed, triangulated, and reflected in the report using the data visualization tools whenever it possible.

Evaluator Skills and Qualifications  

The selected candidate, whether representatives of an agency or working independently, will be evaluated according to the following criteria:

1. A relevant post-graduate degree from an accredited academic institution, or the professional equivalent. Existing linkages to a post-secondary institution are an asset
2. Minimum 7 years of experience in international project monitoring and evaluation including significant field-based experience
3. Demonstrated experience in working with local non-governmental organizations
4. Demonstrated knowledge of international, regional and local contexts, particularly related to humanitarian response strategic initiatives 
5. Strong qualitative analysis skills
6. Experience working in education, child safeguarding, and/or community-based initiatives in an international context
7. Strong oral and written communication in English