Evaluation and program model development for IMC’s and Plan International’s Protection Activities in Lebanon

Terms of reference

Evaluation and program model development for IMC’s and Plan International’s Protection Activities in Lebanon

Please refer to the Document attached for the full TOR, including the timeline.

1. Background
   1. Context

Now entering its ninth year, the conflict in Syria continues to have a drastic impact on Syrian refugee children as well as Lebanese children living in host communities. Children continue to experience different forms of violence, abuse and exploitation in their communities. Increasing levels of poverty and lack of access to quality education among families have put an alarming number of girls and boys at risk of child labour and child marriage. Adolescents and youth have limited educational and economic opportunities, which causes them to experience a sense of hopelessness and lack a future perspective. To address these risks and prevent the long-term harmful consequences on an entire generation of children, Plan International Lebanon implements multi-sectoral Child Protection, Gender-Based Violence (GBV), Sexual and Reproductive Health and Rights (SRHR), Education, and Economic Empowerment programming aimed to empower vulnerable adolescents and young people, in particular girls and young women.

1. 1. The organisation

Plan International is an independent development and humanitarian organisation that advances children’s rights and equality for girls. We believe in the power and potential of every child, but know this is often suppressed by poverty, violence, exclusion, and discrimination. Working together with children, young people, supporters and partners, we strive for a just world, tackling the root causes of the challenges girls and vulnerable children face. We support children’s rights from birth until they reach adulthood and enable children to prepare for – and respond to – crises and adversity. We place a specific focus on girls and young women, who are most often left behind. We have been building powerful partnerships for children for more than 80 years, and are now active in more than 70 countries. Plan International has had a presence in Lebanon since 2017, obtaining formal registration in 2019, and has programmes throughout the country (Beirut, Mount-Lebanon, South, North, and Bekaa).

1. 1. Project description

Plan International has a partnership with International Medical Corps (IMC) since June 2017 to implement a 3-year GBV program across Lebanon. The program aims to prevent and respond to GBV, in close coordination with other departments, agencies, and in partnership with MOSA, local and international NGOs in support of the most vulnerable populations. It is well known that a crucial component to the improvement of the GBV intervention is to provide safe confidential spaces for community members, with particular attention to women and girls who are disproportionally affected by GBV, therefore, IMC opened the community centers and, woman and girls’ safe spaces (WGSS) within each of its geographical hubs of intervention. The community centers aim to engage woman and men, girls and boys within GBV activities, while the WGSSs serve as a safe haven for women and girls to expand their support networks and accessing services, also for GBV survivors, while having a safe, non-discriminatory space to express themselves, their needs and emotions and break the isolation that often women face. .

In response to this, a team of specialized social workers and case managers, are providing the beneficiaries with psychosocial support (PSS) through focused and non-focused PSS activities including awareness raising, focus group discussions, emotional support groups, and livelihood activities using standardized PSS toolkits. Additionally, the teams provide individual GBV case management as per the Interagency GBV Case Management Guidelines (2017), espousing a survivor-centered approach. The services are provided for women and girls as well as men and boys through Stress and Anger Management and masculinities workshops directed towards men, Youth Empowerment activities, Program RA since the engagement of the male population in ending violence against females has proven to be as important as the activities targeting women and girls. At the national level, IMC is a member of the GBV Information Management System (GBVIMS) that is the main reference for all GBV cases data sharing in Lebanon and is a global advocacy tool. On the other hand, IMC is co-chairing the National Technical Taskforce (NTTF) on GBV alongside Abaad, their local partner and the Ministry of Social Affairs.

IMC is implementing the activities through appropriate centers across the country, located in North Lebanon, Beirut, and Mount Lebanon, and the South. Plan is responsible for managing the contract and communication with the donor, and monitoring the implementation and quality of the action.

1. 1. Project objective and outcomes

The overall objective of the project is to strengthen protection mechanisms through improved gender-based violence prevention and response services for refugee and host communities in Lebanon affected by the Syria crisis.

The project seeks to achieve the following outcomes:

1. Improved well-being for Syrian refugees and vulnerable Lebanese population participating in GBV case management services and psycho-social activities
2. Improved capacity of frontline workers to provide quality protection and health services.

The following outcome-level indicators have been identified for the project:

• Percentage of beneficiaries receiving GBV case management for at least three months that achieve at least 80% of their action plan objectives at the end of the project cycle
• Percentage of beneficiaries participating in YEP activities show improvement in their well-being throughout the project period
• Percentage of GBV frontline workers demonstrate increased capacity to facilitate stress and anger management sessions
• Percentage of CMR trained facilities demonstrate increased capacity to support survivors of rape
  1. Project activities

The core activities of the project are:

• Activity 1: Provide individual GBV Case Management support and referral to vulnerable women, girls, men and boys including GBV survivors, residing in South Lebanon.
• Activity 2: Develop billboards, and convene workshops on Stress and Anger management and masculinities to promote gender equality and access to men centers
• Activity 3:  Conduct outreach and awareness raising sessions on GBV topics and implement a Parenting Skills curriculum targeting host and refugees’ community
• Activity 4: Implement Youth Empowerment Program, program ‘RA’, and training of youth on specific sets of skills.
• Activity 5: Perform safety audits and compile safety plans.
• Activity 6: Conduct coaching visits to CMR trained facilities
• Activity 7: Provide self-care support sessions to GBV frontline workers.
• Activity 8: Provide psychosocial support activities including emotional support to vulnerable beneficiaries
  1. Assignment background

Plan and IMC are exploring opportunities to expand our current GBV and protection programming in Lebanon and to other countries in the region, such as Jordan.  To ensure the quality and impact of future programming, Plan and IMC are seeking to:

• Understand the strengths and weaknesses of the current project approach;
• Understand how the project design can be adapted in response to identified strengths and weaknesses, as well as changes in context in Lebanon and the region;
• Develop a program model that situates the program approach in the context of the wider GBV evidence base and practice and that can form the basis of future GBV and protection programming in Lebanon and the region.

Plan and IMC are seeking to engage a consultant or team of consultants to undertake this assignment.  Plan and IMC are seeking applications from qualified consultants as teams or as individuals open to working with other consultants who apply for the assignment.

1. Purpose of the assignment

The purpose of this assignment is to evaluate the Protection Activities in Lebanon Project and develop/ document the program model and associated evidence base, to inform the future design of the program.

1. Objectives of assignment

This assignment has two specific objectives:

1. Evaluative component: To provide an in-depth and independent assessment of the project approach and achievements; and identify enabling/ hindering factors, lessons and recommendations for improvement for future programming.
2. Program model component: To develop and document key aspects of a comprehensive, effective and evidence-based program model, which can be used as a basis for future programming.

The two objectives are closely linked: the evaluative component will inform the “evidence base” for the program model and identify ways in which the program model needs to be modified/ improved for future scale-up and replication.

1. Scope of work
   1. Component 1: Project evaluation

1. Specific objectives

The objectives of the evaluation component are to:

• Evaluate the performance of the project in terms of its relevance, effectiveness, and efficiency.
• Identify factors that influenced the project’s relevance, effectiveness and efficiency.
• Contribute to the evidence base for the effectiveness and efficiency of the program model.
• Identify if/ how the program model can be improved for future programming.

1. Key evaluation questions

The evaluative component of the assignment will be guided by key evaluation questions (KEQs), based on selected criteria – relevance, effectiveness and efficiency.  Questions related to gender and inclusion are integrated within the KEQs in these criteria.

Criterion

KEQs

Relevance & appropriateness

• How relevant are the program’s intended outcomes to the specific, differentiated needs of:
  • Women, men, girls and boys who directly engage with the project
  • People experiencing different forms of vulnerability, including people who identify as LGBTQI and people living with a disability.
• How relevant are the program’s intended outcomes to the priorities of key project stakeholders, including:
  • Duty bearers, including community leaders and sub-national and national government departments
  • Relevant service providers
  • The Australian Department of Foreign Affairs’ (DFAT’s) current and emerging strategic priorities for their Syria response package.
• How appropriate are the program’s approach and strategies for addressing gender-based violence (from both prevention and response perspectives), in terms of:
  • Adapting approaches/ strategies to different forms of GBV and the socio-legal and cultural contexts that the project works within.
  • Targeting, reaching and meeting the specific needs of survivors of GBV, including women, men, girls, boys and people experiencing different forms of vulnerability, including people who identify as LGBTQI and people living with a disability
  • Engaging with families and communities
  • Working with service providers to improve capacity and quality of services
  • Engaging with duty bearers, including community leaders and sub-national and national government departments.
  • Managing risks to project participants – specifically survivors of GBV (consider the different risks facing women, men, girls, boys, people who identify as LGBTQI and people living with a disability.
• What are the key learning and recommendations that should inform future programming in order to ensure the relevance and appropriateness of outcomes, outputs and strategies, from both a GBV response and prevention perspective?  Consider the specific needs and priorities of different stakeholders, including children.

Effectiveness

• How has the project performed in terms of delivering outputs and achieving (or progressing towards) intended outcomes?
  • Are the outputs being delivered as planned, in terms of targets and scheduling?
  • To what extent has the project improved the well-being of Syrian refugees and vulnerable Lebanese population?
    • How did the results differ for women and men, girls and boys, people who identify as LGBTQI and people living with a disability?
  • To what extent has the program improved the capacity of frontline workers to provide quality protection and health services?
• How has the project contributed to addressing the root causes and contributing factors of gender-based violence?  Consider how the project has worked within all levels of the socio-ecological model (individual, relationships, communities and societal/ systems).
• What have been the critical enabling factors and hindering factors that have affected the implementation and effectiveness of the project?  Consider:
  • Internal factors (such as resourcing and project management practices)
  • External factors (such as socio-cultural factors, characteristics of services and referral systems and contextual factors beyond the project’s control)
• What are the key learnings and recommendations that should inform future programming, in order to ensure effectiveness from both a GBV response and prevention perspective?  Consider the specific needs and priorities of different stakeholders, including children.

Efficiency

• How has the project minimised costs/ inputs and leveraged available resources (including resources external to the project) to deliver activities and achieve outcomes?
• What are the key learnings and recommendations that should inform future programming in order to ensure efficiency?

 

1. 1. Component 2: Program model
2. Specific objectives

The objectives of the program model component are to:

• Identify the logic and key components of an effective and appropriate program model, to form the basis for future programming.
• Identify considerations for scaling up the existing project in Lebanon and replicating the project elsewhere.
• Provide a basis for developing a “packaged” program model to showcase the model to donors, government stakeholders and service providers.

1. Guiding questions

Development and documentation of the program model will be based on guiding questions.  The questions are based on a recent report commissioned by Plan International, which reviewed sector practice around developing and documenting program models.

In the scope of this assignment, the priority is to identify the following key aspects of the program model:

• Impact statement and theory of change (ToC)
• Model components (essential and optional)
• Contextual considerations for implementation and adaptation of the model
• Evidence base for the model
• Opportunities/ guidance around advocacy and influencing
• Key indicators that could integrated into a MEL framework for the program model.

Aspect of program model

Guiding questions

Model structure/ design

• What is the program model’s underlying ToC?
• What are the components of the program model, including their key objectives and strategies?
  • Which components are essential to ensure the effectiveness of the model and which are optional?
  • Are there any components that should or could be added, to enhance the effectiveness and/ or impact of the model?
  • What are the links/ interactions between the different model components?
• What minimum standards are/ should be applied in the program model (at the component or overall model level)?

Contextual considerations

• What are the key characteristics of the contexts in which the program model could be applied?
• What are the key considerations for adapting and contextualising the program model?

Evidence base

• What evidence is there around the effectiveness, relevance and efficiency of the program model?
• What areas do we need more evidence around, in terms of demonstrating the effectiveness, relevance and efficiency of the program model?

Advocacy & influencing

• What/ whose external agendas does the program model align with and how? Consider government, service providers, donors and other influential stakeholders.
• How is/ should advocacy and influencing be integrated into the program model, to increase project impact? Consider government, service providers, donors and other influential stakeholders.

1. Approach & methodology
   1. Collaborative & adaptive approach

As this assignment is focused not only on evaluating the current project, but also on planning for future programming, the consultant/s will be expected to work collaboratively with IMC and Plan staff in developing (and when necessary adapting) the approach and methodology, and delivering the validation workshop and reports.

1. 1. Theory-based approach

A Theory of Change (ToC) is currently being developed by Plan and IMC, which illustrates and explains the logic, assumptions and change pathways behind the current program design, including outcomes that are not captured in the original project proposal.  The ToC will be provided to the consultant/s at the beginning of the assignment.  The ToC will guide the evaluation, particularly around the effectiveness assessment.  As the evaluation will be undertaken prior to the completion of the program, the ToC will facilitate assessment of intermediate outcomes and progress towards end-of-program outcomes, while also building evidence around the program’s direct contribution to outcomes.

The ToC will also form the basis of the model development and documentation.  It will be adjusted based on the findings of the evaluation, as part of the process of developing and documenting the program model.

1. 1. Methodology

The final methodology will be agreed between the consultant/s, IMC and Plan.  Some basic parameters for the methodology have been identified, however.

1. “Project orientation” with relevant Plan and IMC staff

The consultant/s will participate in “project orientation” sessions with relevant Plan and IMC staff, as part of the development of the Inception Report.  The purpose of the sessions will be to ensure the consultant/s understands the assignment, the project and how it has evolved, as well as Plan’s and IMC’s thinking around the future programming.

1. Desk review

A desk review will be conducted by the consultant/s, focusing on available project data (including monitoring data and data from the project’s mid-term review) and external literature.  The desk review will inform the methodology to be used in the evaluation and model development – it will focus on identifying gaps in available data about the project, understanding the project context and identifying and investigating comparable programs.

1. Primary data collection

There is substantial quantitative data round the project’s output- and outcome-level indicators.  Primary data collection for the evaluative component of this assignment will focus on outcome-level changes, with a focus on qualitative methods.

The consultant/s will be expected to triangulate data sources to build a robust evidence base around the program’s effectiveness, relevance and efficiency and identify ways to improve the program model.

Methods may include, for example:

• Interviews and focus group discussions with staff and representatives of key stakeholders including cooperating partners, organizations, communities and individuals;
• Case studies for comprehensive examination and cross comparison of cases to obtain in-depth information.

Subject to the security situation in the project areas at the time of the data collection, the assignment will cover all three project sites.  The consultant/s will be expected to visit and collect data from all three sites, as well as review relevant project data and external literature relevant to all three sites.

IMC and Plan will work with the consultant/s to determine the sample size for data collection for different stakeholders.

1. Validation/ sense-making workshop

The consultant/s will be required to deliver a validation and sense-making workshop with Plan and IMC staff, which will aim to:

• Validate the findings of the evaluation
• Adjust the program ToC based on the findings of the evaluation and relevant data collected as part of the program model component of the assignment.  The adjusted ToC will inform the program model.
• Develop/ identify key aspects of the program model, such as the impact statement, essential and optional components and links between model components.

1. Deliverables

Based on the approach and methodological parameters outlined above, the following deliverables are expected from the consultant/s, as part of this assignment:

• Inception Report, which details the methodology and implementation plan for the evaluative component and program model component of the assignment and provides draft data collection tools.
• Final data collection tools and cleaned datasets.
• Validation workshop for IMC and Plan staff (1 day) and associated inputs and outputs.
• Draft evaluation report
• Draft program model document
• Final evaluation report
• Final program model document
  1. Format for reports

The final format for the evaluation report and program model document will be agreed between the consultant/s, IMC and Plan; however, the following structures provide an indication of what is expected.

1. Evaluation component

• Cover page
• Table of contents
• List of acronyms and abbreviations
• Executive summary that should include major findings and recommendations
• The objectives of the evaluation
• A description and justification of the methodology used, timing of assessment and challenges / limitations of the analysis
• A presentation of results and the analysis
• Conclusion, learning and recommendations
• Annexes

1. Program model component

• Introduction
  • Explain the purpose and contents of the document.
• Theory of Change
  • Diagrammatic representation of ToC underpinning the program model and brief narrative description of the change pathways and assumptions.
• Program Impact and Components
  • Detail the program model Impact Statement and describe components, including their key objectives and strategies, whether they are essential or optional and linkages between components.
  • Identify any minimum standards that should be applied at the component or overall model level.
• Evidence base
  • Focus on the effectiveness, relevance and efficiency of the model (based on evaluation component of the assignment and relevant literature supporting the model).
  • Highlight gaps in evidence that need to be considered in future programming.
• Contextual considerations
  • Outline the key characteristics of the contexts in which the model can (and cannot) be applied and some guidance around how the model can be adapted based on different contextual factors.
• Advocacy and influencing
  • Outline how advocacy and influencing should be integrated into the program model, including suggested influencing/ advocacy targets, strategies and tactics and intended outcomes.
  • Highlight where the program model aligns with different targets’ agendas/ interests and where existing agendas/ interests need to be challenged.

1. Ethics & child protection

Plan International is committed to ensuring that the rights of those participating in data collection or analysis are respected and protected, in accordance with the Framework for Ethical MERL and our Global Policy on Safeguarding Children and Young People. All applicants should include details in their proposal on how they will ensure ethics and child safeguarding in the data collection process. Specifically, the consultant/s shall explain how appropriate, safe, non-discriminatory participation of all stakeholders will be ensured and how special attention will be paid to the needs of children and other vulnerable groups. The consultant/s shall also explain how confidentiality and anonymity of participants will be guaranteed.

Additionally, all parties involved in the data collection will receive an induction on Plan International’s Safeguarding Children and Young People Policy, referral pathways, Gender Equality and Inclusion Policy and Non-Staff Code of Conduct.

1. Timeline

The suggested timeline is outlined below.  The timeline is negotiable; however, the assignment must be completed within the first quarter of 2020.  The timeline will also be dependent on the security situation in Lebanon at the time.  The number of consultant days is indicative and will depend on whether an individual consultant or team is appointed and on the final methodology and approach to the assignment.

Task

Responsibility

Outputs

Consultant days

Deadline

Receiving applications

Plan

N/A

0

22^nd November

Recruitment & contracting

Plan and IMC

N/A

0

6^th December

Internal drafting of ToC by Plan and IMC

Plan and IMC

Theory of Change diagram and brief narrative

0

6^th December

Develop/ deliver Inception Report, including:

• Orientation sessions with relevant Plan and IMC staff
• Desk review
• Evaluation matrix/ plan (including sampling approach)
• Model development matrix/ plan
• Formats for key deliverables
• Timeline

Consultant/s

Draft Inception Report (including draft data collection tools)

5

18^th December

Feedback from IMC and Plan

Plan and IMC

Collated feedback from Plan and IMC, in a single document.

0

8^th January

Finalise Inception Report

Consultant/s

Final Inception Report

1

13^th January

Participate in induction on relevant Plan policies including Safeguarding

Plan and IMC

Consultant

N/A

2

22^nd January

Participatory tool testing and finalisation (with relevant IMC and Plan Lebanon staff)

Consultant/s

Final suite of data collection tools

2

24^th January

Undertake primary data collection (including travel days)

Consultant/s (with support from Plan and IMC)

Raw data

7

7^th February

Prepare/ deliver validation and sense-making workshop

Consultant/s (with support from Plan and IMC)

Workshop and associated materials

3

12^th February

Write/ deliver draft evaluation report and program model document

Consultant/s

Draft evaluation report and program model document

10

28^th February

Feedback from IMC and Plan

Plan and IMC

Collated feedback from Plan and IMC, in a single document.

0

13^th March

Write/ deliver final evaluation report and program model document

Consultant/s

Final evaluation report and program model document

5

23^rd March

Total consultant days

35

 

1. Payment & logistics

Payment to the consultant/s will be based on an agreed daily rate/s for the consultant/s’ services and additional reasonable expenses incurred by the consultant/s will be reimbursed.  Plan and IMC will work with the consultant/s to manage in-country logistics and associated costs, including transport, accommodation and organising stakeholders for data collection activities.

The payment schedule for consulting fees is divided as follows:

Inception report and data collection tools:                                          25%

Draft evaluation report and program model document:                50%

Final evaluation report and program model document:                 25%

1. Skills & experience required

The following skills and experience are essential to the assignment:

• Skilled in qualitative and quantitative data collection and analysis
• Extensive GBV expertise, including understanding of international frameworks and evidence base and specifically in conflict/ humanitarian contexts.
• Understanding and experience in applying ethical approaches to data collection with vulnerable participants.
• Extensive experience in conducting evaluations, program design and developing program models.
• Experience working in Lebanon or the region, particularly in the context of the Syria crisis.
• Experience working with vulnerable groups, including refugees and children.
• Solid experience in the development of tools for evaluations involving vulnerable populations.
• Excellent facilitation skills and experienced in delivering engaging, participatory data validation and sense-making workshops.
• Excellent written and verbal communication skills in English.

The following skills and experience will be highly regarded:

• Excellent written and verbal communication skills in Arabic.
• Expertise in Child Protection.
• Prior experience designing and evaluating DFAT-funded programs.