Final Evaluation Study - Voices of Hope: Aswat al Amal

1. Context

The Project

Search is implementing a 24-months project funded by the United States Department of State Bureau of Democracy, Human Rights and Labor (DRL) with the objective to “Foster sustainable CSO-led reintegration and protection of women and children impacted by the conflict”. The project focused on empowering the local CSOs to better address the needs of their communities and thereby contribute to rebuilding a functioning society, enhancement of leadership skills and building their capacities to actively contribute to promoting the reintegration and protection of women and children.

The selected group of local civil society actors from the target areas received technical capacity building through a contextually tailored curriculum relevant to the context. The CSOs were then supported to design and facilitate collaborative activities in their communities (through the issuance of small grants, while forming the content for media production action and involving community members directly in the project’s activities. The CSOs also engaged community members including men, women, youth, and other vulnerable groups to reflect on local dynamics related to reintegration barriers, as well as address the stigma women and children face by being associated with former combatants through community dialogues and platforms.

The CSOs also engaged community stakeholders in neutral spaces to undertake an inclusive healing process that takes into consideration the needs and rights of women and children, as well as communities’ fear and opportunities for resilience. The projects adhered to a ‘whole-of-society’ approach to ensure local communities’ buy-in and allow target stakeholders to embrace inclusive actions, which builds trust between returnees and communities of origin, as well as between traditional power holders (adult males) and vulnerable groups (women and children), thereby promoting greater social cohesion.

In addition, the project engaged a group of media actors across the target areas, who received training on media for social change approach to better develop and disseminate narratives that promote reintegration and rights protection of women and children in a conflict sensitive manner, leading to increased social cohesion through the media campaign.

The results of this intervention include a successful implementation of the Common Ground Approach (CGA) training, reintegration and protection training for CSOs, structured capacity building program, CSO-led community dialogues, reintegration and protection platforms, reintegration and protection initiatives, media for social cohesion training, and media campaigns in each area which provided the CSOs and the media actors with a key opportunity to apply skills gained and contribute to achieving the project’s objectives.

 

Target groups  

The primary target groups of the project are 12 CSOs, six reintegration and protection platforms including 48 community stakeholders (8 from each target area), and about 50 media actors (6 to 10 actors from each target area). The secondary target groups are the wider members of target communities; specifically vulnerable children and women groups, such as internally displaced persons (IDPs), minorities, and poor households.

 

2. Goal and Objectives of Study

Search is seeking a consultancy team/firm to conduct a summative Endline Evaluation study to understand the extent to which the project has contributed in fostering sustainable CSO-led reintegration and protection of women and children impacted by the conflict in the targeted areas. This study will also aim to assess the project’s progress towards its specific objectives and considering the remote implementation:

  • SO.1: To foster positive leadership and collaboration among a diversity of target civil society stakeholders on the reintegration and protection of women and children in target areas; and
  • SO.2: To foster community support for the reintegration and protection of women and children in target communities.

The Final Evaluation Study should enable Search and donors to inform future programme design and will assess the overall success and effectiveness of the implemented project. In specific terms, The Final Evaluation has the following objectives:

  • Assess the overall impact of the project based on evidence-based outcomes whether intended (according to the project’s logical framework) or unintended outcomes (positive or negative) and determine Search’s contribution to identified changes;
  • Update the project’s logframe and indicators with endline values;
  • Assess whether the project had produced the effects as envisioned in the Theory of Change in addition to assessing the project’s Relevance, Efficiency, Effectiveness and Impact, and Sustainability as per the Organisation for Economic Co-operation and Development - Development Assistance Committee (OECD-DAC) evaluation criteria in addition to the overall project’s adaptability; and
  • Capture good practices, compile lessons learned throughout the lifespan of the project and draw actionable and evidence-based programming and policy recommendations to inform future programme design in a similar context.

 

3. Key Questions of the Study

End-line Questions

The analysis will be focused on the following evaluation questions organised per OECD-DAC criteria: 

  • Relevance:
    • To what extent did the intervention’s objectives and design respond to the needs of CSOs in strengthening their roles to lead sustainable, conflict- and gender- sensitive, and dedicated reintegration and protection of women and children in the target communities?
    • In light of the recent developments that occurred, are the objectives of the intervention still valid? Are the activities of the intervention still consistent with the overall goal of the project and fulfilling its specific objectives?
    • To what extent has ‘do no harm’, conflict sensitivity and barriers been taken into consideration in the design and implementation of CSOs-led initiatives and what are the effects of these interventions and how are they contributing to peacebuilding?
    • To what extent did the intervention’s objectives and design align with and contribute to Search’s Levant Strategy and DRL plans in the target country?
  • Effectiveness and Impact:
    • To what extent has the intervention been effective in meeting the essence of the Theory of Change and achieving its results and planned outcomes by addressing common issues and needs of targeted communities- considering the remote implementation, by assessing:
      • The level of change in knowledge and capacities of CSOs and media actors;
      • The level of change in personal agency among CSOs towards the ability of making positive difference in the community to reintegrate vulnerable women and children in target areas;
      • The change in CSOs and community stakeholders' perception of having tangible opportunities to cooperate with one another to promote reintegration and protection of women and children in target areas;
      • The level of engagement, collaboration and coordination between CSOs and community stakeholders across divides to discuss sensitive topics related to the reintegration and protection of vulnerable women and children;
      • The contribution of the media component in amplifying the value of positive CSOs-led efforts and community relationships inclusive of women and children.
    • Has the project been able to engage the CSOs with community stakeholders, including PWDs in their efforts to promote reintegration and protection of women and children in the target communities?  
    • What unintended outcomes (positive and negative) emerged from project implementation?
    • What factors (positive and negative) have had the greatest influence on the achievement of results?
    • To what extent did the intervention contribute to longer-term effects (impact)?
    • Was the level of partnership(s) with CSOs and the implementing partner appropriate to support the effective achievement of the intervention’s objectives?
  • Efficiency and Value for Money:
    • To what extent have resources been allocated and utilised in an efficient manner to achieve value-for-money?
      • How efficient was the delivery of the intervention in terms of expenditure and implementation of activities?
      • How efficient was the remote implementation approach and through partners  towards the achievement of the objectives? How could this remote approach be improved?
      • What was the return on investment attained?
  • Sustainability:
    • To what extent are the benefits of the programme likely to be sustained in the long term, in terms of 1) CSOs capacity building, 2) CSOs’ collaboration and engagement with communities and key stakeholders to promote reintegration and protection 3) relationships, connections, and platforms created in the project.
      • Are the systems, policies, processes and practices built/improved under the project likely to continue in use or benefitting the CSOs?
      • To what extent do key project activities already have multiplier effects that extend beyond the targeted CSOs?
      • What are the efforts required to improve and strengthen sustainability?
    • To what extent did the CSOs show efficiency and ownership towards the project?
  • Lessons Learned:
    • What lessons can be learnt or good practices can be identified from the implementation of the project? This is to be supported by evidence.
    • What notable programming and policy recommendations can be drawn from this project?

 

4. Geographic Locations

The Endline Evaluation study will provide quantitative and qualitative information, gender, age, ethnicity, religious background, and residency status (host community member, IDP and refugee) disaggregated data and analysis in a strategic county in the Levant region.

 

5. Methodology and Data Collection Tools

The Endline Evaluation Study will utilise a mixed method approach consisting of both qualitative and quantitative primary and secondary data collections. The study should triangulate data from multiple sources and stakeholders in order to infer reliable findings. In addition to the proposed conventional approach the consultancy team will be taken to carry out the evaluation through using the evaluation questions (section 3) included in the ToR. The endline is expected to integrate and apply ‘Outcome Harvesting’ techniques to identify the most significant results and the project’s contributions to these changes.

Desk review: the consultant(s) will review key project documents including the proposal, the baseline and mapping report, MEL framework, logframe and other related monitoring and programmatic documents and set meetings with key personnel to better understand the project, its goal and activities. The consultancy firm is expected to review the above background documentation as part of the desk review phase of the study, in addition to the literature review about reintegration and protection interventions, the role of civil society organisations in addressing reintegration and protection issues and the conflict dynamic in the targeted areas. Also, the consultant will utilise primary available M&E data that was formerly collated by project staff including activity surveys and assessments, baseline data, project quarterly reports, etc.

Data collection tools and approach: The consultant(s) will develop quantitative and qualitative data collection tools and collect data in the form of surveys, interviews, focus group discussions (FGDs), and possibly case studies using a solid sampling approach. To ensure a better understanding of the key issues that will be addressed in the project, the data will be disaggregated by age, sex, gender, age, ethnicity, religious background, and residency status (host community member, IDP and refugee), persons with disabilities as well as by geographical location.

Sampling strategy and approach: The consultant(s) shall determine the appropriate sample size and approach in consultation with Search MEAL Manager and referring to the M&E plan. The sampling strategy will take into consideration the activities carried out, target population and the project areas (95% Confidence Level and 5% Margin of error). The consultant(s) will meet with the project participants, partners, Search staff and relevant stakeholders.

Outcome harvesting approach: In addition to the proposed conventional approach of the evaluation, the evaluation is expected to integrate and apply, where feasible steps 2-5 of the six-steps of ‘Outcome Harvesting’ to answer the questions of what actual achieved results are as opposed to those that are planned, what has changed, when and how, identifying the most significant results, and assessing project’s contributions to these changes. Step 6 will be limited to and where feasible, the presentation of actionable recommendations in the evaluation report. The level of integration should be defined clearly in the proposed methodological approach of the consultant(s) technical proposal.

Evaluation matrix: The consultant(s) is also required to develop a detailed evaluation matrix with judgment criteria that captures the study’s objectives and evaluation questions of which will guide the evaluation process into reliable judgments and outcomes.

Search’s MEAL Manager will validate and approve the final methodology and tools before data collection begins, as well as provide feedback on the report.

Data collection, analysis and the results should represent Inclusiveness—the methodology should include a wide range of viewpoints, specifically gender and age-sensitivity when applicable.

 

6. Deliverables

Search expects the following deliverables from the external consultant(s) as they correspond to the timeline and budget:

  • An inception report detailing the methodology, data collection tools, analytical framework, approach and timelines;
  • Developed or used  training materials for data collectors or set-up of systems for data collection;
  • All original datasets should be submitted to Search (Excel format for the quantitative data and all transcripts for the qualitative data); In addition to any necessary training of data collectors or set-up of systems for data collection;
  • Outcome Harvesting Workshops;
  • A draft Final Evaluation report in English for review by Search staff; and
  • A final report in English (40 pages max in length, excluding appendices) consistent with Search branding and standards for evaluation. The report will:
    • Uses the Search reporting template;
    • Provides a clear connection between the conflict or context assessment and the intended results, articulate the project’s Theory of Change (ToC), and include other relevant project specifics;
    • Fully explains the objectives and research questions of the study, limitations and methods chosen for analysis; 
    • Findings respect Search’s evaluation standards, are structured around the main objectives of the study, and are presented in relation to the intended target groups.
    • The findings should speak to the link between the project, its TOC and its contribution to our strategy. It should explain adaptations that occurred during the project and their impact on results;
    • Recommendations should have a clear audience and be specific, accessible, and actionable;
    • Indicator table showing all indicators; and
    • Appendices should include detailed research instruments, list of interviewees, terms of references and evaluator(s) brief biography.
  • A two-pager summary highlighting key lessons learnt and programming and policy recommendations  from this project that will inform future Search and donor programming;
  • A PowerPoint presentation that will be used by the consultancy team to present the findings to Search and the donor after the final report is approved; and
  • A one to two hours online presentation of research findings to the Search and donor team.

 

7. Logistical Support

Regardless of the modality chosen, the firm/consultant(s) should be fully responsible for the data collection, including working with data collectors to be recruited and trained by them in case needed. In case of in-person data collection, the firm/consultant(s) will be responsible for organising their own logistics for data collection (vehicles, fuel, and  drivers). All of this must be budgeted into the financial proposal.
 

In addition, Search and partners will share the following elements with the external consultant(s): background materials including the project proposal and logframe, etc.

 

8. Ethical Considerations

The consultancy should take into consideration  the rights of participants in line with the ethical codes and guidelines of research. The participants should be asked to provide their consent and made aware that their identities would not be revealed. Cultural rights will be respected. The consultant shall remain alert and responsive to any child safeguarding risks, acquire relevant knowledge and skills which will enable promoting strong safeguarding practices, understanding the child safeguarding policy and procedures.

The firm/consultants are required to respect the following ethical principles:

  • Comprehensive and systematic inquiry: Consultant should make the most of the existing information and full range of stakeholders available at the time of the review. Consultant should conduct systematic, data-based inquiries. He or she should communicate his or her methods and approaches accurately and in sufficient detail to allow others to understand, interpret and critique his or her work. He or she should make clear the limitations of the review and its results.
  • Competence: Consultant should possess the abilities and skills and experience appropriate to undertake the tasks proposed and should practice within the limits of his or her professional training and competence.
  • Honesty and integrity: Consultant should be transparent with the contractor/constituent about: any conflict of interest, any change made in the negotiated project plan and the reasons why those changes were made, any risk that certain procedures or activities produce misleading review information.
  • Respect for people: Consultant should respect the security, dignity and self-worth of the respondents, program participants. Consultant has the responsibility to be sensitive to and respect differences amongst participants in culture, religion, gender, disability, age and ethnicity.

Applicants are kindly required to integrate a detailed multiple scenario strategy in their proposed methodologies and work plans that would ensure efficacy in the delivery of the assignment and allow for flexibility to switch between different modalities (i.e., to virtual data collection and vice-versa) whenever there is a need to do so.

 

9. Data Quality Assurance and Management

All deliverables will be reviewed and approved by the country office and global Institutional Learning Team (based in Brussels) prior to acceptance of the final product. The consultant(s) are also required to develop a clear and rigorous quality control plan during the inception phase of which will be implemented and deployed throughout all phases of the evaluation process to ensure reliability and validity of findings.

To ensure effective data management, all the qualitative collected data (through FGDs, KIIs) will be transcribed with the respect to the data collection ethics. All the quantitative surveys will be cleaned and stored in a secured database. Access to the data will be allowed to the staff members who work on the final evaluation and will be limited to those who have the password.

 

10. Timeline

The evaluation timeline is expected to be kicked off in mid-June 2023 and concluded by the end of September 2023 by submitting the final deliverables requested above.

 

11.Budget
The total budget available for this study is a maximum of 17,000 USD. A detailed budget should be provided, including daily rates for personnel, and costs related to data collection (per total number of people sampled, sites for collection, etc.), analysis, and production of deliverables. The selection committee will give preference to the applicant(s) with a relatively lower budget and higher quality technical proposal.

 

12. Requirements of Consultant

Education:

  • Lead consultants must have at  least a Master’s degree (PhD preferable) in research methods and/or evaluations, development, conflict or peacebuilding, M&E or any related social sciences.

Skills and experience:

The following skills and experience are expected by Search for our evaluator for this project:

  • Excellent proficiency in English and Arabic - written (including professional English reporting) and spoken;
  • More than five years of experience in programme/project evaluation, including collecting data in interviews, surveys and focus groups and analysis;
  • Experience  in carrying out studies, evaluations, and baseline on Reintegration and Protection and Civil Society Support Programmes in the Greater Levant region countries such as Iraq, Syria and Lebanon.
  • Experience applying remote data collection modalities in conflict areas in the Greater Levant region;
  • Experience in conflict analysis and working with justice, civil society sectors and gender related topics;
  • Experience in doing evaluations for peacebuilding and democracy, human rights and governance programmes in MENA region;
  • Experience working with international organisations;
  • Experience conducting quantitative surveys and analysis;
  • Monitoring and Evaluation methods and data collection skills; and
  • Familiarity and experience with contextual challenges in the geographic location(s) where the study will take place.

 

13. Selection Criteria

Consultant proposals will be selected for:

  1. Relevance of proposed methodology to the goal, objectives, and research questions for the project.
  2. Quality of proposed methods, conflict sensitivity approaches, and quality control measures.
  3. Qualifications of the candidate(s).
  4. Proposed budget in relation to proposed methodology, deliverables and team.
  5. Timeline for proposed activities.

How to apply

To apply, interested candidates (individuals or teams) are requested to submit the following documents to the below link:

https://jobs.lever.co/sfcg/5242cc11-c13f-4e34-ae3e-afaaf4ff3d2f

  • A Technical Proposal clearly outlining the proposed methodology, understanding of requirements, experience doing similar work, and timeline for the endline together of not more than ten pages (excluding annexes); this to be combined with:
    • Short cover letter (not more than one page) with at least one writing sample (preferably an endline evaluation); and
    • Curriculum vitae of lead consultant(s).
  • A Financial Proposal (with detailed line items per deliverable) for the completion of the aforementioned deliverables (not more than two pages).

     

منتهية الصلاحية
آخر مدة للتقديم
الأربعاء, 31. مايو 2023
نوع الدعوة
دعوة لتقديم الاستشارات
قطاع(ات) التدخل:
حل النزاعات
Remuneration range:
> 6000 (USD)
Duration of Contract:
Please refer to the "Timeline" mentioned in the description
randomness