M&E Consultant for final project evaluation

Institute for War & Peace Reporting (IWPR) is an independent not-for profit organisation that works with media and civil society to promote positive change in conflict zones, closed societies, and countries in transition around the world. 
IWPR is looking to hire an M&E consultant to perform the final evaluation on one of its USG-funded, human rights and digital advocacy projects.  

1. Scope of Work 

1. Objective of the evaluation

The overall objective of this evaluation is to provide the final review of one of IWPR’s USG-funded human rights projects by assessing its performance according to the 6 OECD-DAC criteria and other specific criteria. This final evaluation has three main objectives:

• To assess and analyze the achievements after 5 years of project implementation, compared to the expected results and in line with the theory of change that underpinned the design and ambitions of the project. This will involve determining the current level of achievement of the project's indicators (for project objectives and results) and comparing the final achievements with the baseline situation. As for the theory of change, the aim will be to examine the hypotheses of the causal links between actions and results and to confirm or disconfirm the explicit or implicit causal steps of the theory of change.        

• Capture observable changes in practices, behavior and attitudes adopted by individual actors in the media and civil society sector, by targeted stakeholders including communities, state actors, internationals bodies and any other individuals and structures that are reached directly or indirectly in the activities through direct implementation and sub-granting; highlight the relevant achievements and lessons, with a view to informing the continuation of the project and/or the development of other following projects.
• To assess objectively, quantitatively, and qualitatively, the relevance, efficiency, and effectiveness of the project interventions during this time, the sustainability of the results achieved, following the international evaluation criteria of the OECD/DAC as well as the internal and external management and coordination of the project.
• To identify and document good practices / project successes that are recommended for replication to enable scale up/out into new initiatives.

1. Evaluation methodology

The evaluation methodology will be proposed by the consultant in his/her technical proposal. The consultant will have to propose two types of robust methodology to:

• Test the validity of the theory of change and its underlying assumptions, including the analysis of results and outcomes achieved and the logic chain linking interventions to results;
• Assess the impact, relevance, efficiency, effectiveness, and sustainability of the project interventions, as well as the management of the project including the internal coherence and coordination within project partners.

The project has an overarching theory of change. Baseline and progress MEL data have been collected and reported quarterly by the project staff in accordance with the donor-approved logframe with relevant indicators. They include periodic project reports, event reports, subgrantee reports with monitoring data, website analytics, mentors’ reports etc.

The consultant will combine quantitative and qualitative approaches as necessary to adequately address the evaluation objectives and answer the different evaluation questions mentioned below. The evaluation should also assess the gender sensitivity of the project and whether the different needs of women and men were taken into consideration, this includes the collection of sex-disaggregated data. Preferred qualitative evaluation methodologies, largely utilized by IWPR are:

• Outcome harvesting - where the OH can’t be applied fully, the consultants will be asked to develop outcome statements for significant results achieved.
• Most significant change technique - where the MSC can’t be rigorously applied the consultant(s) will be asked to capture impact stories of change for a selected number of project beneficiaries and undertake qualitative data analysis.
• Stakeholders’ engagement - adopt the IWPR stakeholders and/or network engagement methodologies to measure changes in engagement among stakeholders.

The consultant(s) will be asked to update a qualitative evaluation database, store relevant impacts/outcomes, lessons learned, recommendations. The database will be available to conduct qualitative database analysis, consultants are welcomed to use it.

The proposed methodology should include the following phases:

• Preparatory phase: review of all project documentation, updated analysis of the project implementation context, preparation of a work plan jointly with the project team; outlining the evaluation questions kickoff call with staff, donor (as appropriate), propose the evaluation methodology and the development of data collection tools and a sampling methodology. As a result of this phase the consultant will have to submit a plan/inception report, outlining the approach. Upon sign of the report the consultant(s) may initiate the work.
• Fieldwork: Recruit and train local data collectors (if needed). Field test questionnaires with the different target groups through a pilot or pre-survey; conduct data collection, consolidate, validate, triangulate project existing data. Attend/facilitate/lead projects online/in person meetings/workshops to gather data as appropriate. Note any travel requirement should be outlined in the plan and in the agreed overall budget and will be allowed only after appropriate security checks.
• Analysis of qualitative and quantitative data: triangulated and, if contradictions appear, explore and present hypotheses as to why such contradictions might exist; preliminary findings will be presented to the project team and the evaluation committee, and the consultant will need to take into consideration the project team's feedback.

2. Main Deliverables 

Some key deliverables must be submitted in draft form to IWPR and the evaluation committee before being finalized as indicated in the table below. All deliverables and data from the evaluation will be owned by IWPR and may be used for internal and external reporting and communications.

• Deliverable 1: Plan Proposed plan and methodologies for the evaluation and case studies development, to be approved by the project manager.
• Deliverable 2: Conduct online meetings, workshops with IWPR directors, program manager, field staff and other key stakeholders (beneficiaries, donor) to capture, document, triangulate results.
• Deliverable 3: Draft evaluation: submit to the evaluation committee a draft evaluation and related annexes that outline the main findings with related evidence, including recommendations and lessons learned. Visualizations and case studies are encouraged.
• Deliverable 4: Final evaluation report and summary: in Word format, to be produced in English, should be a maximum of 30 pages (excluding annexes/attachments) and should include the following elements. Summary of the evaluation findings: 2 pages max with overview of overall achievements/performance and key actionable recommendations.
• Deliverable 4: Evaluation dataset A database (excel) with list of outcomes, significance, contribution and related categorization used for the analysis.
• Deliverable 5: Case studies Up to two case studies (2 pages max each) to support “scaling up of success”. Throughout the evaluation the consultant(s) will identify up to 4 successes or “failures” (either good practices, techniques, results, lessons learned etc.) and, upon IWPR team confirmation, the consultant will document them, providing sufficient evidence and guidance to support uptake in new programming.  Each case study should be 2-4 pages max. The program envisaged already one case study focused mainly on structure/approach and its suitability to be adapted, replicated to different contexts and or scaled up in size.
• Deliverable 6 debriefing of findings: through a series of meetings and by sharing reports, in collaboration with the project team, engage with partners, implementers and donors in forward looking discussions starting from the evaluation finding. The evaluator will be asked to outline a plan for dissemination and attend/facilitate up to 3 meetings. 

​​​​​3. Reporting 

A proposed full evaluation structure is outlined in the Annex 1. The final full evaluation report should provide with following information and analysis:

1. Overall assessment of the level of success of the project as per  DAC criteria: coherence, efficiency, impact, effectiveness, relevance, sustainability.
2. Outline key actionable recommendations for future programming in the light of current needs (national/regional context), project goal and project successes, good practices and lessons learned.
3. List of significant outcomes achieved, intended as observable changes in actions, relationships, policy or practice of both beneficiaries and external actors linked to the project activities, categories them using existing and unplanned performance indicators, by DAC criteria and other relevant categories (e.g. gender, type of beneficiary, date etc).
4. Outline significance and relevance of each outcome in light of implementation context at national, regional and international level.
5. Outline contribution of the project for each outcome.
6. Beneficiaries reach: overview of beneficiaries reached directly and indirectly: number, type, gender, country and any other relevant disaggregation.
7. Outline significant challenges and/or negative changes that have limited or affected achievement of success, analyze effectiveness of mitigation strategies applied (if any) and level of impact of those challenges. Outline lessons learned for future implementation.
8. Track progress as per project performance matrix/M&E framework (baseline, targets and year by year progress)
9. Include visualizations (graphs/tables) and quotes as appropriate to support deeper understanding of evidence and analysis.

​​​​​4. Qualifications of Successful Candidate

We are looking for a consultant/team of consultants/international or national consultancy firm meeting the following requirements:

• At least 5 years of significant experience in conducting multi-countries program final evaluations, applying quantitative and participatory qualitative evaluations. Proven expertise on outcome harvesting, most significant change and GESI focused evaluation will constitute an advantage.
• Proven experience in conducting evaluations of complex, multi-stakeholder projects in the areas of peacebuilding, advocacy, inclusion and equity, cyber security, organizational and network governance strengthening, and theory-based evaluations.
• Excellent virtual facilitation skills;
• Prior expertise in conducting MEL in complex settings, using secure online communication channels;
• Ability to work with limited supervision, as part of a small team and independently.
• Experience with qualitative data analysis.
• Excellent writing skills in and English. Knowledge of Arabic will be considered an advantage;
• Excellent report writing skills;
• Knowledge of the socio-cultural, economic and political context of the MENA region;
• If it is a national evaluation team: compliance with administrative documents, complete documents and valid and up to date authorization to operate.
• Excellent command of Arabic and English (Final repot language is English).