Call for Applications: Impact Evaluator - Consultant

Roles and responsibilities of the consultant/team:

The consultant will be expected to implement the various stages of preparation, planning, implementation and analysis of the impact evaluation. The evaluation will be expected to include, but not be limited to, the following groups:

• Targeted beneficiaries (Refugees and Host communities) (women and men); to be included in quantitative and qualitative assessments
• Partners and stakeholders involved in the project at different levels:
  • Targeted Municipalities and staff members: Service providers (case managers, social workers, medical and non-medical staff)
  • Community actors: community health workers, informal settlement focal points
  •  Government agencies: Ministry of Public Health (MPH), Ministry of Social Affairs (MoSA), municipalities

1. Develop an overall impact evaluation methodology: He/she is excepted to share and discuss the methodology with arcenciel.

2. Develop qualitative and quantitative data collection tools: The consultant/team will develop, test and finalize the tools. He/she is expected to discuss the draft tools with arcenciel and finalize the tools after the testing.

3. Develop a framing note: He/she is excepted to share and discuss the framing note with arcenciel.

4. Lead on data collection: conduct desk review but also, through questionnaires, conduct key informant interviews and FGD and fill out surveys with relevant stakeholders and target groups as per the evaluation methodology

5. Lead analysis of data: The consultant/team will share the raw data and basic analysis with arcenciel, then he/she will analyse this data and present a summary of the initial findings. The consultant/team will carry out the literature review and submit a written summary in the report. He/she will be responsible for the safe and secure storage and treatment of this data.

6. Develop a draft report: He/she is excepted to share and discuss the draft of a report with arcenciel.

7. Present and discuss initial findings: The consultant/team will present and discuss the findings with arcenciel in consultative meetings in order to refine analysis and ideas for recommendations.

8. Develop a final report: Bringing together the findings from the literature, quantitative data and qualitative analysis to finalise the report. The report should highlight a set of operational recommendations and lessons learned. Also, the report might address all arcenciel comments.

9. Present and discuss findings: The consultant/team will present and discuss the findings with arcenciel and AFD in a meeting.

10. Develop a summary and present the key findings of the report: In English for partners, external stakeholders and donor. The summary is a maximum of 2 pages long. It will be published on AFD's website.

Methods:

An external consultant or team of consultants will be responsible for conducting the impact evaluation. The consultant/team is expected to conduct a desk review of relevant project documents and monitoring reports/data, meet with senior management, program staff, centre staff, and relevant stakeholder, and participate in consultation and debriefing meetings. The firm/team is then expected to develop a detailed methodology, appropriate data collection tools, and a detailed data analysis plan. Any quantitative survey should include a sample size calculation that will render the results statistically valid. Discussions on the sample size and proposed tools are to be shared with the M&E team before the start of the actual fieldwork.

The consultant will be expected to include a mix of quantitative and qualitative data collection methods such as surveys, focus group discussions, key informant interviews, and field observation visits. Triangulation of data through different tools and sources is considered fundamental.

Overall, the evaluation will be done according to a list of criteria which includes relevance, effectiveness, efficiency and sustainability, and through desk research, meetings, and interviews with relevant groups and individuals.

Implementation information:

a. Contact

The consultant/team will work closely with arcenciel’s project coordinator, M&E officer and Program Managers. Centre staff will be also working closely with the consultant/team.

b. Location

Health centres operated by arcenciel in the regions of Bekaa, Akkar, Beirut and Mount Lebanon.

c. Deliverables (in French or English)

• Methodology tools
• Qualitative and quantitative data collection tools (interview guide, questionnaire, etc.), templates should be provided in English or French and Arabic.
• Framework Note

On the basis of the methodological research, the consultant shall provide a document (which should not exceed 30 pages, including the narrative of the project which must not exceed 10 pages) summarizing the framework of the evaluation after the beginning of the research. This note i) will present the intervention logic of the project; ii) will specify the main questions that will be used to focus the evaluation work on a limited number of key points in order to allow for more targeted information gathering during the field phase, finer analysis and a more useful report; iii) will establish the steps of reasoning that will allow the consultant to answer the questions (judgement criteria); iv) specify the indicators to be used to answer the questions and the corresponding sources of information; v) accurately confirm the steps of the method (including the number of man-days allocated to each of these steps defined in its proposal), as well as the methodological principles that will guide the evaluation and how these principles will be implemented in practice (e.g. if the evaluation is intended to be participatory, then the note should explain how the participatory dimension will be implemented).

This note will include a project narrative in order to reconstruct its progress at its various stages. It will include a general presentation of the context and its evolution during the implementation period, a description of the project (objectives, content, amount, financial product, stakeholders, operating mode) and a summary of the project's progress since its inception. Within this framework, the consultant will make a distinction between the project appraisal phase (from identification to the granting of AFD financing), its implementation phase (from the signing of the financing agreement to the completion of the project), and finally the significant events that occurred between the project completion date and the date of the evaluation. This synthesis will focus on the facts and factors that feed into the evaluative analysis.

This framework document must be validated by arcenciel and by AFD before the work can continue.

• Transcript of interviews and focus group discussions (FGD)
• A progress report as an intermediate deliverable, for example after field visits. This deliverable can be a simple Power point presentation (PPT). It ensures that the consultant does not start on the wrong path.
• Draft report

A draft of the final report, which should not exceed 40 pages outside the appendices, will be produced at the end of the consultant's work. arcenciel will provide its comments and observations to the consultant within three weeks of receiving this report.

• PPT reflecting initial findings for arcenciel
• A clear and concise, well-written final impact evaluation report containing lists and tables, facts and figures demonstrating the state of the end-line situation and other findings at the end of the project, as well as a detailed comparative analysis of other health models implemented in similar contexts. The final report, incorporating observations from the first draft, should be available within 15 days after receiving comments. If the observations express differences of assessment not shared by the consultants, they may be annexed to the final report and commented on by the consultants.
• Meeting with the donors to present the findings of the report
• Summary of the final report for dissemination to external related stakeholders. This summary will include a presentation of the project (context, stakeholders, modalities, objectives, content) and will include the main conclusions, lessons and recommendations of the report. This summary note aims to promote the dissemination of evaluation lessons. It should assume that readers of this note will not necessarily read the full report. It must be both very operational and understandable by everyone.

d. Required qualifications and expertise

The consultancy is open for individual consultants or teams and companies.

The candidate:

• Preferably holds a postgraduate degree in social sciences, development studies, medical studies or a related field and/or has at least three years of experience with familiarity in Health and Social field;
• Preferably has at least 5 years’ experience in conducting similar evaluation in Middle East;
• Excellent skills in research and data collection;
• Ability to work both independently and as part of a team;
• Speaks good Arabic. English and French is also required;
• Ability to conduct interviews in Arabic;
• Preferably has knowledgeable about refugees’ context;
• Is willing and available to travel in all Lebanon.

e. Timeline

Consultant bidding: Beginning of May 2020

Consultant selection: End of May 2020

Contract signature: End of May, beginning of June 2020

Draft report: August 2020

Final report: September 2020

Summary/presentation of the final report: September 2020

Presentation open to partners and co-construction of recommendations: September 2020

f. Length

The research is expected to take up to 12 weeks including: document review; methodology reviewed and validated internally; field visits and centres visits; finalisation of first draft; internal review and validation process by the partner (AFD); final draft approved.

g. Logistic support

The consultant/team will be responsible for the logistical planning for the meetings, interviews and field visits with the support of the project coordinator and M&E officer.