Outcome Evaluation Study for Reducing Economic Barriers to Accessing Health Services in Lebanon
Outcome Evaluation Study for Reducing Economic Barriers to Accessing Health Services in Lebanon
Country of the action:
Location(s) of the action:
Bekaa, South, Tripoli, Akkar, Beirut and Mount Lebanon
Name of the lead applicant
International Medical Corps UK
Almost nine years since the outbreak of war in Syria, Lebanon continues to host the highest per capita number of Syrian refugees anywhere. The additional population has strained essential services, limiting access for both refugees and vulnerable Lebanese. As part of efforts to meet urgent needs, International Medical Corps is providing comprehensive primary health care, mental health services, and gender-based violence prevention and response programming for persons of concern throughout the country. International Medical Corps’ primary goal is to support the expansion of an affordable health care system in Lebanon, one that enables those who are most vulnerable to access a broad range of quality services.
- DESCRIPTION OF ACTION
International Medical Corps UK (IMC), along with identified consortium partners, Première Urgence-Aide Médicale Internationale (PU-AMI) and Fundación Promoción Social (FPS) have submitted a grant proposal to EU regional Trust Fund (the MADAD). In the 24-month project, IMC has been working with other consortium partners to improve access to quality primary health care, community health, and mental health services for Syrian refugees and other vulnerable populations in the regions of Beirut and Mount Lebanon, Akkar, greater Tripoli, Bekaa, and the South. Coordination and consultation with the Lebanese Ministry of Public Health (MoPH) has been critical to the success of the action.
The overall objective of the proposed intervention was to reduce the vulnerability of crisis affected populations through the provision of health and mental health services across Lebanon, with a focus on Syrian refugees. As a means to achieving this goal, IMC, together with PU-AMI and FPS, have been working to bolster the capacity of the local health system to better deliver an affordable and comprehensive package of quality health services in Lebanon that improves the physical and mental health of refugees and vulnerable members of the host population.
The below mentioned objectives reflect those mentioned in the proposal, and are the main pillars that were addressed in the baseline study conducted at the beginning of the project:
Specific Objective 1: Improved access to quality health services for Syrian refugees and other vulnerable populations in Lebanon: Under this objective, IMC/PU-AMI will provide quality primary health services to an estimated 398,090 beneficiaries throughout Lebanon by supporting 48 Primary Health Care Centers (PHCCs). The support will include a wide range of curative health services, as well as preventive health care through routine physical examinations and pediatric check-ups, growth monitoring, immunizations, and reproductive health services, including maternal health and family planning. Within the scope of this grant, the consortium members plan to expand the health coverage to Lebanese individuals who are not covered under the Universal Health Coverage (UHC) program. Lebanese beneficiaries will not need to be NPTP (National Poverty Targeting Program) holders to be covered under this project. The subsidized services will be provided in the supported facilities for refugees and Lebanese approaching the facilities who are not covered under the UHC. Through an innovative Flat Fee Model (FFM), beneficiaries will pay a small subsidized consultation fee (3,000 LBP), and then receive approved diagnostic and laboratory services at no cost.
A diagnostic referral network will be established through contracting with diagnostic centres that will be located close to PHCCs to ensure that patients have easy access to diagnostic services thus enabling patients to access affordable laboratory, x-ray, and other diagnostic services.
Specific Objective 2: Provision of community health services focusing on health promotion, disease prevention and health seeking behaviour: Under this objective, IMC/PU-AMI will reach an estimated 96,000 beneficiaries through community health services. A cadre of 96 Community Health Workers (CHWs) (two CHWs in the catchment area of each PHCC) will carry out community health outreach and education activities. These activities aim to help in the early detection and prevention of health-illnesses and complications as well as decrease the need to seek additional health care services and further financial spending on health.
Specific Objective 3: Improved well-being of Syrian refugees and vulnerable Lebanese population receiving mental health and psychosocial support services: Under this objective, ten mental health case management teams based in Community Mental Health Centers (CMHC) will provide quality, comprehensive mental health case management and psychosocial services to an estimated 7,840 beneficiaries throughout Lebanon.
- PURPOSE OF CONSULTANCY
The consultancy aims at conducting an outcome evaluation for the project Reducing Economic Barriers to Accessing Health Services in Lebanon. The purpose of this outcome evaluation is to allow IMC and EC to learn from the outcome of program and present results, conclusions, lessons learnt and further recommendations with regard to the program and the implementation of its approach.
The evaluation will be used as part of the organisational learning process aiming to replicate best practices and increase programme effectiveness. IMC will follow the findings and recommendations of this evaluation with a management response approach that will include action points to be implemented in a defined, timely period.
The outcome scope will cover the intervention’s geographic coverage in the Bekaa, South, Tripoli, Akkar, Beirut and Mount Lebanon governorates.
HEALTH targeted areas
Al Kayan Beirut
Ataa Bela Hdoud
Nazih elbizri medical center
MENTAL HEALTH targeted areas
Mental Health Clinics
Al Imane Clinic
Al Kayan Clinic
The outcome evaluation will be designed based on the project’s logframe, indicators and some pre-identified data collection methods and tools that will be adapted for the purpose of the outcome evaluation. Additionally, the evaluation will also aim to answer/address certain findings from the baseline assessment previously conducted.
The main objectives of the outcome evaluation are:
- Assess progress towards the outcomes and the factors and limitations affecting them
- Assess MHPSS beneficiaries and patient satisfaction with the services
- To document good practices, generate evidence-based lessons and provide recommendations for future projects with a similar financial model of service provision
- Assess the sufficiency of selected outcome indicators in measuring the program outcomes
Each of the above mentioned points are to be evaluated using the main OECD evaluation criteria: relevance, effectiveness, efficiency, , , sustainability, and others when relevant. Additionally, the below output-outcome links should be examined:
- Whether the project’s outputs or other organizations providing nuanced services can be significantly associated with the achievement of the outcome
- If the FFM were to be replicated in future projects, what is the stipulated timeframe that would allow for a significant association between the outputs provided and outcomes achieved?
- The extent of sustainability of the overall project implementation (capacity building of staff, financial aspects, etc.)
SCOPE OF THE EVALUATION
The evaluation will cover all activities undertaken in catchment areas throughout the project life-cycle. It will compare and assess planned against actual results at output and outcome level. The evaluation is expected to meet International OECD evaluation standards and guidelines, and at a minimum, apply the following evaluation criteria:
Relevance: (Relevance of the IMC intervention in addressing the REBAHS commitments)
- To what extent are the subsidized services relevant to the reduction the economic barriers to accessing health care services for the target population?
- To what extent are the current activities meeting the health needs in Lebanon?
- To what extent is there a need to reformulate the package design given changes in the country, health status of the population and operational context?
Effectiveness: (Achievement of programme goal, objectives and results)
- How well did the project achieve its intended outputs and outcomes with respect to indicators, given identified risks?
- Throughout project implementation, to what extent did the project performance indicate expected achievements of objectives?
- What are the critical enabling factors that are responsible for success and other inhibiting factors that have affected the project, including those beyond the control of the programmes management structures?
- To what extent did the changes in the packages implemented by the program contribute to the achievement of the intended outputs and project outcomes?
- How appropriate were the planned results and their associated indicators?
- How measurable were the project indicators?
- To which extent was the programme design adequate and effective in responding to the mental health needs in a way that guarantees results?
Efficiency: (Achievement of results in relation to costs)
- To which extent are the results being achieved at an acceptable cost, compared with alternative approaches to accomplishing the same objectives?
- How well has the programme reached the expected measures as per the targets of the results within the expected time frame?
- To which extent are the programme’s activities in line with the schedule of activities as defined in the annual work plans?
- To which extent did project implementation costs differ than budgeted project costs?
- To which extent was there enough coordination and planning among the program teams to ensure proper and timely implementation of the project? (i.e., Monthly Coordination Meetings, Technical Meetings, e.t.c.?)
Impact: (Achievement of high level results including contribution to the difference in lives of targeted beneficiaries)
- To which extent did the subsidized services contribute to the reduction the economic barriers to accessing health care services for the target population?
- To which extent did the programme lead to changes in knowledge, attitudes and access to mental health services in the targeted communities?
Sustainability (The likelihood of the programme activities and results continuing beyond IMC funding)
- How likely are programme activities and results to continue after programme funding, which ones are not likely and why?
- How well and to which extent have activities throughout the project focused on ensuring sustainability of services post REBAHS funding?
The consultant will be expected to implement the various stages of preparation, planning, implementation and analysis of the outcome evaluation. The evaluation will be expected to include, but not be limited to, the following groups:
- Targeted beneficiaries (Syrian, Lebanese, Iraqis and/or Palestinian) (women and men); to be included in quantitative and qualitative assessments
- Partners and stakeholders involved in the project at different levels:
- Implementing partners (IMC, PU-AMI, FPS): programme management and field teams.
- Targeted PHCC and Community Mental Health Centres staff members: Service providers (case managers, medical and non-medical staff)
- Community actors: community health workers, informal settlement focal points
- Government agencies: Ministry of Public Health, Ministry of Social Affairs, municipalities (as relevant to particular PHCCs)
An external consultancy firm or team of consultants will be responsible for conducting the outcome evaluation. The firm/team is expected to conduct a desk review of relevant project documents and monitoring reports/data, meet with senior management, program staff, partners, and relevant stakeholder, and participate in consultation and debriefing meetings. The firm/team is then expected to develop a detailed methodology, appropriate data collection tools, and a detailed data analysis plan. Any quantitative survey should include a sample size calculation that will render the results statistically valid. Discussions on the sample size and proposed tools are to be shared with the MEAL Team before the commencement of the actual fieldwork.
The consultant will be expected to include a mix of quantitative and qualitative data collection methods such as surveys, focus group discussions, key informant interviews, and field observation visits. Triangulation of data through different tools and sources is considered fundamental. Following is the potential list of methods that International Medical Corps expects to the consultant to use in order to measure various indicators
- Primary Health Care, Mental Health and Community Health household survey
- Training needs and sustainability assessment
- Focus group discussions with targeted communities
- Key informant interviews with PHCC management and staff, Project management teams.
- KEY TASKS
The assignment involves the participatory preparation and implementation of the project outcome evaluation. For the assignment, it is expected that the consultant will ensure participation at different levels including communities, partners, IMC, and other relevant stakeholders.
The following specific tasks are to be carried out:
- Contact with IMC and partners’ teams to:
- Gather expectations for the outcome evaluation process
- Review available resources for carrying out the outcome evaluation
- Ensure common understanding on the ToR
- Conduct a desk review of the project’s key documents including but not limited to:
- Project proposal and logframe
- Review project standard operating procedures and conduct inception meetings with project stakeholders
- Report produced from the baseline assessment
- In consultation with IMC and partners, develop a first draft inception report for the outcome evaluation (in English) including:
3.1 For each indicator, suggested data collection methods and tools.
3.2 Detailed sampling strategy per indicators as relevant
3.3 Detailed field implementation plan
3.4 Tools and template for data analysis and reporting
3.5 Data entry system and analysis plan
- Study plan for relevant review of secondary data, reports, studies, surveys, etc.
- Translate relevant tools in Arabic if needed, implement field test of tools and methods suggested and collect feedback from IMC, key partners’ staff
- Review and finalise inception report for the outcome evaluation based on feedback collected and field test findings. The inception report should include in both English and Arabic:
- For each indicator, data collection methods and tools, sampling strategy and guidance notes. Tools and guidance notes should be provided in both English and Arabic.
- Final field implementation plan with protocols for the enumerators and supervisors
- Field manual for enumerators
- Final tools and template for analysis and reporting
- Final data entry and analysis plan (in English only)
- Recruit, train, and manage the enumerators and data entry staff
- Collect data in selected communities and PHCC
- Enter, clean, and analyse data
- Provide an excel draft of the raw and clean data collected
- Submit a first draft report to IMC and partners for comments
- Finalize the report within 10 working days of receiving the final round of comments.
- THE EXPECTED OUTPUT OF OUTCOME EVALUATION CONSULTANT
- Inception report including the proposed methodology, sample size, team composition and detailed implementation plan.
- Qualitative and quantitative data collection tools
- Qualitative and quantitative data along with summary tables and statistics
- PowerPoint presentation reflecting the findings
- A clear and concise, well-written outcome evaluation report containing lists and tables, facts and figures demonstrating the state of the end-line situation and other findings at the end of the project, as well as a detailed comparative analysis of other health models implemented in similar contexts
- Final workshop with all partners on the findings of the outcome evaluation (cost for workshop venue to be covered by IMC)
Required Format for the outcome evaluation report:
The final report will be submitted to IMC and should not exceed 35 pages (excluding the annexes). The final report should be produced in English and must include at least the following:
- Cover page
- Table of contents
- List of abbreviation
- Executive Summary that should include major findings of the outcome evaluation, including indicator tables, conclusions and recommendations;
- The objectives of the evaluation
- A description and justification of the methodology used, timing of evaluation, and challenges / limitations of the analysis
- A scientific presentation of results
- Challenges, recommendations, and conclusions
Report annexes should include:
- The terms of reference for the outcome evaluation;
- The data collection tools;
- The sampling table along with the list of locations covered under the outcome evaluation;
- The composition of the team of the consultancy firm;
- List of documents and bibliography.
The consultants will be responsible for logistical arrangements such as transportation, communication, translation, management of enumerators, and outreach. IMC programme teams will provide limited support on tasks related to coordination and scheduling at field level.
IMC is expecting the completion of the consultancy tasks as described in this ToR over a maximum period of 2.5 months starting upon contract signature.
- General Conditions of the consultancy
- The consultant will agree to abide by International Medical Corps code of conduct, Child Safeguarding policy requirements. The consultant will be orientated to these policies before engaging in data collection.
- Final payment to the consultant will be dependent on the completion of all deliverables.
- Consideration of Ethics: The consultant will ensure that written consent is obtained from all participants in the study.
- Intellectual property rights:
All documentation including data related to the assignment shall remain the sole and exclusive property of IMC
 The list of targeted communities does not include all selected areas and will be updated accordingly
How to apply
- Submit a proposal to [email protected] no later than December 15th – 2019.
- A letter of interest addressed to [email protected]
- A technical offer showing the interpretation of the ToR and indicating a detailed proposed methodology, team composition and draft work plan.
- A detailed financial offer reflecting the cost in USD required for the undertaking of the work required
- Portfolio of the consultants or consultancy firm showing previous relevant experience. Examples of previous related work is highly preferable.