PROJECT EVALUATION

Brief Presentation of the project

The project has three main outcomes:

 

Outcome 1: Service provision through establishing and operating two Community Mental Health and Substance Use Centers (CMH-SUC) in Beirut and Baalbek, as well as provide Mental Health Awareness and Sensitization in four selected Primary Health Care Centers (PHCCs) in Tripoli.

Main activities under this outcome:

1. Establish and operate two CMHC-SUCs in Beirut and Baalbek
2. Build a network of PHCCs in Tripoli who is sensitized on MHPSS
3. Conduct assessment to assess needs and gaps in MH and SU in Baalbek and Tripoli
4. Conduct assessment to assess the level of understanding, stigma and acceptance of MH among hospital staff and PHCCs + organize sessions with health staff on identified topics
5. Integrate MH and SU services within the same center
6. Provide appropriate and focused MHPSS and SU sessions (one on one session, focus groups etc.)
7. Capacity building of staff on MH and SU, as well as other cross cutting issues (gender, GBV, inclusion etc.)
8. Deliver workshops and awareness sessions on suicide prevention
9. Ensure the operation of Embrace lifeline, the national support life for suicide prevention
10. Conduct assessments to see the level of inclusion and gender sensitive approaches within organizations and services delivered
11. Promote the creation of an effective referral pathways among stakeholders
12. Organize and deliver mhGap trainings

 

Outcome 2: Outreach activities and service users’ participation through organizing activities with community members in Beirut, Tripoli and Baalbek.

Main activities under this outcome:

1. Recruit, train and build the capacities of outreach workers
2. Conduct outreach activities in Beirut, Baalbek and Tripoli
3. Launch 5 micro-grant calls to support community initiatives
4. Develop communication materials
5. Conduct MH and SU prevention activities in schools
6. Conduct psychological support groups with community members
7. Organize two social cohesion events
8. Organize three peace building workshops
9. Conduct Psychosocial Support activities focused on Child protection, in North Bekaa, to parents and their caregivers

 

Outcome 3: Advocacy and media activities through conducting a number of research/studies and communication activities to improve knowledge and awareness on mental health and substance use

Main activities under this outcome:

1. Develop one technical document on integrated, quality and inclusive community mental health center
2. Conduct two studies on gender issues and how they relate to MHPSS and SU access
3. Support NMHP in their yearly MH campaign
4. Organize two roundtables on implementation of SU law targeting judicial stakeholders
5. Produce two videos for Anti-stigma campaign, ensuring media outlets utilize appropriate terminologies when reporting on MHPSS and SU topics
6. Train journalists on appropriate terminology guideline when reporting on MH and SU
7. Support the creation of national service user association
8. Advocate with relevant stakeholders to include substance use in existing MHPSS taskforces at the national and local levels

Objectives of the evaluation

The purpose of this evaluation is to assess the level of achievement and result of the current project intervention, taking into consideration the context, and more specifically:

1. To what extent did the project meet its objectives, and implement the activities according to the original timeline, budget and target set, taking into consideration the prevailing context in Lebanon and administrative processes of concerned authorities/stakeholders?
2. How did the consortium partners ensure its continuous delivery of quality services to meet the needs of vulnerable populations? (How did the consortium adapt to the situation in terms of activity implementation?)
3. How can the consortium partners further adapt their activities (service provision, trainings and coordination) to better meet the needs of vulnerable populations in Lebanon as well as meet the targets set at the start of the project?

 

This evaluation will review MdM’s and the consortium approach in terms of results, activities, lessons learned and recommendations for the project period. It will also be used to learn and to formulate recommendations useful for MdM and the consortium partners to adjust their strategies and project activities as needed, or for other countries where MdM operates in the field of MH; to improve implementation and to identify best practices and lessons learned.

 

Scope of the Evaluation

The project evaluation (desk review, data collection, analysis and report writing) will be done over a period of six months (February 01, 2021 – July 31, 2021). The evaluation will take place at the project implementation sites (Beirut, Baalbek, Tripoli) and partners’ offices in Beirut. The evaluation will focus on the achievements of the project, compared to the project narrative, log frame, timeline and budget agreed on with the donor.

 

Target audience

The evaluation report will be shared internally at MdM Lebanon mission and HQ level, as well as with the implementing members of the consortium and the donor agency, AFD. The report will also feed into to the final report of the project that will be shared with AFD.  

 

Essential criteria of the evaluation

Relevance

From this evaluation, the consortium wishes to assess to what extent the initial outcomes, objectives and activities of the project are in line with the immediate needs of the population it wishes to serve, considering the change in context:

• Is the action of establishing and operating community mental health and substance use centers a priority to the emerging needs of the people living in Lebanon?
• Is the action of training staff and building their capacities on Mental Health and Substance Use, as well as on cross cutting issues (gender and disability) a priority considering the current situation of recurrent road closures and social distancing?
• Is the action of organizing social stability events and peace building workshops a priority in the current context?
• Is the action of establishing a Mental Health service user’s association a priority in Lebanon?

Effectiveness

From this evaluation the consortium wishes to assess the degree of achievement of the expected results of the intervention:

• Did the population concerned have access to sufficient quality and quantity of care?

Sustainability

From this evaluation, the consortium wishes to analyze the chances that the actions will continue after external aid has ceased:

• To what extent are the hospital directors willing to support the continuation of the two community mental health and substance use centers, after the current funding will end?
• To what extent will the mental health service user association continue to exist after the current funding will end? Are the members willing to continue working towards the mission of the association?

Efficiency

From this evaluation, the consortium wishes to evaluate the economic/financial efficiency of the project:

• Are the results obtained been at an acceptable cost?
• What else could be done to improve the implementation of the project in order to maximize the impact at an acceptable cost?

Impact

From this evaluation, the consortium wishes to evaluate, from a broader sense (technical, economic, financial and social), the long term effects, positive and negative, expected and unexpected outcomes induced by the project:

• Did the stigma and discrimination against mental health and substance use decrease in the areas of implementation of the project?
• Do community members in the areas of implementation know more about mental health and substance use?
• Did the integration of mental health and substance use within one center create a pilot model for the National Mental Health Program to build upon?

 

Other criterias of the evaluation

From this evaluation, the consortium wishes to evaluate the additional below criteria’s:

• Partnership: The degree of involvement of each partner in the consortium meet the original expectation of each organization? What can be done to improve the partnership and better meet the needs of the communities we serve? Was there enough transparency? Did a delay in a partner activity caused delays in other partner’s activities or action plan?
• Coordination: Were the interventions of different actors harmonized and complementarity, rather than duplicated? Was there enough level of coordination between the consortium members at an operational level? What can be done to improve the coordination?
• To what extent did the road blockages have an impact of the activities planned under this project ?

Methodes of data collection

The evaluation is expected to be a mix method approach, using qualitative (semi-structured individual interviews, focus groups and observations) and quantitative (questionnaire) tools to collect data.

Documents to be used

At the start of the consultancy, the consortium will share with the consultant the below:

• Initial project proposal, action plan, log frame and budget
• Amended project proposal, action plan, log frame and budget, and
• Semi-annual reports shared with the donor.

Stakeholders

Data will be collected from beneficiaries, partners of the consortium, staff of the community mental health and substance use centers and members of the service user association. Initially identified stakeholders:

• Focal point from each of the consortium members (6 individuals)
• Two representatives from the established national service user association
• One focal point from each of the operating CMH-SUC under this project (3 individuals)
•  One representative from each of the two hospitals where the CMH-SUC are established
• Two representatives from the primary health care centers supported in Tripoli
• Sample of beneficiaries (to be discussed with the external consultant)
•  One representative from the NMHP
• Other stakeholders as identified by the external consultant during the preparation phase of the consultancy

 

The activity should be carried out between February 01, 2021 and July 31, 2021.

 

A tentative plan is suggested below; however, this can be amended to how to external consultant can best see fit:

Preparation phase (project briefing, desk review and preparation of tools for data collection): between February 01, 2021 and February 28, 2021;

Data collection phase (data collection and analysis): between March 01, 2021 and May 31, 2021;

Drafting phase of the provisional report: between June 01, 2021 and June 30, 2021; and

Restitution and finalization phase: between July 01, 2021 and July 31, 2021.

 

The indicative schedule is subject to change at any time depending on the evolution of the context and the security situation.

 

The final report of this evaluation, in Word format, should be between 40 and 50 pages (not including annexes), 12 character, single spaced, and should include the following parts:

• Executive summary (5 pages’ maximum)
• Introduction
• List of acronyms
• Context (including a description of the project)
• Objectives of the evaluation and criteria selected
• Methodology and limitations
• Main results and analysis
• Conclusion and recommendations (classified according to the criteria’s evaluated)
• Annexes: ToR’s, list of stakeholders interviewed, data collection tools etc.

Logistic and administrative organization

Equipment

A laptop can be made available to the evaluator/consultant, upon request. The consultant can use MdM offices in Lebanon to work and schedule interviews.

Means of communication

The consultant is expected to use their own mobile/phone number for work purposes. However, when the consultant is at MdM offices in Lebanon, local landline and internet can be used.  

Travel and accommodation

The consultant is responsible for their own movement and accommodation for this mission. The consultant is expected to have their personal health insurance coverage.  

Others

The final report needs to be submitted in English only. However, during the data collection, some tools will be translated to Arabic; this will be the responsibility of the consultant to arrange.

 

Security

The consultant will be added to MdM Lebanon security group, so that s/he receive updates on the security situation. However, the consultant will be responsible to take decisions regarding their movements.

 

Managing and reporting

The Consortium Manager at MdM will be the focal point for the consultant in all matters related to contract, finance, logistic and other administrative issues. The consultant should provide the consortium manager, every two months, a written update of the evaluation, listing completed activities, ongoing activities, challenges and way forward. However, the consultant can reach out to the consortium manager at any time, especially when challenges arise.  

 

A detailed budget suggested by the consultant, using the below format, should be shared along with the technical offer submitted by the consultant (Monetary amount suggested by the external consultant are “All tax included”). It is not mandatory to add all the budget lines, and it is possible to add new budget lines, however, the same template should be used.

Simplified budget for the project evaluation :

 

 

 

 

 

 

 

Number of days

Unit cost in USD

Total USD

Remarks

Fees

      

      

     

      

Transportation

      

      

      

      

Translation

     

     

     

     

Communication

      

      

     

      

Other (specify)

      

      

      

      

TOTAL TTC

      

      

     

      

The vacancy is open to a single or group of consultants;

 

Technical skills and qualifications

• Minimum of 6-8 years of proven experience on humanitarian programming and evaluation,
• Knowledge of the local context in Lebanon is required,
• Professional experience working with non-governmental organization is desired, with focus on mental health programming and public health,
• Professional experience in conducting similar evaluation/research,
• Experience in stakeholders’ analysis techniques,
• Fluent in Arabic, English (spoken and written),
• Knowledge of data analysis software is desired, and
• Reporting and analytical skills.

Soft skills

• Able to communicate with mental health and substance use service users in a sensitive and informed manner,
• Able to handle tight deadlines in a professional manner,
• Communicative,
• Organized and timely, and
• Creative in overcoming the barriers present in humanitarian contexts.

Female participants and people with disabilities are highly encouraged to apply.

The consultant(s) is invited to submit a file of not more than 10 to 15 pages, including the following documents, to [email protected] with the following title in the email subject «Project evaluation-Improving Mental Health and Wellbeing of People Living in Lebanon»:

 

Technical proposal including:

• Understanding of the terms of reference,
• The technical approach suggested and the detailed methodology,
• The constitution of the team (if applicable), the distribution of responsibilities between its members (if applicable), the CVs offered and the availability of members,
• The provisional schedule of the evaluation as predicted by the consultant,
• Contact information of two previous references of similar works (references will not be contacted before getting approval from the candidates), and
• A declaration of honor attesting to the absence of conflict of interest.

Financial proposal including:

• The total budget including all taxes and the budget distribution (see template above).