AFD ReCEVAL_Mid-term project evaluation_CMH-SUC Tripoli

Brief Presentation of the project

The project has three main outcomes:

 

Outcome 1: An integrated and gender-sensitive mental health and substance use service system is available in Beirut, Baalbek and Tripoli areas and offers quality, gender-specific and focused care to women, men, boys, girls and non-binary individuals dealing with MHPSS and SU problems, in accordance with the care model defined by the National Mental Health Program strategy

Main activities under this outcome:

  1. Conduct well-being survey to assess the number of MH service users communicating improvement in their well-being at the end of their treatment
  2. Track the substance use cases communicating improvement in their well-being during the treatment process
  3. Continue the operation of the CMH-SUC in Beirut (RHUH), under the umbrella of the NMHP
  4. Assess and select a facility to relocate the existing CMH-SUC in Baalbek
  5. Assess and select a facility to establish a new CMH-SUC in Tripoli
  6. Assess and select PHCCs to be part of the integration of MH into Primary HealthCare Centers (PHCC)
  7. Deliver direct mental health and substance use services through the 3 CMH-SUCs
  8. Deliver indirect mental health services through the PHCCs staff
  9. Conducted satisfaction survey with PHCC patients
  10. Conducted satisfaction survey with CMH-SUCS patients
  11. Set a proper contingency mechanism for Opioid Substitution Treatment (OST) program program
  12. Sustain the Embrace Clinic capacity to offer psychiatric consultations to beneficiaries through the Mobile Medical Unit
  13. Operation of the National Emotional Support and Suicide Prevention Helpline
  14. Manage mental health or suicidal emergencies through responding to MH calls received by Embrace lifeline, red cross and ISF
  15. Integrate gender specific inputs in program tools
  16. Capacity building of CMH-SUCs and PHCCs staff in Safe Identification and Referrals
  17. Strengthen patient pathways in CMH-SUCs and PHCCs for Gender-Based Violence (GBV) cases specifically cases specifically

 

 

Outcome 2: Capacities and knowledge of community members and service users are strengthened with gender-sensitive and inclusive information on mental health and substance use services and rights to treatment to increase effective use of MHPSS and SU services by women, men, boys, girls and non-binary individuals in the areas of intervention

Main activities under this outcome:

  1. Recruit Community Outreach Volunteers (COV) and Street Workers (SW)
  2. Provide training and capacity building to COVs and SWs on MHPSS, SU and gender biases
  3. Deliver MH awareness sessions by MdM staff and PHCCs staff in PHCC waiting area
  4. Deliver awareness sessions in the community by COVs adapted to men, women, boys and girls, and non-binary individuals needs
  5. Deliver street interventions in the community by SWs adapted to men, women, non-binary individuals needs
  6. Conduct Anonymous Voluntary Counselling and Testing (VCT) for the public (Y1 Skoun in Beirut, Y2 Skoun to train MdM outreach + expand to Tripoli and Baalbek)
  7. Plan and participate in national and international harm reduction and drug use and MH campaigns, festivals, open doors etc.)
  8. Organize a ToT on drug prevention and awareness and protective skills for CSOs frontliners and case workers working with refugee youth (Syrian/Palestinian and other)
  9. Production of an animated video on knowledge on substances and substance use, signs of use and indicators, intervention and parenting skills etc.
  10. Deliver Know Your Rights (KYR) training for SU patients
  11. Deliver KYR/legal aid training for CMH-SUC staff
  12. Referral to Skoun for treatment by the drug addiction committee
  13. Build the capacities of lawyers
  14. Build the capacities of law students
  15. Lead the advocacy efforts of a coalition with the aim of enacting legislative reforms towards the decriminalization of substance use in Lebanon

 

Outcome 3: Capacity building and on-the-field supervision of practices to ensure quality services and to strengthen the capacities of mental health CSOs, utilizing a gender approach to tackle gender-inequalities and gender discrimination that hinder access to and provision of MHPSS and SU services

Main activities under this outcome:

  1. Provide training on MHPSS, SU and gender related access barriers and bias in service provision
  2. Deliver training and clinical supervision for the CMH-SUC teams
  3. Deliver mhGAP training and supervision to PHCCs staff
  4. Deliver support and capacity building sessions to PHCCs staff
  5. Supervise Embrace Clinic graduate trainees building the capacity of mental health professionals
  6. Deliver capacity building workshops with consortium partners and PHC/CMH- SUCs staff on active listening, de- escalation and the management of suicide crises
  7. Provide technical support to all partners to integrate gender in their daily practices
  8. Accompany all staff in their gender beliefs to increase the quality of their practices
  9. Creation of gender inclusive environment in all of partners' offices and CMH-SUCs targeting men, women, non-binary individuals and LGBTQI+ individuals
  10. Develop awareness among the civil society and general public about health and gender
  11. Develop factsheet, technical documentation and media on gender and MH, and gender and SU for wider dissemination

 

Based on a successful model built in Beirut, then in Beqaa, MdM and Skoun opened a joint Community mental health and substance user Center in Tripoli as of September 2022. The center, through its multidisciplinary team composed of psychologists, social workers, a psychiatrist, and nurses, provides specialized mental health and substance use services including:

-           Case management;

-           Individual psychotherapy consultation;

-           Family intervention and psychoeducation;

-           Referral to other assistance (social, educational and financial assistance outside of the center);

-           Psychiatric assessment, diagnosis, treatment and follow up;

-           Urine test for people who use substances;

-           Voluntary counselling and testing service (HIV, Syphilis, Hepatitis B and C);

-           Follow up by the nurse on opioid substitution therapy medication;

-           Urgent hospitalization, including for detox program.

In addition, MdM supports 3 PHCCs in Tripoli in integrating mental health in the staff practices, and providing mental health services for low cases, and referrals to specialized services. Referrals are provided internally through the PHCC, El Rahma Special needs center and Hospital. Referrals are also done from the center to El Rahma PHCC for beneficiaries with a prescription of psychotropic medications or for the ones needing blood tests or any medical assistance. MdM and Skoun increase awareness in the community on mental health and substance use through outreach volunteers.

Objectives of the evaluation and target audience

Objectives of the evaluation

The purpose of this evaluation is to assess the level of achievement and results of the current project intervention, taking into consideration the context, and more specifically

  1. To what extent are the activities conducted according to the original timeline, budget and target set, taking into consideration the prevailing context in Lebanon and administrative processes of concerned authorities/stakeholders.
  2. How did the consortium partners ensure its continuous delivery of quality services to meet the needs of vulnerable populations in Tripoli?
  3. How can the consortium partners further adapt their activities (service provision, training, outreach and coordination) to better meet the needs of vulnerable populations in Tripoli area?

 

This evaluation will review MdM’s and the consortium approach in terms of results, activities, lessons learned and recommendations for the project period and more specifically for Tripoli’s activities. It will also be used to learn and to formulate recommendations useful for MdM and the consortium partners to adjust their strategies and project activities as needed, or for other countries where MdM operates in the field of Mental Health and/or Substance Use; to improve implementation and to identify best practices and lessons learned.

Scope of the Evaluation

The project evaluation (desk review, data collection, analysis and report writing) will be done over a period of three months (April 01, 2023 – June 30, 2023). The evaluation will take place in Tripoli and partners’ offices in Beirut. The evaluation will focus on the achievements of Tripoli’s activities in the CMH-SUC and the PHCCs of the city where the consortium is implementing Mental Health integration as well as outreach activities (from August 2022 up to February 2023), compared to the project narrative, log frame, timeline and budget agreed on with the donor.

Target audience

The evaluation report will be shared internally at MdM Lebanon mission and HQ level, as well as with the implementing members of the consortium and the donor agency, AFD. The report will also feed into the next interim report of the project that will be shared with AFD.  

Evaluation criteria

Essential criteria of the evaluation

Relevance

From this evaluation, the consortium wishes to assess to what extent the initial outcomes, objectives and activities of the project are in line with the immediate needs of the population it wishes to serve in Tripoli, considering the current context:

  • Is the action of establishing and operating community mental health and substance use center in Tripoli a priority to the emerging needs of the people living in Lebanon?
  • Is the action of training staff and building their capacities on Mental Health and Substance Use, as well as on cross cutting issues, a priority considering the current situation?
  • Are the awareness activities done in the community by the Outreach Volunteers on Mental Health and Substance Use relevant and a priority in the current context?
  • Are the capacity-building activities of PHCCs staff on Mental Health relevant and a priority in this context?

 

Coherence

  • Is the action of Mental Health integration into PHCCs aligned with the National Mental Health Program’s national model of care?
  • Were the interventions of different actors in the area harmonized and complementary, rather than duplicated?

 

Effectiveness

From this evaluation the consortium wishes to assess the degree of achievement of the expected results of the intervention:

  • Did the population concerned have access to sufficient quality and quantity of care in Tripoli?
  • Is the connection between outreach volunteers and the CMH-SUC efficient (in terms of referrals to the CMH-SUC’s services, increasing awareness of the center’s services, accessibility to the services)?
  • Did the community members increase their knowledge and awareness on MH and SU issues, including on the available services?
  • Do the activities, done in the community and in the PHCCs, improve the access to the CMH-SUC?
  • Are the different training provided by the consortium partners effective in building the capacity of CSOs on MH, SU, management of suicide crisis, etc ?
  • Are the training provided to the outreach volunteers effective in increasing the awareness related MH and SU of the community?
  • Are community outreach activities (including VCT services) reducing the stigma on Mental Health and Substance Use in the community?
  • Are the PHCCs staff more equipped to provide services to persons with mental health problems?
  • To what extent did the medication shortages, OST unavailability and the inflation have an impact on the activities planned under this project and on the beneficiaries’ satisfaction of those?

 

 

Sustainability

From this evaluation, the consortium wishes to analyze the chances that the actions will continue after external aid has ceased:

  • To what extent are the hospital directors willing to support the continuation of the community mental health and substance use center, after the current funding will end?
  • To what extent will the mental health and substance use services continue to exist after the current funding will end?
  • To what extend will the integration of mental health in the PHCCs be sustainable beyond the scope of the project?
  • To what extend the project’s exit strategy for Tripoli’s CMH-SUC can be reinforced to ensure its sustainability?
  • To what extent are the activities led in the community and the PHCCs designed in a way to promote the sustainability of the project?

 

Efficiency

From this evaluation, the consortium wishes to evaluate the economic/financial efficiency of the project:

  • Are the results obtained been at an acceptable cost and in a timely way in Tripoli?
  • What else could be done to improve the implementation of the project in order to maximize the impact at an acceptable cost?

Impact

From this evaluation, the consortium wishes to evaluate, from a broader sense (technical, economic, financial and social), the long-term, positive and negative, expected and unexpected effects to which the activities implemented in Tripoli area participate:

  • Did the stigma and discrimination against mental health and substance use decrease in particular for the beneficiaries of the services in Tripoli?
  • In what way the CMH-SUC direct services impacted the patients and their families regarding to their daily lives and well-being?
  • To what extend community members in Tripoli who received awareness about MH and SU changed their perception about MH and SU in general? From which media did they receive awareness about MH and SU?
  • In what way did the integration of Mental Health in PHCCs impact the patients’ quality of care?
  • Did the integration of mental health and substance use within the Tripoli center create a pilot model for the National Mental Health Program to build upon?
  • Did the activities implemented in Tripoli have had any negative or unintended impact on the beneficiaries or the community so far?

 

Other criteria of the evaluation

From this evaluation, the consortium wishes to evaluate the additional below criteria’s:

  • Partnership: The degree of involvement of each partner in the consortium meet the original expectation of each organization? What can be done to improve the partnership and better meet the needs of the communities we serve? Was there enough transparency? Did a delay in a partner activity cause delays in other partner’s activities or action plan? Was there enough level of coordination between the consortium members at an operational level? What can be done to improve the coordination?
  • To what extend did the lessons learnt and recommendations of the first phase project’s evaluation have been taken into account and followed for Tripoli’s CMH-SUC?
Preferred methodology

Methods of data collection

The evaluation is expected to be a mix method approach, using qualitative (semi-structured individual interviews, focus groups and observations) and quantitative (questionnaire) tools to collect data.

In its technical proposal, it is recommended that the consultancy integrates indicators for each assessment method in order to list observable, and measurable characteristics that demonstrate the progress in the evaluation, including but not limited to:

  • An estimation of the number of focus group discussions conducted with beneficiaries, staff, etc and when with beneficiaries, what would be the selection criteria
  • An estimation of the number of surveys that should be done (sampling methodology)
  • A small description on how the observations, interviews, discussions will be measured and documented

 

Documents to be used

At the start of the consultancy, the consortium will share with the consultant the below:

  • Initial project proposal, action plan, log frame and budget
  • Tripoli’s CMH-SUC assessment
  • Amended project proposal, action plan, log frame and budget
  • Semi-annual reports shared with the donor
  • Evaluation report and tools from the first phase of the project (Beirut and Bekaa)

 

Stakeholders

Data will be collected from beneficiaries, outreach volunteers, partners of the consortium, staff of the community mental health and substance use center in Tripoli. Initially identified stakeholders:

  • Focal point from each of the consortium members (6 individuals)
  • One focal point from Tripoli’s CMH-SUC under this project (Supervisor)
  • One representative from the hospital where the CMH-SUC is established
  • Two representatives from the primary health care centers supported in Tripoli
  • Sample of beneficiaries (to be discussed with the external consultant)
  • One representative from the NMHP
  • Other stakeholders as identified by the external consultant during the preparation phase of the consultancy

calendar

The activity should be carried out between April 20, 2023 and July 20, 2023.

 

A tentative plan is suggested below; however, this can be amended to how to external consultant can best see fit:

Preparation phase (project briefing, desk review and preparation of tools for data collection): between April 20, 2023 and May 4, 2023;

Data collection phase (data collection and analysis): between May 04, 2023 and June 05, 2023 including 10 working days minimum on the field;

Drafting phase of the provisional report: between May 22, 2023 and June 22, 2023; and

Restitution and finalization phase (including half a day for a restitution workshop): between June 23, 2023 and July 20, 2023.

 

The indicative schedule is subject to change at any time depending on the evolution of the context and the security situation.

 

Deliverables

Inception report

An inception report will be produced at the start of the mission by the evaluation team after having reviewed the key documents and the initial interviews. Amongst other things, the inception report defines and formalises the planned methodology and the evaluation matrix, sets out the necessary data collection instruments (interview guides, etc.) and suggests a realistic work plan with a detailed and final calendar. The inception report will be submitted to the Steering Committee for comments and validation during the inception meeting

 

Evaluation report

 

An interim version of the final evaluation report will be produced at the end of the field phase. It will be submitted to the Steering Committee for comments and discussion. The final evaluation report must take into account the comments made by the Steering Committee on the interim report.

 

The final report of this evaluation, in Word format, should be between 40 and 50 pages (not including annexes), font size 11 and simple line spacing, and should include the following parts:

  • Executive summary (5 pages’ maximum)
  • Introduction
  • List of acronyms
  • Context (including a description of the project)
  • Objectives of the evaluation and criteria selected
  • Methodology and limitations
  • Main results and analysis
  • Lessons learnt
  • Conclusion and recommendations (classified according to the criteria’s evaluated)
  • Annexes: ToR’s, list of stakeholders interviewed, data collection tools, observation grids, etc.
Organization of the evaluation mission

Logistic and administrative organization

Equipment

A laptop can be made available to the evaluator/consultant, upon request. The consultant can use MdM offices in Lebanon to work and schedule interviews.

Means of communication

The consultant is expected to use their own mobile/phone number for work purposes. However, when the consultant is at MdM offices in Lebanon, local landline and internet can be used.  

Travel and accommodation

The consultant is responsible for their own movement and accommodation for this mission. The consultant is expected to have their personal health insurance coverage.  

Others

The final report needs to be submitted in English only. However, during the data collection, some tools will be translated to Arabic; this will be the responsibility of the consultant to arrange.

Security

The consultant will be added to MdM Lebanon security group, so that s/he receive updates on the security situation. However, the consultant will be responsible to take decisions regarding their movements.

Managing and reporting

The Consortium Coordinator at MdM will be the focal point for the consultant in all matters related to contract, finance, logistic and other administrative issues. The consultant should provide the consortium coordinator, every month, a written update of the evaluation, listing completed activities, ongoing activities, challenges and way forward. However, the consultant can reach out to the consortium coordinator at any time, especially when challenges arise.  

Budget available for this evaluation

A detailed budget suggested by the consultant, using the below format, should be shared along with the technical offer submitted by the consultant (Monetary amount suggested by the external consultant are “All tax included”). It is not mandatory to add all the budget lines, and it is possible to add new budget lines, however, the same template should be used.

Skills required/desired to carry out the evaluation

The vacancy is open to a single or group of consultants;

 

Technical skills and qualifications

  • Minimum of 6-8 years of proven experience on humanitarian programming and evaluation,
  • Knowledge of the local context in Lebanon is required, knowledge of Tripoli’s context specifically is preferred
  • Professional experience working with non-governmental organization is desired, with focus on mental health programming and public health,
  • Professional experience in conducting similar evaluation/research,
  • Experience in stakeholders’ analysis techniques,
  • Experience related to gender, especially gender and mental health and/or gender and substance use issues is preferred,
  • Fluent in Arabic, English (spoken and written),
  • Knowledge of data analysis software is desired, and
  • Reporting and analytical skills.

Soft skills

  • Able to communicate with mental health and substance use service users in a sensitive and informed manner,
  • Able to handle tight deadlines in a professional manner,
  • Communicative,
  • Organized and timely, and
  • Creative in overcoming the barriers present in humanitarian contexts.

Female participants and people with disabilities are highly encouraged to apply.

How to apply

HOW TO SUBMIT AN APPLICATION

 

The consultant(s) is invited to submit a file of not more than 10 to 15 pages (excluding CVs), including the following documents, to [email protected] with the following title in the email subject “Lebanon_Mid-term project evaluation- Improving mental health and well-being of all women, men, boys, girls and non-binary individuals living in Lebanon”:

 

1. Technical proposal including:

  • An understanding of the terms of reference;
  • The technical approach developed and the detailed methodology;
  • Details regarding team members, how responsibilities are shared between them, the CVs proposed and the availability of the team members as well as the legal status of the members of the consultancy team and/or evaluation firm to which the consultant(s) belongs;
  • The provisional calendar for the mission and an estimate of charges per person, per day;
  • References from 2 similar projects previously completed.

2. Financial proposal including:

  • A detailed itemized breakdown (including MOF tax 7,5% consultancy tax & VAT if applicable) incorporating the budget allocation (fees, transportation, translation, communication, and other fees to be specify).
  • The offer could also include extra services not mentioned above with the detail’s fees
  • Terms of payment and payment modality.
  • Proposal validity.
  • All contractual conditions should be also clearly explained in the offer.
  • All relevant information or deemed of value to MdM.

 

Deadline to submit the file:  March 30, 2023 – 23 :55 pm, Beirut time

Applications with missing documents will not be considered.

 

Expired
Deadline
Thursday, 30. Mar 2023
Type of Call
Call for Consultancies
Intervention Sector(s):
Health, Mental Health
Remuneration range:
> 6000 (USD)
Duration of Contract:
The activity should be carried out between April 20, 2023 and July 20, 2023.
randomness