PSS Impact Evaluation Consultant

Under the general supervision of SAMS’ Beqaa Program Manager, and working closely with key members from the program teams in Lebanon, the PSS Impact Evaluation Consultant will be responsible for designing the evaluation approach, collecting and analyzing mixed methods data, and producing an impact evaluation report.

Overview of the Helping Hand (HH) Program

The PSS program that will be the focus of this evaluation is called The Helping Hand (HH) - a digital mental health game and blended learning program that uses cognitive behavioral techniques to provide young people with accessible and engaging socioemotional support. Syrian refugees in Lebanon are subjected to a number of stressors resulting from poverty, difficult living conditions, and traumatic experiences due to conflict or displacement. The HH program supports youth to address life challenges by providing a space to work through issues, facilitating dialogue about psychosocial topics that are often challenging to discuss, and changing unhealthy thought patterns or behaviors. The program is primarily implemented among displaced communities through community centers, schools, and informal tented settlements.

SAMS used funding from Grand Challenges Canada (GCC) to successfully complete five cycles of the HH program from 2021-2022, and used private funds to extend the program through 2023 given the high demand from partners and participants. Analysis of pre- and post-test data show that the program was effective in the short-term - particularly during the COVID-19 pandemic - and helped reduce anxiety and depression among participants by 92%. An impact evaluation is needed to determine if these positive effects persisted after the program ended and what, if any, larger impacts have been created by the program.

Suggested Evaluation Questions

1. Did the HH program contribute to sustained improvements in mental health and wellbeing for participants?
2. Did the HH program’s effects spread beyond the participants?
3. Did the HH program produce any unexpected outcomes (positive or negative)?
4. Which components of the HH program were most impactful?
5. What factors helped or hindered the HH program in achieving the desired goals?

Methods: The consultant should select qualitative and quantitative methods to answer the key questions outlined above. The methods should engage SAMS prior program participants and facilitators.

Locations of Activity: The work will primarily be conducted in the Bekaa Valley and Arsal, Lebanon through coordination with SAMS offices in Bekaa and Beirut.

Supervision: The evaluator is expected to work closely with SAMS focal points and support teams in Lebanon, and the MHPSS Specialist at HQ who will oversee implementation of the evaluation from the beginning through to completion.

Key Responsibilities and Deliverables:

Task

Deliverable

Desk review: Acquire an understanding of the HH program and implementation, and propose a design for evaluating its impact on participants

Evaluation design with proposed methods and work plan

Data Collection: Ensure the collection of quantitative and qualitative tools through developing collection methods that may include, but not limited to: surveys, interviews, KPIS, FGDs

Assessment tools that will be used to collect data (e.g. questionnaires, interview questions)

Data analysis: Ensure proper analysis through triangulating the qualitative and quantitative data.

Raw datasets from questionnaires, interviews and other methods used

Report: Draft a comprehensive report on the HH program’s impact, including results from data analysis, discussion of key evaluation questions, and recommendations for strengthening the program and/or its implementation

Final impact evaluation report, submitted in English

Review of MEAL Tools: Review the measures and indicators used in the HH program and offer feedback or alternative tools and indicators. 

Recommendations for improving HH metrics and MEAL indicators

Desired Qualifications and Skills

• Extensive background in psychology, social sciences, MHPSS, or a related field
• Demonstrated experience in designing and conducting impact evaluations in MHPSS-related fields.
• Familiarity with the humanitarian context in Lebanon, particularly related to refugees
• Experience working with vulnerable populations and understanding of ethics
• Strong command of statistics and data analysis
• Fluency in English and Arabic required
• Advanced English writing skills.

A Detailed TOR will be provided based on the interview outcome.