Consultancy: Evaluation of Oxfam’s Community Based Protection Approach in Lebanon

Terms of Reference-Evaluation of Oxfam’s Community Based Protection Approach in Lebanon.

Project locations: North Bekaa

Intervention Lifespan: From 2013 till January 2021 (The evaluation would mainly cover the period from February 2017 until January 2021 as the last CBP review was conducted in 2017)

Evaluation to be conducted: Within one month Between January and March 2021

1. Introduction:

Oxfam in Lebanon responds at local, national and international levels to “ensure that women and men are protected and empowered to enjoy their basic rights and access services enabling them to live in dignity, within a more equitable society.” To achieve this vision, Oxfam in Lebanon implements activities under three main programmatic pillars: A Humanitarian Pillar, a Human Rights and Economic Justice Pillar, and the Beirut Response Pillar. The purpose of the Humanitarian Pillar is to ensure that marginalized women and men, including refugees from Syria and Lebanese, are able to have safe access to water, sanitation and hygiene promotion (WASH), attain their rights to life, security, protection and assistance, and that they are enabled to support their own basic needs, develop their livelihoods, all in a manner that respects and maintains their dignity. Oxfam implements program activities through a community-based approach and in partnership with specialized local organizations providing essential and life-saving services. Oxfam in Lebanon started supporting the establishment of community structures in urban areas and informal tented settlements of refugees from Syria since 2013[1] in both the Bekaa and Tripoli areas. Community-based protection lies at the heart of Oxfam’s work; it recognizes that some of the most important protection actions are those undertaken by affected people themselves – particularly in the immediate face of violence or abuse. It builds on existing or potential capacities and recognizes people at risk as having agency to play a central and leading role in determining how others assist them in the risks they face. These actions may take the form of organizational capacity; developing emergency protection plans and strengthening the community’s leading role through knowledge sharing, capacity building and financial support (whenever relevant).

Oxfam is seeking a consultant to undertake during the first 2 months of its DANIDA funded project, a full and comprehensive review of the community-based approach, particularly the CBP approach, including within a COVID-19 context.

2. Objectives of the evaluation:

The primary objective of the evaluation is to provide an assessment of the overall effectiveness of the approach and of its’ individual components. The evaluation is expected to focus on the community-based approaches that have been used across the various components of the humanitarian projects in North Bekaa, with the aim of informing the way forward for more effective approaches especially within the COVID19 context. The evaluation process is expected to enrich our learning process, and to support strengthening of future programming by Oxfam and partners. The evaluation will have to focus on areas of improvements, highlight best practices and help Oxfam and partners to modify and use innovative approaches when working with community groups and structures. The evaluation shall specifically seek to achieve the following objectives:

1. To assess the extent to which the community-based approach has delivered against its objectives and expected results.

2. Highlight CBP successes and shortcomings, areas of improvement and ways of moving forward within the current context (Economic Crisis, COVID19 context…).

3. Clarify the extent to which the CBP approach was adopted through a gender lens and ensured the meaningful participation of women and men. This should include an analysis of the challenges to such participation, especially that of women, and of the ways in which Oxfam can support women to initiate and lead mixed groups and/or women’s groups.

4. To draw key lessons from the program and incorporate them in recommendations that will help inform the design and implementation of future CBP interventions by Oxfam and partners and approaches with different community groups. The scope for examination is determined in line with Oxfam’s Evaluation policy, and relevant evaluation OECD-DAC criteria (relevance, appropriateness, effectiveness, representativeness and sustainability) are associated with several key questions that are to be addressed and explored.

5. Sustainability of the approach and the kind of risks community-based structures are exposed to because of the work they do and the role they play in the CBP approach.

3. EVALUATION CRITERIA AND KEY QUESTIONS TO BE ADDRESSED IN THE EVALUATION:

The questions to be addressed by this evaluation are expected to be further modified or refined in consultation with the consultant during the inception phase of the evaluation process.

Relevance and Appropriateness:

• How relevant and appropriate did the CBP approach remain given the changes in the context especially during the last year? To what extend was the work with community representatives adapted to fit the changes in the context? What were the main gaps, good practices and lessons learnt that could be highlighted? 
• To what extent was the community-based approach relevant to the objectives of the
• programme?
•  How do communities (women, men, girls and boys and other marginalized
• groups), partners and other stakeholders value the CBP intervention?
• Which strategies have proven to be more influential to what CBP aims to achieve?

Effectiveness:

• To what extent, did the community-based approach contributed to reducing protection threats affecting communities and/or their exposure to such threats?  
• How effective were the strategies and modalities employed and implemented by Oxfam and partners in working with communities and community structures?
• To what extent do targeted women and men have improved capability to accessing the needed services in safety, to self-refer or to request assistance to access necessary services?
• To what extent has our CBP programme contributed to communities’ self-protection strategies and capacities?”
• To what extend do members of CBP structure feel empowered in their position?
• How effective were the activities conducted by community structures (FPs, Women’s groups…) and what could have been contributed to an increased effectiveness of this action?
• How effective was Oxfam engagement with community facilitators and community groups over the years? What was missing and how could it be improved?
• To what extent have partner organisations participating in or contributing to community engagement? To what extent did partners develop a relation with targeted populations that contributed to an increased ability to access essential services and conduct risk mitigation activities? What gaps were there and how can they be addressed?
• What should a revised CBP approach include in the current context?
• Did the CBP contribute to reducing tensions between communities? Why, or why not?
• What internal and external factors have influenced the effectiveness of the CBP approach in a positive and both challenging ways before the COVID19 context? What has changed after the COVID19 pandemic?
• To what extend communities were engaged effectively? What could have improved that?

Sustainability:

• To what extent is the CBP approach sustainable? what evidence is there of the likelihood of sustainability of activities following the completion of the recent projects?
• To what extent do the resources in the community allow communities to continue to respond to shared risks and concerns without the support of Oxfam and partners?
• In what ways could Oxfam and partners enhance the sustainability of the community-based approach?
• Are affected communities better positioned to independently, or with increased independence, address their own current protection needs? And how are they prepared to respond to future risks or crisis? How did this change over time? What are the challenges around it? And how these challenges could be addressed by (1) Oxfam and partners, (2) by communities themselves.

Representativeness & Feminist Approach

• To what extend CBP structure were representing all social groups living in the community?
• To what extend community members felt represented by CBP structures?
• to what extent did the members of the structures feel supported by the community and/or representatives of a community?
• How effectively has the CBP approach integrated gender analysis and gender considerations (particularly with a view to GBV preventions) & mainstreaming actions, and to what effect? How were power dynamics reflected and considered? What were the gaps and how can they be addressed and improved? 
• In what ways can the current CBP approach engage women more effectively?
• Is there any evidence of the community-based approach contributing to increasing the efficiency of Oxfam’s programmes?
• Is there any evidence that CBP has increased the efficiency of communities to address protection issues? Or has increased the accountability of the authorities to their own communities? 
• Is there evidence that the CBP approach has led to a behavioural change among refugees in how they view their own protection and that of others?

Monitoring Evaluation & Learning:

• How is CBP being monitored? And how effective is the monitoring of the desired outcomes of CBP?
• How did the CBP intervention evolve since the previous protection evaluation in 2017?
• What recommendations have been incorporated from this evaluation? And how effective were these recommendations? Are these recommendations still relevant given the major changes in context?
• What factors have supported and hindered accountability to affected populations?

4. METHODOLOGY:

The evaluation will be conducted in line with evaluation best practices and will use qualitative methods and draw on both primary and secondary data collection techniques. The evaluator is expected to develop a detailed methodology (to be endorsed by Oxfam). The evaluation methodology should make use of previous evaluation sections specific to CBP and other data already generated by Oxfam’s monitoring and evaluation team.

We anticipate that this formative evaluation will be a participatory review and learning exercise. Thus, it requires the consultant(s) to be experienced in participatory approaches to learning and inquiry, and especially in seeking the views and perceptions of key stakeholders that include:

• Targeted population including different members of community structures (CFPs, CTVs, CHVs, and Women’s groups).
• Partners and actors directly involved in the project and at different levels with community structures: The implementing partners KAFA, NABAD, and CLDH;
• Oxfam staff involved the implementation of the project e.g. HPC, PM, Advisors and field teams.

5. TIMEFRAME, KEY ACTIVITIES AND EXPECTED LEVEL OF EFFORT

Timeframe: The CBP evaluation is to be started as early as possible in January 2021, with the final report submitted to Oxfam as per the timeline below. The exact dates of the evaluation are to be confirmed with the selected evaluator, depending on their availability. 

Total expected level of effort: 25 working days

• Review essential documents of the project, including but not limited to evaluations of the previous DANIDA and GAC projects, ECHO Protection evaluation conducted in 2017, CBP documents, previous proposals, tools and SOPs and any other relevant documents and review the key questions suggested and if necessary propose adjustment (6 days – work to be done remotely, all documents will be shared by Oxfam via email);
• Develop a detailed Evaluation plan (to be endorsed by Oxfam) (1 day); It is expected that the timeframe and methodology will be shared with and discussed with the protection program team ahead of the start of the evaluation. Once that is agreed upon, the data is expected to be completed within 10 days.
• Primary data collection (10 days in person/ changes might occur depending on the COVID19 lockdown situation);
• Data analysis and preparation of draft evaluation report (6 days);
• Meeting (online) to share and validate the findings from the evaluation (0.5 days). Selected Oxfam and partner staff will participate in this meeting;
• The final evaluation report within one week, following the date of receiving feedback. The

report will include changes/modifications, agreed between Oxfam and the evaluator. The report should be of approximately 20 pages (excluding annexes) and structured as per the criteria presented in this TOR, of publishable quality with a stand-alone executive summary of no more than 4 pages (in English).

Some minimum guidelines on the evaluation report: 

• The report should systematically answer the key questions posed; 
• It should fairly and clearly represent the views of the different actors/stakeholders; 
• It should give the conclusions of the evaluator, in a way that is clear and substantiated by the available evidence. 

Reporting: The evaluator will report to the Protection Advisor, with engagement from the MEAL

Coordinator, the Humanitarian program coordinator, PM and Protection Senior Officers. Logistics will be arranged by Oxfam and field travel (if/when required) will be accompanied by an Oxfam Protection Staff, and/or partner staff.

6. EVALUATOR QUALIFICATIONS:

This formative evaluation should be led by a person with a minimum of 7 years’ experience in humanitarian interventions in complex settings including demonstrated experience in the monitoring and evaluation of CBP and protection programming, with preferably some experience in gender in emergencies programming, including the use of participatory qualitative methods. Strong facilitation and English writing skills are also required and a familiarity or direct experience working in Lebanon.

Essential

• Advanced university degree or equivalent in Humanitarian/Development Studies, Social
• Sciences, and other related fields.
• At least five years of experience in conducting program evaluations
• Strong experience in protection programmes implemented in displacement and refugee contexts
• Experience in conducting evaluations of community-based programmes
• Strong analytical and research skills

Desirable

• Experience of working and/or conducting evaluations within Lebanon
• Arabic language skills

7. Payment and instructions for interested consultants

Payment will be done in two instalments, 25% upon contract signature, and 75% upon Oxfam’s approval of the final evaluation report.

What costs to include in the offer: Consultants should include the following costs in their offer’s budget: daily rate, per diems and communication costs.

What costs not to include in the offer: Oxfam will pay for and procure the following for the evaluation and therefore the following costs should NOT be included in the offer: travel inside Lebanon for data collection and meetings, interpretation services during meetings (if needed) with beneficiaries/the partner, printing/photocopying costs.

Note that payment will be made based on the budget in the offer (not based on actual expenses incurred by the consultant). No receipts will be requested from the consultant towards the end of the evaluation.

8. CODES OF BEHAVIOUR:

The evaluation process will be directed by Oxfam’s guidelines for the ethical conduct of evaluations and research, guiding the evaluation team through careful consideration of the key ethical implications at every stage of the evaluation. These guidelines are available at this link: http://policy-practice.oxfam.org.uk/publications/undertaking-research-with-ethics-253032 

9. ETHICS AND SAFEGUARDING 

Oxfam is committed to ensuring that the rights of those participating in data collection or analysis are respected and protected. All applicants should include details in their proposal on how they will ensure ethics and safeguarding in the data collection process. Specifically, the consultant(s) shall explain how appropriate, safe, non-discriminatory participation of all stakeholders will be ensured and how special attention will be paid to the needs of vulnerable groups. The consultant(s) shall also explain how confidentiality and anonymity of participants will be guaranteed. 

10. SHARING AND USING FINDINGS:

The Oxfam International’s Policy on Program Evaluation requires Confederation members to act on the commitment to transparency by making public the Executive Summary and a Management Response to all final evaluations. The Policy is available at this link: http://www.oxfam.org/sites/www.oxfam.org/files/oxfam-program-evaluation-policy-dec10.pdf

11. DISCLOSURE:

Under the terms of reference, the consultant is not authorized to make any commitments on behalf of Oxfam. All data collected as part of this consultancy belongs to Oxfam and public dissemination of the data and evaluation products can only be done with the written consent of Oxfam.