Terms of Reference End of Project Evaluation - The Just Right Start Project

Project Summary

  • Project title: The Just Right Start
  • Project reference code:326.161711
  • Project duration: 01/10/2016 to 01/10/2019, extended till August 2020
  • Project implemented by:   Ana Aqra Association
  • Donor: Private Philanthropic Organization
  • Project Location: Lebanon
  • Project Sites: Mount Lebanon & West Beqaa
  • Budget in the proposal: Euros, 2,999,466
  • Period of the Evaluation: (March-April 2020)
  • Previous Mid-Term Evaluation: November 2018
  • Background:
  • The Just Right Start project aims to prepare Syrian children and disadvantaged host community boys and girls aged between 4-6 years old to be enrolled and retained at formal schools. Many outputs are set to reach this main goal. The project aims to:

  • Enroll learners, who have no access to early childhood preparatory programs, in the community based shortened Early Childhood Education.
  • Children who completed at least one cycle of the shortened ECE and are of age 6 and up are oriented towards formal schooling provided by MEHE and
  • still have access to the Just Right Start homework support classes within the community based ECE. In support of their continued learning, the project provides:
  • Parents of out of school Syrian and host community girls and boys a program that enables safe and learning environment at home. This program is designed to first, raise the awareness of parents through parents’ meetings conducted by the center coordinators; and second, to enable parents with younger children (younger sibling of the enrolled child)

  • to enhance their abilities as their child’s first teacher within an enabled home learning environment; as well as the program aims at
  • Improve teacher capacity building through trainings and coaching.
  • The evaluation is carried out at this point in time to assess and present results (output, outcome), conclusions, lessons learnt and recommendations for a possible future programmatic directions that focus more on changing the mindset towards responsibility for learning results and on ways to scale up Ana Aqra ECE program and influence relevant (formal) education systems.

    In August 2020, the project will be completing, 8 of the 6 expected ECE cycles and 6 of the 4 expected Homework support cycles (2 cycles were added during the no cost extension period). A cycle is defined to be the period of 77 teaching days. Each year is composed of 2 cycles and a summer program. Children who reach the age of formal compulsory education are referred to enroll in the formal school. 

  • Evaluation questions

  • More specifically, it is expected that the evaluator answers the following evaluative questions:

  •  Effectiveness:
  • What is the level of implementation of all activities so far? (number of activities implemented vs. how many are supposed to be completed by the end of the project)

  • What level/type of achievement has the project attained when compared with the baseline status?
  • Are the actions taken enough to reach the project’s goals?
  • What are the major factors affecting the quality of work?
  • What are the trigger factors enhancing the quality of work?
  • How effective is AA at measuring the impact on learning outcomes? Does it have the right tools and systems in place to provide evidence of both intentional and unintentional outcomes?
  • What are the minimum conditions that will allow scale up or replication?
  • Efficiency:
  • Were activities cost-efficient?
  • Were objectives achieved on time?
  • Was the project implemented in the most efficient way compared to alternatives?)
  • Output Indicators

    (To be retrieved from Available Progress reports/logframe during the desk review)


  • Impact
  • -  How has the education program (learning space, facilitators, curricula and staff) contributed to the protection, personal, social and emotional development of children in the program?
  • - What impact has AA education program had on access and retention of Syrian children in formal education since the beginning of the program?

    - How did the ECE and Homework support program contribute to retention of children in formal education and to enhancing their promotion rates?

    - What impact has AA Education program had on community participation and refugee engagement of both parents? How has it improved their lives?

    -Which intended and unintended negative and positive effects can be possibly attributed to the project/program?

    -What role is technology playing to support the education for Syrian children. Is it appropriate and relevant and which areas should be scaled up or adapted in the future?

    -How many girls, boys and parents in total have already benefited from the project/program (immediate impact)?

    -How can the impact of the Just Right Start project be maximized?

    - Are there any other important aspects regarding impact?

    Has the partnership with the Private Philanthropic Organization brought any positive or negative impact on the program?


  • Relevance and Appropriateness
  • -How relevant and appropriate is the current program design and implementation to the educational, life skills and social needs of Syrian refugee and host community children?

    -To what extent has AA adapted to changes in the operating context since 2016?

    - What should be done to improve the relevance and appropriateness of the program?

    -What programmatic areas should be scaled up or adapted in future?

    - To what degree is the project designed and implemented based on the opinions and the priorities of Syrian refugee children and parents? Is it perceived as relevant by them? If so, how?

    - To what extent are the different needs of the various sub-groups (children with PSS needs, children with disabilities, children with special education needs) taken into account? How can the program better target sub-groups to become more inclusive?

    -Does the curricula and pedagogy meet the needs of the students? How can they be made more appropriate?

    - Does the education program meet the INEE minimum standards for education in emergency?

    -Are the expected results/outputs of the Just Right Start Project consistent with the outcome, immediate impact and overall goal/impact?

    Learning and Sustainability

  • - What are the suggestions for more effective programming to meet the overarching objective of access and retention in quality formal education? (structurally, work modality and staffing)? Identify current gaps and suggest good practices for future responses and organizational learning.

    - What, if any, are the scale up options for the education program in both formal and non-formal education?

    - How can AA contribute to a sustainable educational strategy in the host communities? What are the major factors influencing the achievement or non-achievement of sustainability of the Just Right Start Project?

    - What steps need to be put in place to ensure the ECE centers and homework support program can continue to provide quality education for the Syrian Refugee and Host community?

    - What could be strengthened to achieve greater impact in areas identified as important by children and their parents? 

    -To what degree have AAA established or improved mechanisms, structures and platforms for generating and reflecting on evidence on the impacts of interventions? What has impeded or supported such efforts?

    -To what Degree the evidence generated contributed to influencing AAA internal programming/ national and other NGO ECE platforms/ stakeholders/ and MEHE in Lebanon?


  • Coordination:
  • -How well is the AA program internally coordinated? And how can it be improved?

    -What is the potential for synergies with other relevant education providers (formal and non-formal)?

    -What has been the role of AA in the advocacy and coordination of quality certified education for refugee children within relevant MEHE working groups and sector relevant coordination meetings? How can this be improved?

    -How well does the education program coordinate with Formal schools?

    -What systems have been established to ensure efficient referrals between formal schools and AA ECE centers? How can these be improved?


  • Approach and Methods
  • The evaluation consists of several phases:


    Phase 1: Preparatory stage


    Desk Review: A desk review of relevant project documents or sources would be conducted. Key documents to review include: project proposal, project logical framework, the mid-term evaluation report, the monitoring reports (baseline report and second intake report, assessment tools, children’s development and academic progress, relevant minutes from the relevant education working group or task force meetings)


    Evaluation matrix and tools development: A detailed evaluation matrix will be produced by the evaluator; which will inform the design of the various evaluation tools that will reflect mixed method. The evaluation matrix and tools (Survey questionnaire and forms) will be part of the inception report that the evaluator will submit to Ana Aqra for approval prior to moving forward in the evaluation process.

  • Phase 2: Data collection


    It is expected that the evaluator sets up a strong data quality insurance process to avoid errors during data collection or data entry both quantitative and qualitative. All 3 centers will be equally included in the data collection plan, and will cover all aspects to inform the analysis.

    The evaluator is responsible of collecting, recoding and compiling data.



    Phase 3: Data Analysis


    The quantitative and qualitative data collected will be analyzed and used in order to present charts, correlations and to write the report of the end-term evaluation study. Ana Aqra Association expects the evaluator to ensure that the evaluation reports meet the objectives as stated in those TOR, that all indicators data be presented and that clear answers to the evaluation questions appears in the report.



    Phase 4: Presentation:

    Presentation of key quantitative and qualitative findings (feedback workshop) at the end of the conducted evaluation


  • Final Draft Report: Submission and presentation of final draft report that reflect both quantitative and qualitative findings
  • Final Report: Submission of complete final report with recommendations and discussions
  • For the different phases it is expected that data and information will be obtained through different methods such as: data from AA system, review and analysis of documents, structured interviews, semi-structured interviews face-to face or by phone, group discussions, others.


    Application Process and Requirements

  • AAA will be the main focal point for the assignment and direction of the evaluation and will receive directly the evaluation report.

    AAA will guide and facilitate the evaluator work in Lebanon by:

  • Providing a briefing for the evaluator to clarify open questions related to the “Just Right Start” project and the end-term review (methodology, logistics, deliverables).
  • Providing the evaluator with relevant documentation and contacts as requested clearly and timely.
  • Supporting the organization of the field phase as planned and shared.
  • AAA will:

  • Provide a management response to the results of the end-term review to the donor, upon receipt of the draft final report.
  • The evaluator will:

  • Manage his/her schedule and make appointments for interviews.
  • During the preparation phase, the evaluator will submit a work plan that should clarify the focus of the end-term review and the TOR, highlighting any reservations regarding the feasibility. The report should not exceed 1 page and should include as a minimum:

    • Key data of the end-term review (project title, project data, commissioner of the end-term review etc.);
    • End-term review design: Methodology, approach, steps for implementation and suggested timelines;
    • Annex with tools and survey templates etc.
  • The evaluator will develop the final methodology for the evaluation that will be discussed and approved by AAA and the donor.

  • Organize a local debriefing before the end of the field phase.
  • Collect and analyze qualitative data.
  • Compile and analyze quantitative data.
  • During the preparation phase, the evaluator will submit a work plan that should clarify the focus of the end-term review and the TOR, highlighting any reservations regarding the feasibility. The report should not exceed 1 page and should include as a minimum:

    • Key data of the end-term review (project title, project data, commissioner of the end-term review etc.);
    • End-term review design: Methodology, approach, steps for implementation and suggested timelines;
    • Annex with tools and survey templates etc.
  • The evaluator will develop the final methodology for the evaluation that will be discussed and approved by AAA and the donor.


  • Timetable

    The work might be concluded in a period of 31 days and shall include the tasks below:

    - Preparation (preparation methodology, work plan, inception report) -

    -  Field work/data gathering &analysis

    - Report writing & presentation

    The start of the evaluation is March 2nd, 2020.


  • The Evaluation Team

    Key Qualifications in the team should be:

  • Master level in any field related to early childhood education and/or management.
  • A minimum of three years’ experience and expertise in the field/sector of early childhood education and a deep understanding of the refugee context in Lebanon.
  • Team leader has conducted at least three evaluations in the last five years ideally in the relevant field.
  • Team member has participated in at least three evaluations ideally in the relevant field.
  • Working experience in Lebanon.
  • Experience in project cycle management.
  • Experience preparing and analyzing a theory of change.
  • Experience in participatory, qualitative methods, to complement the significant amount of quantitative data.
  • Experience in social science methods.
  • Excellent oral and written English skills.
  • Sound MS Office and IT skills.
  • Reports

  • The consultants will submit the following reports:

  • an inception report (10-15 pages without annexes),
  • a final draft evaluation report (about 25-30 pages without annexes), including a draft executive summary and the results-assessment form (part of the reporting requirement)
  • and the final evaluation report (25-30 pages without annexes), the final executive summary and the results-assessment form (part of the reporting requirement)
  • All reports need to be written in English.

    The executive summary should summarize key findings and recommendations (three to five pages) and needs to be submitted as part of the final draft report.

    The findings and recommendations of the draft final report and final report have to be structured according to the evaluation questions. An outline of the report’s structure needs to be agreed upon during the inception phase. 

The quality of the reports will be judged according to the following criteria:

  • Is the results-matrix format part of the report?
  • Does the report contain a comprehensive and clear executive summary?
  • Were the Terms of Reference fulfilled and is this reflected in the report?
  • Is the report structured according to the donor criteria?
  • Are all evaluation questions answered?
  • Are the methods and processes of the evaluation sufficiently documented in the evaluation report?
  • Does the report describe and assess the intervention logic (e.g. log frame, program theory) and present/analyze a theory of change and its underlying assumptions?
  • Are the conclusions and recommendations based on findings and are they clearly stated in the report?
  • Does the report clearly differentiate between conclusions, lessons learnt and recommendations?
  • Are the recommendations realistic and is it clearly expressed to whom the recommendations are addressed to?
  • Were the most significant stakeholders involved consulted?
  • Does the report present the information contained in a presentable and clearly arranged form?
  • Is the report free from spelling mistakes and unclear linguistic formulations?
  • Can the report be distributed in the delivered form?
  • Is the report well-structured and designed for friendly and ease to read and understand.
Overall Objectives: 
III. Objectives of the Evaluation The main purpose of the evaluation is to support learning and provide guidance for future program direction. In addition, the evaluation should be an opportunity for Ana Aqra to be accountable to beneficiaries, partners and donors. The evaluation will also evaluate the project’s relevance, efficiency, effectiveness, impact and sustainability, in line with the DAC criteria and assess the project performance of Ana Aqra Association at the end of the project. These activities mainly fall under the project’s main objective and outputs below: Project objective: The project’s main goal is to enhance school preparedness for integration and retention of Syrian children and disadvantaged host community.
Project outcomes: 1. Out of school Syrian refugee and host community girls and boys (ages 4-5) have access to safe protected inclusive community based ECE learning environments and are ready to enroll in public schools in Bekaa, Mount Lebanon, and Akkar 2. Out of school Syrian refugee and host community girls and boys (ages 6-8) who completed at least one cycle of Community based ECE and have enrolled in grade 1 of public school are promoted to the next grade level on the following school year in Bekaa, Mount Lebanon, and Akkar 3. Out of school Syrian refugee and host community girls and boys (ages 4-8) who are enrolled in the Community based ECE or HWS reveal increased social and emotional well being as they engage in learning and become part of the learning community in Bekaa, Mount Lebanon, and Akkar 4. Parents of out of school Syrian and host community girls and boys provide safe and enabling learning environments to their children in support of their continued learning (ages 2 to 8) 5. Teachers (of HWS, ECE, Just Right Start) of out of school Syrian refugee and host community adopt best practices in differentiated teaching for the provision of safe and enabling learning environments in Bekaa, Mount Lebanon and Akkar. Under the relevant DAC evaluation criteria (relevance, impact, effectiveness, efficiency, sustainability and coordination) the evaluator has to analyze the following points: 1. The extent to which the project has already achieved its objectives and results or is likely to achieve them, including the extent to which the lives of the beneficiaries (Syrian and host community girls and boys) has already been improved. 1.1. The form for the results-assessment needs to be filled in by the evaluator as part of the reporting requirement. 2. The strengths and weaknesses in terms of planning, management, implementation and monitoring The evaluation will assess the output and outcome indicators undertaken in line with the aforementioned objectives.
Start Date
Tue, 04/02/2020 - 9:00am to Wed, 04/03/2020 - 9:00am
Project Status
Project Timeframe
2 Weeks
Intervention Sector(s):
Project Location:
Beirut , Beirut
Beirut LB
Collaboration with Other Organisations