3.1 Overall objectives and expectations for the evaluation

The overall aim is to assess the level of achievement of the project and the extent to which the project has brought positive changes and added value to beneficiaries, relevant stakeholders. This summative evaluation will also assess the quality of project approaches, strategies, and interventions as stated in HI Project Quality Framework. Further, this evaluation will take into account the constraints and challenges, as well as adaptations during project implementation, in the context of the COVID-19, and the events of the summer 2021 which led to the change of power in the country and also to analyze the impact of project implementation to see kind of activities that has work well and/ or need to improve in the future as part of lessons learn of the project.

Specifically, the evaluation aims to:

• Analyze the extent to which the project has contributed in bringing about short and long-term positive effects on the lives of beneficiaries and their families in 4 Provinces of intervention ;
• To examine and analyze the achievements of project expected result
• Analyze how the project resources are mobilized and optimized during the

Implementation;

• Investigate and analyse the level of involvement of other humanitarian actors to ensure proper referral mechanism

 Finally, the final evaluation will provide HI,  and ECHO with sufficient information to:
- assess and analyze the project results, efficiency and relevance of the implementation strategy
- review, analyze, provide evidence and document the impact of the intervention through its progress, challenges, barriers and lessons learned with recommendations for a logical and potential next phase.

Additionally, the evaluator will conduct the endline Knowledge Aptitude and Practice (KAP) survey for the EORE component of the project. Baseline and midline surveys   were conducted under the previous phase of this project in December 2018 and 2020 in Herat, Kandahar, and Kunduz. The objective of this exercise was to better understand the realities of at-risk groups’ and guide HI’s response in designing the most adequate RE messaging for the specific needs of the targeted audience as well as to help prioritize where to deploy HI RE teams to the most vulnerable communities.  HI will share the previous reports as well the methodology and will ensure continuous communication with the EORE Technical Advisor to get technical guidance.

Evaluation questions / criteria

The consultants will articulate their analysis around a set of evaluation questions related to Changes,  Efficiency, Relevance, and Effectiveness. Some questions are listed below. These questions are not exhaustive and will be reviewed by the consultants at the beginning of the inception report stage and planning. The following criteria should be looked into, though other criteria can be suggested by the evaluator:

Relevance: appropriateness of project objectives to the problems it was supposed to address, including assessment of the quality of the project preparation and design. Specifically:

- Does the project meet the needs of the target population?

- Is the project design appropriate to the specific context?

- Are the mechanism and approaches developed in coherence with existing plans and policies of HI?

Effectiveness of the project, including specific assessment of the benefits accruing to target groups:

- To what extent does the project achieve the expected results?

- Did the project contribute to improved access to services of the beneficiaries, especially persons with disabilities?

- How does the team adjust the project and its methodologies to the constraints faced during the

implementation phase to achieve the expected results?

- More specifically, do the mobile teams meet the needs of persons with disabilities and considered best practice (within context constraints)?

- Do the project’s activities designed and implemented address the needs of both men and women

on an equal basis as well as the needs of all persons with disabilities in the camps?

• Did the project’s implementation have any unexpected consequences which should be considered in the design of future projects with a similar program logic

Efficiency, including comparing alternative approaches to achieving the same results, to see whether the most efficient process has been adopted:

- Are the methodologies and tools appropriate to meet the project’s objective?

- Is the project team profile and organization efficient for implementing the project in the area

covered?

- Are there any barriers to the efficient implementation of the key activities?

- Do the strategy and approaching method enable the achievement of the results in a cost-effective

manner?

Change

- To what extent can it be said that the effects/impacts are attributable to project interventions? Are there other external factors which have played a role in the effects/impacts during the project period? How does these effects/impacts vary across intervention groups (namely men, women, people with disabilities)?

- For those effects/ impacts which are attributed to the project, what have been the processes, component or qualities of the project intervention which have led to the change?

- What modification/reorientation should be made in areas of intervention and activities to better achieve the expected the effects/ impacts? The evaluator should consider both the short term and the longer-term directions for future strategy.

- Does the project have any negative impact or is likely to have in future?

- What is the impact of the project on its wider environment (social level impact, policy level impact, economic level impact, technical level impact)?

Sustainability to assess the likelihood of benefits produced by the project to continue to flow after external funding has ended, and with particular reference to:

• Financial sustainability
• Institutional sustainability
• Policy level sustainability

• 3. METHODOLOGY

   The exact methodology should be proposed by the consultants in their applications.

   The evaluation should take the opinions of the different actors and beneficiaries into account and compare their views and perceptions of the progress made by the program. The methodology should include but not be limited to the following:

   

  Desk phase

  - The consultant (or team of experts) will review existing project documents, (database, benefpack, previous reports, meeting minutes, technical mission reports). On this basis, the consultant will refine the evaluation questions, propose a detailed methodology and a detailed working plan including the list of stakeholders to meet during field phase. These elements will be combined in an inception report.

  - HI team will validate the inception report.

   

  Field phase

  - Briefing with Operations Manager, Meal Manager, Technical Unit Manager, Project Managers and Technical Advisors.

  - Interviews at Province level with a selection of staff having supervised and/or implemented the project.

  - Interviews in the four provinces with a selection of stakeholders and beneficiaries associated with the Project.

  -Focus groups discussions” with each of the Self-Help Groups and individual interviews with the end beneficiaries.

  - Case studies outcomes.

  - Presentation of findings to the project team with the aim of clarifying details and issues and soliciting further input and feedback.

   

  Analysis phase

  - At the end of the field mission, the evaluator will present the initial results and recommendations to the national team for discussion.

  - On the basis of these discussions, the evaluator will draft a summary report of 5 pages maximum, excluding appendices, within five working days and send it to HI Afghanistan.

  - The HI teams will provide any feedback on this report within five working days.

  - The evaluator will submit a final report.

   

   Ethical measures
  As part of each evaluation, HI is committed to upholding certain ethical measures. It is imperative that these measures are taken into account in the technical offer.
  - Guarantee the safety of participants, partners and teams: the technical offer must specify the risk mitigation measures, especially in the context of Covid-19 pandemic.
  - Ensuring a person/community-centred approach: the technical offer must propose methods adapted to the needs of the target population (e.g. tools adapted for audiences with low literacy rates/sign language/child-friendly materials, etc.).
  - Obtain the free and informed consent of the participants: the technical proposal must; participants.
  - Ensure the security of personal and sensitive data throughout the activity: the technical offer must propose measures for the protection of personal data.

   

  Protection and Anti-Corruption Policy

   

  Code of Conduct

  Protection of beneficiaries from sexual exploitation, abuse and harassment

  Child Protection Policy

  Anti-fraud and anti-corruption policy

   

   

  4. DELIVERABLES
  - Produce an inception report in English, including all proposed tools, methodology and detailed work plan to be introduced at the end of the desk phase. The inception report will have to be validated prior launching the field phase.

  - Produce a KAP endline report for the EORE component of the project.
  - A final report in English (max.45 pages).
  
  - The final report should be divided into the following sections:

  - Executive summary of the evaluation findings

  - Introduction to the context

  - Evaluation methodology, including selection and sampling methods, and mention any constraints and challenges encountered, and strategies used to overcome them.

  - Detailed key findings and conclusions related to the main objectives

  - Conclusions, Lessons Learned, Best Practices and recommendations (including HI Inclusion good practices)

  - Annexes – all data collection tools,

  - List of persons met during the evaluation process and salient points of the meetings

  - All reports will be delivered in English and the report will be introduced in soft copy and 3

  hard copies.

  - Within the report confidentiality will be respected when representing personal information.
   

  5. BUDGET

  Maximum budget available for the assignment is 23,500 EUR. Any cost related to the evaluation is the responsibility of the evaluator and included in the budget proposal. The evaluator may not charge HI extra for it.

  The proposal for payment modalities and terms must also be included.

   

  6. TIMELINE
  The evaluation mission The final deadline for the submission of the final evaluation report is 10 September 2021 the latest including HI validation

   

  7. PROFILE OF THE CONSULTANT / TEAM
  The evaluation will be carried out by an expert or a team of experts. If a team of experts is selected, the
  evaluation will be put under the responsibility of one team leader chosen among the team of experts. This person will ensure all communication with HI Afghanistan office and will be the sole responsible for managing the organization of the evaluation.
  The team leader who will endorse responsibility of this assignment should have the following skills, experience, and knowledge:
  - Extensive experience, not less than 5 years, in program development/design, program management & implementation, monitoring & evaluation.
  - Experience or assignments in project impact evaluation and participatory approaches in data gathering.

  The team of experts (including the team leader) should combine the following skills, experience, and
  knowledge;
  - Background in disability, or other vulnerable or marginalized groups, preferably with a working knowledge on civil society organization.
  - Experience in conducting participatory (qualitative and quantitative) evaluation techniques
  - Experience working in Afghanistan actors, stakeholders.
  - Excellent spoken and written English.