Description

1. Overview

Project Title

Managing Risk through Economic Development Phase 4 - MRED IV program

Implementing Agency

Mercy Corps

Donor Agency

United states based foundation

Type of Study

Final Evaluation

Expected start/end dates, number of workdays

February 3rd to March 31st, 2025 - Total 20 working days

Location

Dili, Timor Leste

2. Background:

Mercy Corps is a leading global organization powered by the belief that a better world is possible. In disaster, in hardship, in more than 40 countries around the world, we partner to put bold solutions into action — helping people triumph over adversity and build stronger communities from within.

Since establishing operations in Timor-Leste in 2007, we have earned a reputation for innovative and high-quality programming. We promote a community-led, market-based approach to addressing key challenges to improved wellbeing through greater resilience, resolved root causes of poverty, and inclusive economic growth.

3. Project Description

Mercy Corps’ Managing Risk through Economic Development Phase 4 (MRED IV) program is a 3-year program implemented in Timor-Leste in Ainaro and Covalima Municipalities from April 2022 until March 2025. The purpose of the project is to accelerate the replication and scale of the M-RED disaster-readiness approach in target geographies through a multi-pronged strategy: increasing self-sustaining disaster readiness capacities in Phase III target communities, accelerating M-RED expansion in new communities, and influencing the enabling conditions needed to promote sustainability, replication, and scale while promoting innovation through early action and localization.

The Program implemented its interventions in areas such as:

Established and strengthened Suco Disaster Management Committee (SDMC) and Community Disaster Support group (CDSG).
Trained and socialized GESI integration in program implementation.
Installation of a new Early Warning System (EWS) OR strengthening existing EWS.
NEXUS crop/ Risk Sensitive livelihood promotion.
Constructed/ repaired structural and non- structural mitigation activities.
Coordinated and partnered with stakeholders for implementing the DRR and Livelihood work.
Innovated a Resilience roof against strong wind.
Conducted Research and Learning for adaptative management decisions and influence and advocacy of M-RED promoted practices.

Three main Outcome of the MRED IV program are as follows:

Outcome 1: 39 vulnerable communities will have improved disaster readiness, engaging inclusive community disaster risk management groups, and promoting adoption of disaster risk management and risk-mitigating livelihood practices to strengthen assets, capacities, and behaviors to prepare and respond to natural hazards.
Outcome 2: Formal Partnerships with National 1, Post Admin 6, Sucos 19, Aldeias 39, and 35 private sector actors will be established and strengthened to build enabling environments that can enhance the disaster-ready capacities of communities before, during, and after disasters.
Outcome 3: Nine internal learning events and 4 external learning sharing events will be conducted for adaptive management within the program and dissemination of M-RED best practices to advocate for and influence stakeholders to apply the project approach and activities.

4. Purpose

The purpose of the consultancy is to conduct an endline evaluation of the MRED IV program. This endline evaluation will assess overall program performance to answer the evaluation questions outlined in the section below against the list of DAC frameworks. The consultant will review indicator targets, aggregate program successes and lessons learned and provide recommendations for future programs by specifically answering the evaluation questions. This evaluation should be utilization-focused, as the follow-on MRED IV project will aim to implement feasible recommendations in the follow-on program/s.

The evaluation aims to answer the following evaluation questions:

Effectiveness

How well have the program objectives been achieved? And if not, why?
How well have the outputs of the program been achieved? And to what extent have they contributed to the program objectives?
How effective are the approaches and structures in delivering the desired outputs? How can they be improved?
Is the project bringing about desired changes in the behavior of people and institutions? If so, what is the extent of this change?
Are there or could there in future be any unintended positive or negative impacts arising from particular outcomes?
What could have been the likely situation (of the environment and its management) without the project?

Efficiency

To what extent were the available technical and financial resources adequate to fulfill the project plans?
Did the program face any unforeseen problems? If so, how well were they dealt with?
Was there an effective process, built into the management structure for self-monitoring and assessment, reporting and reflection? How well did this mechanism or process work?

Relevance

How well were the needs, problems, challenges and issues of the communities in the program area addressed by the design of the program?
To what extent is the project contributing to the strategic priorities, policies and plans of GoTL and other stakeholders.

Sustainability

How well is the approach used likely to ensure a continued benefit after the end of the project?
How well are all key stakeholders sufficiently and effectively involved? And how well are their expectations met and are they satisfied with their level of participation?
Are alternative or additional measures needed and, if so, what is required to ensure continued sustainability and positive impact?
Recruitment Overview

Project Title

Managing Risk through Economic Development Phase 4 - MRED IV program

Implementing Agency

Mercy Corps

Donor Agency

Margareth A. Cargill Philanthropy (MACP)

Study Period (tentative)

February 3rd to March 31st 2025 - Total 20 working days

Research area of endline survey

Consultant with background of Disaster Risk Reduction Expert, Agriculture system, and Economic Recovery Market System

Qualifications

Mercy Corps is looking for an experienced consultant. to lead the evaluation study which will include an endline survey, and qualitative methods to further explain and triangulate data. Therefore, the consultant should be well versed in using mixed method approach, design and tools. S/he will be responsible for ensuring the endline evaluation utilizes methodologies that meet quantitative and qualitative research standards.

Detailed responsibilities should be determined at the beginning of the mission and outlined in the methodology.

Area

Qualification

Experts in Disaster Risk Reduction Expert, Agriculture system, and Economic Recovery Market System

1. Five to seven years of experience conducting and leading program evaluations, preferably final evaluations.

2. Prior experience evaluating MACP funded programs

3. Fluency in English required, Tetun language skills preferred

4. Excellent written English skills

5. Graduate degree, from a recognized institution preferably in subjects related to climate change/agriculture/DRR/economics/evaluation studies or other social sciences with statistics

6. Demonstrate ability to design evaluations using mixed method approach using both quantitative and qualitative methods

7. Demonstrated experience in writing evaluative reports using sboth quantitative and qualitative data to synthesize findings for recommendation to a wide range of audience

8. Demonstrated experience in training and managing data collectors/enumerators and managing data quality with supervision of enumerators including pre-testing for validity and reliability of instruments

9. Advanced computer skills with use of data analysis softwrae such as R program/SPSS/STATA and Nvivo/Maxqda including excel with visualization skills

10. Ability to work independently and lead the evaluation team

11. Familiarity with MACP evaluation procedures, data quality and standard indicators is a plus

12. Fluency in English required, with fluency in Tetun preferred

13. Prior experience evaluating programs in Southeast Asia, with prior experience in Timor Leste is a plus

5. Evaluation Design and Methodology

The evaluation design will follow a theory-based evaluation approach that will make inference on the pathway of change depicted by the design of the Theory of Change of this program. The design will analyze the pre post status of indicator performances by considering their actual values to compare with that of the baseline. For indicators for which endline study is required the consultant will design this to generate their endline values using a survey methodology in line with what was used in the baseline for coherence. A simple pre post comparison will be used to analyze comparative difference of baseline and endline on the basis of the method, including sampling strategy, proposed by the consultant depending on the nature of the data. The validity of the simple pre post will be further explained and triangulated using primary qualitative data collected during the evaluation fieldwork from key stakeholders of the program. The design of this qualitative data gathering will be done by the consultant. Based on these premises the consultant should develop a thorough design and methodology for this evaluation and presents part of the inception report to be approved by Mercy Corps. In addition, the consultant should include the method s/he will use for synthesizing data to present the evaluation findings and recommendations.

The final design and methodology for the evaluation are to be developed through consultation with the Mercy Corps team taking into account the budget and the ToRs. The consultant after review of the documents for secondary data and information should update the evaluation questions outlined within the SoW, specifying the specific review issues, questions, methods of data collection and analysis. The desk review will consider the following:

i. A desktop review of all relevant documentation, including (but not limited to):

The project document, contracts and related agreements
Baseline report
Annual workplans and budgets
Bi-annual and Annual Reports
Pause and Reflect Report
Research and Learning Report
MRED IV 2 pager
Community Level DRM Toolkit guidance reports/ Datasets (3 Years Datasets)
Indicator Performance Tracking Table (IPTT)
Final Internal Performance Report (FIPR)

ii. Key Informant Survey (KII): A list of key partners and stakeholders will be identified at an early stage and a consultation process developed. All stakeholders consulted should be in a position to present their views in confidence to the team and to identify issues, opportunities, constraints and options for the future.

Face to face interviews and discussions with key stakeholders involved in the project
Electronic interviews through teleconference or written comments – e.g. email, where partners cannot be reached for face-to-face interview.

iii. Household Survey with baseline participants (We will aim to reach the same participants and same number of households from each of the communities)

Mercy Corps Timor Leste Monitoring, Evaluation, Research and Learning (MERL) team will lead the field data collection for quantitative data under the supervision of the Consultant.

A detailed analysis plan will be developed during the inception report, using indicator definitions and data collection tools.

6. Consultant Deliverables

Inception Report with Final Design and Methodology for the evaluation (3 Days)
Review of secondary sources of information: Collect and review relevant data, reports and publications, as part of the overall analysis and reporting
Finalize design and methodology for the evaluation and share with Mercy Corps for finalization
Detail Analysis Pan
Submit and present inception report with detailed chapter outlines (the final evaluation report should be limited to 25 pages excluding Annexes)
Data Collection Tools for both qualitative and quantitative techniques (in close consultation with Mercy Corps) (1.5 Days)
Finalize data collection instruments such as survey questionnaire and FGD or KII tools
Pre-test the data collection tools for validity and reliability
Assist MERL Team to transfer questionnaire(s) into relevant technological platforms to enable mobile data use.
Assist translation of data collection tools into local language(s) where relevant.
Provide ToT to the MERL team to train data collectors/enumerators for quantitative data collection.
Draft Endline Report and Presentation (14 Days)
Conduct KIIs
Ensure quality of household survey data collection
Establish mechanisms to ensure data quality and Real time review of data collected
Data management, and analysis
Data cleaning and consolidate findings from surveys and assessments.
Perform data analysis
Submit Draft Endline Report and presentation of Endline Results
Prepare final endline report, incorporating Mercy Corps feedback (1.5 Days)

7. Mercy Corps Support

Review and approve key research questions developed.
Provide relevant data and documents to the Consultant (for a targeted document review)
Review and translate interview guides, as needed
Host presentation and discussion on the study findings.
Support on the logistics for the data collection, including arranging enumerators
Translating local language if necessary.

8. Logistic and Administration Support:

Mercy Corps will assist the consultant for travel (international and local) and accommodation arrangements

9. Timing & Schedule

The review is scheduled to take place in the month of February 2025 for an estimated 20 working days broken down as follows:

Review of background documentation and preparation of methodology – 2 days
Training on data collection with Enumerators – 2 days
Assessment of project progress and performance – including field visits and interviews with project partners and key stakeholders – 10 days
Analysis of findings and production of draft report – 4 days
Debriefing - presentation and discussion of findings to project partners – 1 day
Finalization/revisions of the report and submission – 1day

10. The Consultant will report to:

Jules Keane, Country Director of Mercy Corps Timor-Leste

The Consultant will work closely with:

MRED Regional Manager

MRED Regional MERL Advisor

Program Manager and

MERL Manager and team members as relevant

Application Procedure

Interested applicants should submit:

● Detailed CV including a list of previous work.

● A copy of a recent project evaluation that demonstrates applicant’s ability to perform the expected deliverables.

● A financial proposal Note: Travel arrangements and costs including per diem will be covered by Mercy Corps and therefore should not be included in the financial proposal.

Diversity, Equity & Inclusion

Achieving our mission begins with how we build our team and work together. Through our commitment to enriching our organization with people of different origins, beliefs, backgrounds, and ways of thinking, we are better able to leverage the collective power of our teams and solve the world’s most complex challenges. We strive for a culture of trust and respect, where everyone contributes their perspectives and authentic selves, reaches their potential as individuals and teams, and collaborates to do the best work of their lives.

We recognize that diversity and inclusion is a journey, and we are committed to learning, listening and evolving to become more diverse, equitable and inclusive than we are today.

Equal Employment Opportunity

We are committed to providing an environment of respect and psychological safety where equal employment opportunities are available to all. We do not engage in or tolerate discrimination on the basis of race, color, gender identity, gender expression, religion, age, sexual orientation, national or ethnic origin, disability (including HIV/AIDS status), marital status, military veteran status or any other protected group in the locations where we work.

Safeguarding & Ethics

Mercy Corps team members are expected to support all efforts toward accountability, specifically to our stakeholders and to international standards guiding international relief and development work, while actively engaging communities as equal partners in the design, monitoring and evaluation of our field projects. Team members are expected to conduct themselves in a professional manner and respect local laws, customs and MC's policies, procedures, and values at all times and in all in-country venues. Furthermore, the design of the evaluation will include ethical standards in line with Mercy Corp Policy. Primarily all data collection and photos taken by the evaluation team will need to take written consent of respondents which will include communities and other stakeholders. All data collected during the evaluation will be the property of Mercy Corps and should be submitted in raw and cleaned versions.

Keywords

Evaluation Disaster Risk Agriculture Economic Development Data Analysis Qualitative Methods Quantitative Methods Sustainability Climate Change Timor Leste