Project Final Evaluation Consultant

Mennonite Economic Development Associates (MEDA) invites applications for a Consultant to support the Final Evaluation of the Technolinks+ Project.

MEDA’s work is built on a foundation of Mennonite business roots and faith-based values, within the global context of the UN Sustainable Development Goals. We believe business solutions are effective interventions to address poverty. Through a market-systems approach and strategic partnerships, MEDA enables access to finance and provides business and technical expertise to build transformative agri-food market systems that create decent jobs, allowing traditionally excluded groups to become active participants in a sustainable economy. MEDA welcomes all who share our values and want to join us in our mission. To find out more about MEDA, please visit our website at www.meda.org.

Background

The Technolinks+ Project: Advancing towards Inclusive Prosperity in the Agri-food Export Sectors in Nicaragua, is a development initiative implemented by Mennonite Economic Development Associates (MEDA) with financial support from the Government of Canada (GOC). Technolinks+ aims to contribute to the sustainable economic growth of women and men experiencing poverty living in selected rural communities in the northern region of Nicaragua, improving access to more productive, profitable, gender responsive and climate-resilient technologies necessary to improve agricultural productivity, production quality and profitability of small producers and distributors of agri-food at the farm, company, and export supply chain levels. For more information please visit our Technolinks+ Project page. Technolinks+ is in the last year of its operating cycle, and MEDA conducts final evaluations of all significant projects with the understanding of the importance of conducting regular reviews of development interventions that help in documenting results, discovering lessons learned, and improving future project implementation. This is a summative evaluation intended to assess the overall outcomes of the project. The evaluation will be performed after the period of implementation of the project.

Evaluation Purpose

The purpose of the final evaluation is to assess the initial outcomes established in the Project Implementation Plan (PIP),provide evidence of the level of achievement of these outcomes and disseminate key findings to local partners and beneficiaries as part of ongoing learning process embedded in the technical capacity building activities. In detail, the final evaluation will help:

• Assess the performance of the project against intended results and unanticipated results.
• Identify important findings for all three key areas of work.
• Determine and interpret the percent or total values of the project outcome indicators compared to their baseline and the annual outcome measures.
• Identify project strengths, weaknesses, and challenges with special attention to the mechanisms used to achieve the results (e-voucher, matching grants, loan capital fund, among others).
• Determine whether technologies introduced are gender-responsive and environmentally sustainable, and how their introduction has lessened gender inequalities across the agricultural sector and improved environmental conditions (e.g. water, soil, biodiversity, etc.)
• Identify facilitating and restrictive factors vis-à-vis project results.
• Document lessons learned, and group them into specific and cross-cutting – gender equality, coordination, etc. Indicate with examples why and how the challenges were mitigated in making course corrections.
• Provide recommendations on future designs, and group the recommendations under each area of work, including recommendations for GAC as a donor for successful approaches and activities that could be replicated and considered for future programming.
• Disseminate findings and recommendations among local partners, projects participants and stakeholders.

Overall, the results of this external evaluation are primarily intended for use by MEDA GOC to assess and communicate the project outcomes. Results and findings may also be shared with other key stakeholders such as project clients and industry stakeholders. However, the utility of this summative study is that the findings will build evidence on the results achieved by the project.

Evaluation Scope and Criteria

The evaluation will consider all the activities, outputs and results produced during the life of the project (2017/2022) and will gather data from multiple stakeholders and geographies where project activities are implemented.

The evaluation will apply the following OECD/DAC and MEDA evaluation criteria:

1. OECD/DAC
   1. Relevance. Did the intervention do the right things? To what extent did the intervention objectives and design respond to MEDA clients’ needs? Did the intervention respond to MEDA’s Vision 2030 strategy and the SDGs? To what extent the intervention responds to GAC FIAP core action area 1 – gender equality and the empowerment of women and girls[2] as indicated in the design and the Project Implementation Plan?
   2. Coherence. How well did the intervention fit? How compatible was the intervention with other interventions in the country, sector, or institution?
   3. Effectiveness. Did the intervention achieve its objectives? To what extent did the intervention achieve its objectives and results (including any differential results across groups)? To what extent the project put effort to engage with women stakeholders from diverse backgrounds in achieving the results?
   4. Efficiency. How well were resources used? Was there any additional resource needed to engage with women stakeholders? To what extent did the intervention deliver results in an economic and timely way? What was the project budget spending rate? To what extent do internal audit reports assess the efficiency and compliance with the organization's internal policies?
   5. Impact. What difference did the intervention make? To what extent did the intervention generate or is expected to generate significant positive or negative, intended or unintended, higher-level effects? To what extent the project contributed in narrowing the gender equality gap?
   6. Sustainability. Will the benefits last? To what extent did the net benefits of the intervention continue or are likely to continue after the end of the project? Were there sufficient project interventions designed to ensure sustainability of results? Will partnerships at local level continue? Include system level impacts. To what extent the project helped in enhancing capacity of women and girls in moving towards transformative change?
2. Priorities
   1. Gender Equality.
      1. To what extent were the gender equality outcomes achieved by the project as laid out in the project RBM (Results Based Management) tools and gender equality strategy?
      2. Was the gender equality strategy implemented as envisioned? Did it change, and if so why and how?
      3. How is the project contributing in advancing gender equality and the empowerment of women and girls – decision making, access to resources and transformative change?
      4. To what extent were the technologies gender responsive? Were there unintended gender equality results i.e negative backlash against women’s increased employment or economic opportunities?
      5. Are the gender equality results sustainable or will they disappear once the program ends?
      6. GBV referral pathways: During COVID 19, Gender Based Violence increased which affected many women during. We should inquire more if women know where to get help/assistance when they are abused/violated?
      7. If women Farmers have increased income, do they still have control/decision making power?
      8. Feedback from the women farmers and agribusinesses however negative should be reported
      9. The areas the project didn’t focus on more should be brought out clearly (Unpaid care work, GBV etc.) this will be a learning lesson for other subsequent projects in promoting GE and WEE.
      10. In male dominated areas where women are working, dig deeper and understand their constraints, how this impacted women farmers and are there GBV issues and how is it handled as a business/cooperative.
      11. There is need to criticize more on why women are more in one area than another e.g., sorting and not spraying, is it the issue of pay or is the gender pay gap enhanced?
      12. Strategic patriarchy – are farmers fronting women to access finances from and yet they are not involved as decision makers? (This is the new trend that is emerging because of the donor requirement yet women don’t have any decision-making power or control in these businesses, need to dig deeper).
   2. Environmental Sustainability & Climate Change. Did the project conduct environmental studies, and if so what environmental impacts were identified and what mitigation measures were put in place? To what extent has the intervention achieved environmentally sustainable results? Did the intervention activities consider and reassess climate change mitigation and adaptation during the project implementation? Is there evidence (or what are the evidence) that the intervention achieved results related to climate-smart agriculture, i.e., achieving climate change mitigation and adaptation as well as improving productivity?
   3. Investment for financial inclusion: To what extent has the intervention contributed to increase lending options for farmers especially women to access to technology? Have the increases been done in an equitable manner?
   4. Innovation and Effectiveness: How did the project deliver on Canada’s Feminist approach to innovation and effectiveness?
3. Other Evaluation Strategic Considerations
   1. Results-Based Management. To what extent was the project primarily managed by results? What improvements could be made in the project’s RBM system? Were there any lessons learned on adaptive management? To what extent monitoring findings and recommendations were integrated in project operational plan to make course corrections?
   2. Coordination: To what extent MEDA has sought opportunities to collaborate with other stakeholders, including other development partners, and what were the results of this collaboration? How was MEDA coordinating and communicating with GAC in raising concerns, strategic change and in making course corrections from monitoring based on local realities?
   3. Optional: Benefit-Cost Ratio. The evaluator could work with MEDA HQ to calculate the relationship between the relative costs and benefits (income, jobs, assets) change due to the project.
   4. Context: which facilitating and restrictive factors influence in project results

The criteria should be applied thoughtfully to support high quality, useful evaluation. They should be contextualized – understood in the context of the individual evaluation, the intervention being evaluated, and the stakeholders involved. The evaluation questions (what you are trying to find out) and what you intend to do with the answers, should inform how the criteria are specifically interpreted and analyzed. Use of the criteria depends on the purpose of the evaluation. The criteria should not be applied mechanistically. Instead, they should be covered according to the needs of the relevant stakeholders and the context of the evaluation. Time and resources may be devoted to the evaluative analysis for each criterion depending on the evaluation purpose. Data availability, resource constraints, timing, and methodological considerations may also influence how (and whether) a particular criterion is covered.

Evaluation Approach and Methodology

The evaluation will use of Participatory Action Research (PAR) approach that maximizes learning and action. An Evaluation Advisory Committee (EAC) will be formed comprising key project stakeholders to guide the process of internal evaluation. The evaluation will use multiple lines of evidence, both qualitative and quantitative, to triangulate findings. For primary qualitative data, the evaluation team will rely on key informant interviews, group discussions, focus group discussions, and firsthand observation in communities in which the project operates. The study will make use of existing MIM data (e.g. baseline and monitoring data) to develop data gathering instruments. Composition of the FGDs will be mixed and women only. Mixed groups can cover unpaid care work, so the discussions will help men realize where women need more assistance or where they are burdened whereas there will be women only for sensitive topics such as decision making and GBV. Data gathering tools and findings must also capture progress and results related to such cross-cutting themes as gender equality and environmental sustainability.

Special attention to the mechanisms and tools used to achieve the results including vouchers, matching grants and asset financing (including the loan capital fund) would be critical to assess their contribution and recommendations for replication. The EAC should include, at a minimum: Project Manager, Country Director, Project Technical Manager, Project Agribusiness and Environment Coordinator and HQ MIM Program Manager and HQ Gender Program Manager.

Sample Size for Quantitative Survey

The evaluation will be carried out with the small producers that accessed the e-voucher and the agribusinesses that benefited from the business innovation grants.

In the case of small farmers, the project conducted the baseline on a sample of 148 producers out of a population of 8,000 producers who requested electronic vouchers. The margin of error was 8%, the confidence level was 95% and the distribution of responses was 50%. To carry out the final evaluation for this group, the project will take a sample of 139 producers out of a population of 1,746 producers who took and redeemed the electronic vouchers. The sample size was calculated using again the 8% margin of error, as for the baseline. The confidence level and distribution of responses were also kept the same.

A stratified sampling methodology will be chosen, and the population will be divided into 6 groups or strata. To constitute the strata, we will use the 3 cohorts used to carry out the annual surveys divided by sex, hence the 6 strata. This will allow us to understand the differences between the strata according to the sex of the project's clients. In a stratified sample, the population is divided into homogeneous subpopulations called strata based on specific characteristics (e.g., race, gender identity, location, etc.); in the case of Technolinks+ project it will be (cohorts and sex). Every member of the population studied will be in exactly one stratum. Each stratum is then sampled using the Probability Proportional to Size (PPS) Sampling method, allowing researchers to estimate statistical measures for each sub-population and the entire population. Researchers rely on stratified sampling when a population’s characteristics are diverse, and they want to ensure that every characteristic is properly represented in the sample. How women and men will be represented (%) will be given by the PPS sampling method. For agribusinesses, including women farmer’s agribusinesses, and cooperatives included in the matching grant, all clients of these mechanisms will be included, and the annual survey instrument will be applied to them.

Evaluator's Tasks & Responsibilities

The Final Evaluation will be carried out by an external Evaluation Team (consultant /firm) under the overall guidance of the EAC. The Evaluation Team will prepare a detailed Evaluation plan that provides information on the evaluation to be carried out with timelines of each activity. At minimum, the Evaluation plan should include the following:

• Overview of the project
• Evaluation methodology, including sampling strategy and research questions
• Data gathering tools
• Data analysis and reporting
• Evaluation schedule/timeline

Team members will develop data collection tools in their areas of responsibility and will also prepare presentations for a final, field-based validation and analysis workshop. The team members will analyze the findings and formulate the recommendations linked to the findings.

The Evaluation Team will:

• Organize and lead the overall evaluation.
• Ensure a thorough review and analysis of available secondary data by the appropriate team member(s),
• Develop relevant qualitative/quantitative data collection tools with set of questions covering each evaluation criteria and outcome indicators.
  • Explain using simple terms, interpret the questions to the women Farmers don’t use vocabularies with big words to enable them to understand more, the enumerators should be trained well on how to ask the questions in simple terms and language that the Farmers understand. They should also allow them to explain themselves further, this prompts more discussions
• Lead the selection of a purposely selected sample of activity sites, projects stakeholders and participants for primary data collection and assure adequate triangulation and validation of findings.
• Lead the collection and analyses of primary and secondary data to evaluate the program’s M&E processes and the integration of program sectors and activities.
• Assure that 1) final report presentation is logical and presented in a way that clearly separates findings, conclusions, and recommendations, and 2) all findings, conclusions and recommendations are based on evidence.
• Lead the collection and analyses of primary and secondary technical data related to his/her field(s) of expertise and form recommendations.

The roles and responsibilities of each team leader should be clearly defined.

Evaluator's Qualifications

The evaluation will be performed by a consulting firm including multi-disciplinary team members, one being the designated team leader. The firm shall have prior experience in evaluating similar projects. Experience with GAC (Global Affairs Canada) financed projects is necessary. The evaluator(s) selected should not have participated in the project preparation and/or implementation and should not have conflict of interest with project related activities.

Required qualifications:

• Master’s Degree in Social Sciences, Development studies, Demography, Economics, or related field.
• Minimum 10 years of relevant professional experience evaluating development projects
• Knowledge of GAC principles and projects, in particular Gender Equality Policy and the Feminist International Assistance Policy mandate, objectives and action areas – gender equality and the empowerment of women and girls
• Good reputation and well-known to local and international evaluation practitioners.
• Previous experience with results-based monitoring and evaluation methodologies.
• Technical knowledge in the targeted focal area(s):
  • gender equality and social inclusion.
  • environmental sustainability and climate change.
  • value chain development.
• Fluency in reading, speaking, and writing Spanish and English.
• Excellent report writing skills in Spanish and English.

Desired qualifications:

• Evaluation experiences in Nicaragua with projects funded by GAC.
• Understanding of the development context of the project country.

Evaluation Deliverables

The following products will be produced as part of the evaluation.

• Evaluation Plan. The Evaluation Plan should be developed in consultation with MEDA and partners and include evaluation criteria, questions, indicators, any existing data and / or targets, data sources, instruments, sampling strategy, evaluation methods (data collection tools and analysis) disaggregated by gender and, when relevant, disaggregated by other factors such as geographic location, age, marital status and people with disability; revised schedule/timeline; and ethical considerations. Intersectional evaluation is very important (women are not a monolithic group), should consider intersectional analysis (age, gender, geography) and also the levels of empowerment of the Farmers (ideally project activities are not only “empowering the empowered women” (e.g. more mobile, economically stable women), but also the more disempowered (e.g. more marginalized, lacking mobility, etc.)
• Final Evaluation Scope Finalization Workshop. The Team Lead should hold a 1-day workshop to present the final evaluation plan to all relevant stakeholders.
• Enumerator Training and Launch. The Team Lead will facilitate a 2–3-day training and orientation for all team members and/or enumerators, as decided in consultation with partners and MEDA.
• Ground Truthing/Verification Workshop. The Team Lead will facilitate a one-day workshop to review preliminary results of the field data collection.
• Evaluation Report. The Evaluation Team will produce an evaluation report not exceeding 40 pages. The draft report will include the raw data sets. The report will include, at a minimum, the following:
  • Executive Summary (3-5 pages)
  • Introduction/Background (3-4 pages)
  • Final Evaluation Objectives (1-2 pages)
  • Final Evaluation Methods (including strengths and limitations) (up to 5 pages)
  • Final Evaluation Findings
  • Conclusions (3-4 pages)
  • Recommendations (up to 5 pages)
  • Annexes (e.g., list of acronyms; Final Evaluation ToR; Team Composition; Interview Guides; Bibliography; list of sites visited).
• Final Evaluation Report Presentation.
• A Power-Point presentation with the preliminary findings and recommendations to be presented with MEDA
• Final Evaluation Report Presentation. Once the final reports have been completed, the Evaluation Team will present the final report findings. These presentations will be 2 hours long with participation from the partner teams and key stakeholders’ group.

Logistical Arrangements, Budget & Timeframe

• The local project team will be responsible for making logistical arrangements for field visit and scheduling interviews with stakeholders.
• The project will cover the full costs of the evaluation.
• Payment process: 15% Work plan and methodology delivered, 25% after field collection data, 15% Data report delivered, 45% Final report delivered
• A total of 60 days is allocated for this assignment (excluding travel). All activities will be conducted during the following timeframe.

August 2022: ToR Launch, Consultant Selection, Contract Signature

First Week of September: Kickoff Call with EAC

Second Week of September: Final Evaluation Scope Finalization Workshop, Desk Review & Final Evaluation Methodology and Work Plan developed (sampling, tools, evaluation matrix, etc.)

Third Week of September: Enumerator Training and Launch. The Team Lead will facilitate a 2–3-day training and orientation for all team members and/or enumerators, as decided in consultation with partners and MEDA.

October to Second Week of December: Field work in country

December 2022: Ground Truthing/Verification Workshop. The Team Lead will facilitate a one-day workshop to review preliminary results of the field data collection. Draft Report Submitted

January 2023: Final Report Submitted

Candidates are invited to submit a technical proposal and a financial proposal. Each proposal must be written in English or Spanish. The proposal file shall consist of the following documents:

• Legal papers of the firm (Fiscal Quitus, and Business Permit –Patent-, etc.).
• The letter of invitation
• The letter of submission duly completed
• A document describing the firm and the employees assigned to this study
• The terms of reference of the study
• The technical proposal
• The financial proposal

Technical Proposal
It must be presented in a concise and structured way:

• Legal identity and brief description (technical and financial) of the Firm. In the case of a group of consultants established for the circumstance, a convention indicating the team leader designed to represent the group vis-à-vis of MEDA
• Description of the firm, its technical staff and skills
• Curriculum vitae initialed and signed by all team members
• Understanding of the needs and services required in relation to the terms of reference attached
• Methodological approach, activity timeline
• The number of people they deem necessary for the performance of the required service
• An estimate of the total working time to devote to the service while indicating the number of people
• Schedule of the work and achievements of delivery of outputs.

The Financial Offer
The financial offer should cover all services to be rendered for the study. It must at least include:

• A submission letter duly signed and stamped
• Bank information of the Firm
• The preferred method of payment,
• The preferred payment mean (Bank transfer, Check, etc.).
• Fixed rates per person / days in the field and / or headquarters of the firm
• Per Diem, hotel to stay, training, communication, transportation costs on the field for each team member
• Logistics costs (transportation, etc.)
• Data collection tools/devices costs

Language of the Proposal

The bid prepared by the bidder and all correspondence and documents relating to the bid exchanged by the bidder and the MEDA will be written in English or Spanish**.** Priority will be given to local and national consultants and firms.

APPLICATION DEADLINE: Tuesday, August 23 at 4:00 pm EDT (applications will be considered on a rolling basis).