Statement of Work: Evaluation of the Safe from the Start Initiative

Nature and Purpose of the Evaluation

The purpose of this solicitation is to obtain the services of a contractor to carry out an evaluation, lasting up to twelve months, of PRM’s Safe from the Start (SftS) Initiative^[1], an initiative launched in 2013 as a United States commitment to the Call to Action to better address the needs of women and girls from the onset of humanitarian emergencies. This investment has been channeled to interventions that improve upon and build the capacity of the humanitarian system– training for staff, deployment of experts to high-level emergencies, implementation of life-saving responses and services, and improving the capacity of selected multilateral and NGO partners to prevent and respond to gender-based violence (GBV). This evaluation will focus on the changes—programmatic, operational, and otherwise—that have been supported and prompted by SftS from 2013-present. It will focus specifically on the operations of UNHCR, IOM, UNICEF, and International Medical Corps as the largest recipient of funds and longest-standing partners.

Protection is fundamental to the mission of PRM and GBV has been among its highest priorities. By dedicating robust and new resources, the United States has been a leader in preventing and responding to GBV, from taking on leadership of the Call to Action in 2014, to setting aside specialized funds to address this issue, educating other donors and policymakers, and elevating the issue as a priority within the humanitarian community. PRM has also been front and center talking about investment in women and girls as not only essential but strategic over the long-term. We believe that quality, timely, and effective response to GBV survivors is life-saving and ultimately contributes to a reduction in the incidence of GBV as countries transition into post-conflict or development. Significant investment of Safe from the Start funding has gone to UNHCR, one of PRM’s major international organization partners. ICRC received the next largest investment followed by UN agencies such as UNRWA, UNFPA, UNICEF, and IOM.

While no PRM-funded external evaluation of SftS has been conducted thus far, ongoing monitoring shows that some interventions have been useful and productive while the wider system remains hesitant, unwilling, and unable to implement international standards and guidelines for GBV in emergencies.  This evaluation is intended to give greater insight into progress and roadblocks, as well as provide concrete inputs to PRM to better tailor, design, and build upon the initiative for years to come. It will consist of: (1) an analysis of reports and evaluations initiated and submitted by PRM partners; (2) analysis of consultations with key individuals at the headquarters of selected partners as well as in the field; and (3) the consolidation of recommendations that will inform and strengthen the ability of the SftS initiative to be more impactful going forward.

The desk review, consultations, and analysis will focus on the following areas: (1) ways in which SftS investments have changed the overall humanitarian system addresses violence against women and girls in emergencies and how it works or doesn’t work; (2) what changes need to be made so that PRM partners and organizations are better situated (i.e. staffing, structure, capacity, coordination, etc.) to quickly and effectively respond to GBV in emergencies; and (3) stories from the field that illuminate the effect (positive or negative) that SftS activities and interventions have had. The primary audience for the evaluation is PRM although PRM partners along with OFDA may also benefit from possible findings and recommendations. Evaluation findings shall be targeted, concrete, actionable, and tailored to the work of PRM (and its partners) and the particular focus of the initiative. Unless doing so creates security or raises other concerns, the final evaluation report will be shared publicly and/or posted for external consumption.

Background and Current Status of the Effort

PRM

PRM’s mission is to provide protection, ease suffering, and resolve the plight of persecuted and uprooted people around the world by providing life-sustaining assistance, working through multilateral systems to build global partnerships, promoting best practices in humanitarian response, and ensuring that humanitarian principles are thoroughly integrated into U.S. foreign and national security policy on behalf of the American people. At PRM, we understand and acknowledge GBV as a key issue that must be addressed at the earliest stages of every emergency and believe that every actor has an important role to play. Importantly, no single organization or agency can respond to the enormity and diversity of needs that exist in the field. We work to prevent and respond to GBV in holistic ways so that during and at the onset of crises, the most vulnerable are safe.  By protecting at-risk groups, particularly women and girls, we advance gender equality and build stronger, more resilient families and communities that can more quickly heal, recover, and thrive after disasters and crisis.

Launched in 2013, SftS complements and reinforces existing government-wide gender and protection policies through financial, human resource, and programmatic investment and good policy as it relates to violence against women and girls. Time and again, we see that humanitarian actors are not prioritizing or acting quickly enough to address the comprehensive needs of GBV survivors. Inaction, poorly designed programs, a lack of monitoring, and misunderstanding of do no harm and ethical principles continue to be obstacles to quality programming and timely action. That is why PRM continues to support collective and collaborative efforts, develop new partnerships, and promote mutual accountability to basic humanitarian responsibilities.

The goal of SftS is to reduce the incidence of GBV and ensure quality services for survivors from the very onset of emergencies through timely and effective humanitarian action. In order to meet this goal, the initiative is focused on three mutually reinforcing objectives that were identified through a framework for action: 1) Increase dedicated GBV programming, 2) Integrate GBV risk mitigation across all sectors; and 3) Improve global accountability. The overall purpose is to ensure quality, timely, and effective response to GBV survivors and ultimately a reduction in the incidence of GBV occurring in conflicts or disaster.

Despite efforts to date, the humanitarian system is still slow to respond and reluctant to comprehensively and systematically address GBV at the outset of every emergency. This is why, in FY 2017, PRM continued its significant investment and contributed $17.8 million to the SftS initiative to build and strengthen the core capacity of key humanitarian partners to address GBV in emergencies. This is $5 million more than PRM provided to the initiative and its partners in FY 2016. The funding supported both institutional and NGO partners, including UNHCR, UNICEF, ICRC, IOM, UNRWA, WHO, Action Against Hunger, CARE, International Medical Corps and the International Recuse Committee to conduct activities such as GBV training for their staff and deploy experts to high-level emergencies for leadership and coordination purposes, as well as to advocate for GBV needs to be included in the earliest stages of a response.

What remains evident, even with PRM investments, is that a combination of education, skills building, accountability measures, incentives, and advocacy still continue to be among the key approaches for ensuring GBV is addressed by the humanitarian community at the outset of every crisis. We must try harder to move beyond individual people and personalities to ensure more consistency and systematic response, particularly by UN agencies. Safe from the Start funding supports PRM’s Functional Bureau Objective 1.2: Mobilize the international community to respond to gender-based violence (GBV) as a life-saving priority in emergencies through enhanced coordination and service provision. PRM reports annually on this objective through the global indicator: “Percentage of emergency declarations responded to through deployments led by PRM-funded Safe from the Start partners.” This indicator measures the number of deployments from IO partners who receive PRM funding for GBV to ongoing Level 3 (L3) emergencies.^[2] In FY 2017, the result of this measure was 67% against a target of 60%.  The result is an aggregate of the deployment percentages of the key Safe from the Start partners.

The contractor of this evaluation will report to and coordinate closely with PRM’s Office of Multilateral Coordination and External Relations (MCE) and Office of Policy and Resource Planning (PRP).  MCE manages the institutional relationships with PRM’s multilateral and non-governmental partners and houses PRM’s GBV subject matter experts, as well as manages all of PRM’s global GBV projects.  PRP tracks cross-cutting issues and coordinates the evaluations that are undertaken within the Bureau. The two offices work closely together, with a shared vision of protecting and empowering women and girls, and will jointly support this evaluation to ensure it meets internal PRM needs and is of the highest quality possible.

Evaluation Questions

With the creation of Safe from the Start, the United States has continued its leadership in the promotion and response to GBV in emergencies. We invest in protection and empowerment programming, train and highlight the work of first responders, and build in safety mechanisms into humanitarian systems through international organization and non-governmental partners. We also believe that we have accomplished a great deal that is specific to this initiative. We work with agencies like UNHCR to undertake training for their staff and deploy experts to high-level emergencies for leadership and coordination purposes. We also advocate for gender-based needs to be included in the earliest stages of an emergency response. That being said, no formal evaluation has confirmed as such and this evaluation is meant to generate evidence to corroborate, challenge or add nuances to the assumptions underpinning the SftS initiative. The ultimate goal is to build upon, refine, and tweak the initiative in ways that will better respond to needs in the field and maximize PRM investment to make the biggest impact possible within the wider community.

This evaluation shall answer the following questions with an emphasis on documenting stories of change, informing the future focus and scope of the initiative, and capturing lessons that will help both PRM and its partners make the most of future SftS commitments:

1. To what extent are PRM’s Safe from the Start investments meeting their intended aims?
2. What evidence exists, if any, that PRM’s Safe from the Start investments have affected the way the humanitarian community addresses violence against women and girls in emergencies (and in what ways)?
3. What changes or updates need to be made to ensure PRM’s Safe from the Start investments are as strategic, sustainable, and impactful as possible?

Evaluation Design and Data Collection Methods

It is expected that this evaluation will involve mostly qualitative methods (i.e. stories of change, etc.) with the inclusion of quantitative approaches, where possible (i.e. evaluation of indicators, etc.). PRM will rely on the contractor to both propose and design the research methodologies that make the most sense and answer research questions in the best and most efficient way possible. It is anticipated that methods will include some combination of desk research/analysis, key informant interviews, group discussions (as relevant), and collection of case studies or stories to produce findings, draw conclusions, and present recommendations. Participation of “beneficiaries” is required and should be defined by the consultant within the context of a global initiative. It is essential that all research is conducted in line with ethical and safety standards and be done in a way that upholds PRM’s commitment to professionalism. All data collection methods and tools shall be shared with PRM in advance and strive to get the richest and most useful information in the quickest way possible. This is in recognition of the limited time available and significant workload of PRM partners, particularly those that work in the field in ongoing humanitarian emergencies.

Evaluation Team

PRM is open and will consider various evaluation team compositions based on proposals submitted by the consultant. However, the overall team responsible must consist of one Senior or Mid-Level Evaluation Advisor or Evaluation Methods/Implementation Specialist and one Senior or Mid-Level GBV Expert (preferably with experience in humanitarian or emergency settings for both positions).  The persons filling these positions, and more specifically those who are responsible for field visits and interviews, should also be familiar with refugee and displaced contexts given the mandate and emphasis of PRM’s work.

One team member must be designated as a Team Lead who will oversee and be the focal point for the project. There must also be a U.S.-based position that is positioned to easily and reliably liaise with PRM on a regular basis on the status of the evaluation. The evaluation team’s knowledge, attitude, and skills must demonstrate the ability to complete the following:

1. Initiate a kick-off meeting with PRM technical staff to understand the SftS initiative and its goals within the humanitarian field, as well as agree on expectations for communication/involvement moving forward.
2. Undertake a comprehensive desk review of PRM SftS projects and programs, including an examination of project reports, evaluations, and products that have been developed/ submitted during the above timeframe.
3. Introduce the SftS evaluation to relevant partners and stakeholders to explain its intention, scope, and purpose as well as to answer any questions they have.
4. Initiate headquarters and field-based evaluations to understand the way in which SftS has been implemented and how it has affected the state of play in refugee and displaced settings.
5. Analyze collected data, design targeted recommendations, and produce a succinct final report that is informed by PRM and partners through regular consultation and engagement.
6. Debrief PRM, partners and other stakeholders, at various points throughout the process and once the final report is completed, as agreed to with PRM.

Qualifications

1. The Evaluation Advisor/Specialist should have experience designing and implementing evaluations in complex humanitarian settings (i.e. refugee camps) and with displaced populations. The GBV Expert should have experience in conducting research on GBV and/or working on GBV programming and mainstreaming in humanitarian and emergency operations. The Team Lead should have experience and a track record of managing large-scale evaluations in a professional, effective, and collaborative manner. Staff not meeting these requirements may be considered in special circumstances, determined by PRM.
2. The evaluation team must be proficient in English (speaking and writing), familiarity of both technical and humanitarian terms, and an ability to translate concepts as well as write in ways that are easily understood by a public audience. For field evaluations that include interviews with people of a different language, the evaluation team must have someone proficient in that language and/or an ability to identify and hire high-quality interpreters.
3. The evaluation team must have prior understanding of the mandate/responsibility/work of State Department/PRM and its partners in protecting and empowering women and girls as well as familiarity with other relevant evaluations (best practices, lesson learned, etc.) in order to prevent duplication, overlap, and anticipate potential challenges.
4. The evaluation team must have a thorough understanding and commitment to upholding ethical and safety standards as it relates to researching GBV in humanitarian contexts. Familiarity with existing guidelines and a plan for how to protect confidentiality is essential. Under no circumstances should survivors of GBV be specifically interviewed or recruited for this evaluation given the existing burden on these individuals as well as the additional precautions required for research of this kind.
5. The evaluation team should have experience in participatory research, including the stories of change methodology. As storytelling, case studies, and other approaches of this kind should be used as a primary method of data collection and documentation, the team members should be similarly oriented in these approaches. Communication, listening, and collaboration will be characteristics highly valued in the project evaluation team.
6. As the recommendations of this evaluation shall be tailored to the United States government, and more specifically PRM, as one of the originators of the initiative, familiarity with donor institutions and UN agencies is beneficial. As PRM aims to create an initiative that is both effective and impactful vis-à-vis other donors and funding mechanisms, the evaluation team would benefit from including someone with relevant familiarity and understanding, where possible.
7. Although survivors will not be proactively identified, a commitment to a survivor-centered approach is imperative throughout the evaluation. This includes ethical approaches to data collection such as getting informed consent, maintaining confidentiality, and ensuring referral pathways are established before interviewing, etc. As such, team members should be familiar and comfortable taking this approach as well as articulating themselves in this way.
8. Evaluation team members may be based in or outside of the U.S. U.S. citizenship is not required.
9. Security clearance is not required.
10. Contract organizations and people with prior experience and knowledge or PRM is beneficial. In such circumstances, lessons learned from past evaluations shall be taken into account and integrated into the research design.

Timetable and Deliverables

The contractor will begin work within 2 weeks after the contract award.  The duration of the evaluation will be up to 12 months with a planning, implementation, and completion phase.

The contractor shall provide the following outputs and deliverables to PRM throughout the evaluation period. The timeframe and due dates for each deliverable will be informed by the evaluation start date and design. However, projected or expected delivery dates shall be provided within the original proposal.

1. Kick-off Meeting: A kick-off meeting between PRM and the contractor should be scheduled as soon as possible after the award is signed. The meeting will take place at PRM’s office in Washington, D.C. The contractor is expected to develop a meeting agenda prior to the meeting and come prepared with documents, ideas, and plans to discuss at the meeting.
2. Evaluation Work Plan: A detailed work plan with is due within two weeks of the kick off meeting. This plan should be as detailed as possible, with a detailed timeline indicating where and when PRM will be consulted or informed, throughout. The work plan shall include the anticipated schedule and logistical arrangements and delineate the roles and responsibilities of members of the evaluation team.
3. Progress Reports: The contractor is required to submit 3 progress reports to PRM, as detailed in the work plan, as well as monthly teleconferences. The reports should summarize progress to date and status of the major activities being undertaken in relation to the work plan; any deviation from the work plan with associated explanations; indications of any problems encountered or expected, and proposals for corrective action, as appropriate; and projected activities for the coming reporting period. These reports are due 3 months, 6 months, and 9 months from the start of the contract. The monthly phone calls will be used as opportunities to problem-solve where problems arise, identify areas for additional PRM support or assistance, and be held for the purposes of creating mutual understanding and preventing misunderstanding down the line.
4. Desk Review Report: A desk review of partner reports, products, and publications is due within 90 days after the start of the contract. The desk review shall include an updated evaluation design document as an annex. The evaluation design document will include a detailed evaluation design matrix (including the key questions, methods, and data sources used to address each question and the data analysis plan for each question and known limitations to the evaluation design. It may include draft questionnaires and other data collection instruments. The desk review and shall not exceed 20 pages (maximum), not including annexes. PRM will provide feedback on the draft report within 10 business days. The contractor then has 10 business days to finalize the report. Phone calls can be used to resolve any disagreements or provide clarification. A report revisions matrix must also be submitted with the final report, unless discussed and waived by PRM in advance.  The report shall be in compliance with section 508 of the Rehabilitation Act.
5. Field-based evaluations: The field-based evaluations shall include three country visits: Geneva, Switzerland, where the majority of SftS partners are headquartered, and two countries, selected by the contractor, in consultation with PRM and relevant partners, which have been a part of the SftS initiative. Potential options, to be decided within 2 weeks of contract kick-off meeting, include: Ethiopia, Kenya, Uganda, South Sudan, Jordan, Iraq, or Turkey (with Uganda being a priority). The total time spent on the field shall be no more than 5 weeks, not including travel days. This is in recognition of the lack of time and availability of both HQ and field staff to partake in evaluations, and the importance of their field work. This allows time for consultation with SftS partners, program “beneficiaries” (to be determined by the contractor), and other stakeholders (such a NGOs, experts, working groups, etc). With PRM support, the contractor will consult with the U.S. Embassy prior to data collection activities in-country. The evaluators are required to coordinate closely with PRM’s Humanitarian Affairs unit in Geneva and PRM’s Regional Refugee Coordinators to schedule in-country meetings and interviews with PRM partners. When in the field, a six day work week is authorized.
6. Final Report:
   1. The contractor should deliver a draft final report incorporating findings from the desk review and field evaluations at least 45 days before the completion date of this contract. The final report shall include the following:
      1. Executive summary
      2. Description of the SftS initiative
      3. Evaluation purpose and scope
      4. Evaluation design and data collection methods
      5. Data and findings
      6. Conclusions
      7. Recommendations
      8. Annexes: SOW, research instruments, ethical considerations, details about data collection, etc.
   2. In addressing the evaluation questions, the final report shall include the ways in which SftS has been impactful, where the initiative has not filled or addressed major gaps, and in what ways the initiative can be tweaked to address ongoing and new challenges in the field of GBV in emergencies.
   3. The final report shall include stories, case studies, and examples that show the breadth and depth of work undertaken by SftS. They should also be used to explain and detail how the initiative has worked, supported good practice, or changed the way that partners or staff do their work.
   4. Recommendations shall be concrete, actionable, and tailored to PRM so as to inform the initiative and funding decisions going forward. Any additional recommendations are separate and should be addressed to a specific partner or stakeholder so that it can be shared, appropriately. Recommendations that are detailed, extremely relevant, and limited in number, are preferred.
   5. The final report shall include reference to how SftS meets PRM’s Functional Bureau Strategy (FBS) objectives, how it connects to funding provided by other donors, and in what ways it has aligned with current gaps in the field. PRM will provide the contractor with an electronic version of the FBS as well as any other strategy documents that may be useful.
   6. The final evaluation report shall be no more than 30 pages in length, not including annexes. The final report must also include an executive summary, which shall be no more than 2 pages. The goal is to have something readable, useful, and targeted so shorter is better.
   7. PRM will provide feedback on the draft report within 10 business days. The contractor then has 10 business days to complete the report after it is returned by PRM. A revisions matrix must also be submitted with the final report, unless waived by PRM. The final report is intended to be shared publically on the State Department’s foreign assistance evaluations web page and must be in compliance with section 508 of the Rehabilitation Act.
7. Evaluation Summary for Dissemination: A summary of the evaluation shall be written to be made available for a public audience and sent separately from the evaluation report. This may be the executive summary, if appropriate, for an external audience. The summary should be brief, not more than two pages, and should include a short overview of the evaluation, evaluation questions, data collection methods, findings, and recommendations. It should not include confidential issues or anything that is deemed inappropriate or irrelevant to the public. PRM will provide a template for this summary. It should be submitted before the completion date of this contract and made available to PRM for review beforehand.
8. Oral Briefs
9. 1. Monthly teleconferences shall be initiated and scheduled by the contractor as opportunities to review performance against the detailed work plan. These calls should be used less for the purposes of updates than working through any problems, challenges, or delays. As phone calls are ideal for collaboration, any assistance or support needed from PRM should be expressed during this time.
   2. One debrief meeting shall be scheduled with PRM before departure at each field location. If appropriate, other relevant stakeholders and partners should attend these meetings. With permission, this debriefing can also happen remotely with advance notice and justification provided to PRM.
   3. One final presentation to PRM prior to the completion of the contract. OFDA should be invited to this meeting as well as other stakeholders including IOs and NGOs, as appropriate. The presentation will take place at PRM/Washington, DC.

Expectations

1. The contractor shall maintain open, respectful, timely, and collaborative communications with PRM, resulting in a relationship that proactively addresses potential or current problems with flexible, workable, and appropriate solutions. It is a priority for PRM to be a good and supportive partner in this evaluation.
2. The contractor shall be responsive to PRM throughout the project, and demonstrate ability to design an evaluation, analyze findings, and present results in line with the needs of the bureau. When in doubt, the contractor shall reach out to discuss questions with PRM in a proactive and transparent way.
3. The contractor shall provide all documentation and reports to PRM for review and clearance prior to disseminating to or sharing with beneficiaries, UN agencies, NGOs, or other evaluation participants. This is for the purposes of ensuring a targeted, clear, and coherent evaluation approach and not to influence or change the data in any way.
4. The contractors shall coordinate with, and be responsive to, PRM in all aspects of project management and implementation. The contractor is expected to answer communication and submit agreed upon deliverables on time.
5. Proceeding the desk review and analysis, the contractor shall present findings, results, conclusions, recommendations. At all times, qualitative stories, case studies, and examples should be used to show the breadth and depth of activities. This will be used going forward to help PRM explain the SftS initiative and share our work with others.
6. The contractors shall forward all project deliverables to PRM according to the final timeline, pending unforeseen delays. When there are unforeseen delays, or other project or financial issues arise, the contractor must inform PRM immediately.
7. The contractor shall deliver high quality final products (deliverables) suitable for PRM purposes. The products shall be professional, well written, and tailored to influence and inform PRM’s SftS initiative and work. Other potential targets for the report include IO’s, NGO’s, other donors/governments, etc.  The final evaluation is considered a PRM product. PRM also decides who will receive the report and in what way.

Security

PRM does not anticipate that the evaluation team will face security concerns in Geneva, Switzerland.  However, safety precautions should be taken and will be necessary according to the security situation in the second and third field evaluation country, and must be tailored as such. The evaluation team should explain what security measures will be taken to ensure their safety and navigate any safety challenges while in country.

Location

With the exception of field evaluations, project activity is anticipated to take place at the contractor’s place of work. Data collection and analysis will take place in the United States, Switzerland and Ethiopia, Kenya, Uganda, South Sudan, Jordan, Iraq, or Turkey (with a priority being Uganda). Prior to the desk review, the evaluation team will visit PRM for consultations at State Department Annex (SA- 9) at 2025 E Street NW in Washington, DC. All relevant reporting required for the desk review will be provided to the contractor once the contract is issued.

Logistics Support

PRM will provide the contractor with access to relevant program documents, including those not in the public domain such as reporting, strategies, etc. once the award has been issued. PRM will also provide the contractor with contact information of PRM (Headquarters and field) and partner staff as well as facilitate introductions, where needed. In addition, PRM will provide evaluation report and summary templates. All other support will not be of a logistics nature.

Other Information

The evaluation report and its findings are proprietary and will not to be made public or shared externally without the consent of PRM.  PRM reserves the right to disseminate and circulate the evaluation report to colleagues (USG, international, and NGO partners), as determined appropriate.  PRM may also choose to post the final report on the Department’s internet site for further visibility.  The contractor will be acknowledged on all circumstances.

[1] The Safe from the Start initiative is a joint one implemented by PRM as well as the Office of Foreign Disaster Assistance (OFDA) at USAID. This evaluation will focus on PRM’s investments and programs with a recognition that the initiative is wider in scope. Where possible, coordination and consultation with OFDA colleagues, is critical.

[2] System-wide L3 responses are activated when the highest level of mobilization is required, across the humanitarian system, to ensure that the right capacities and systems are in place to effectively meet humanitarian needs.