External evaluation 2008-2012: advancing sexual and reproductive health

Publication date: 2013

External evaluation 2008–2012 Advancing sexual and reproductive health WHO/RHR/HRP/13.07 Contents Detailed contents and page numbers are provided within each chapter Chapter 1. Introduction . 1 Nicholas Dodd Consultant, Sexual and Reproductive Health Chapter 2. Overall assessment of HRP’s relevance and effectiveness . 23 Nicholas Dodd Consultant, Sexual and Reproductive Health Chapter 3. Efficiency and effectiveness of HRP’s governance, management and administration . 71 Nicholas Dodd Consultant, Sexual and Reproductive Health Chapter 4. Evidence generation and synthesis to improve family planning, prevent unsafe abortion and prevent and control sexually transmitted and reproductive tract infections . 101 Affette McCaw‐Binns Professor of Reproductive Health and Epidemiology and Head, Department of Community Health and Psychiatry, University of the West Indies, Jamaica Jasneth Mullings Research Fellow, Department of Community Health and Psychiatry, University of the West Indies, Jamaica Chapter 5. Research‐capacity strengthening and network building . 153 Chisale Mhango Senior Lecturer, Department of Obstetrics and Gynaecology, College of Medicine, Blantyre, Malawi Chapter 6. Strengthening implementation research . 183 Fernando Althabe Director, Department of Mother and Child Research Institute for Clinical Effectiveness and Health Policy in Buenos Aires, Argentina Daniela S Colaci Researcher, Department of Mother and Child Research Institute for Clinical Effectiveness and Health Policy in Buenos Aires, Argentina Chapter 7. The status of, and opportunities for strengthening, engagement with the private sector and civil society . 219 International Development Team Pricewaterhouse Coopers SA . Geneva, Switzerland Chapter 8. Bibliometric analysis . 245 Karen Gurney Manager, Bibliometric Reporting, Thomson Reuters (Evidence), Leeds, United Kingdom of Great Britain and Northern Ireland Bethan Sheridan-Jones Data Analyst, Thomson Reuters (Evidence), Leeds, United Kingdom of Great Britain and Northern Ireland Simon Thomson Data Analyst, Thomson Reuters (Evidence), Leeds, United Kingdom of Great Britain and Northern Ireland Page | 1 Chapter 1 Introduction Nicholas Dodd Consultant, Sexual and Reproductive Health Page | 2 Page | 3 Contents 1 Background to the Special Programme of Research, Development and Research Training in Human Reproduction . 5 2 Previous external evaluations of HRP . 6 3 The current external evaluation of HRP . 7 References . 9 Annex 1: Terms of reference for the 2008–2012 external evaluation of HRP . 11 Background . 11 Proposal for HRP external evaluation 2008–2012 (to be presented in June 2013) . 13 Annex 2: the Memorandum of Understanding of HRP . 16 Page | 4 Abbreviations HRP UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction ICPD International Conference on Population and Development IEGWB Independent Evaluation Group of the World Bank MDG Millennium Development Goal MSH Management Sciences for Health OED Operations Evaluation Department (World Bank) PCC Policy and Coordination Committee PDRH Programme Development in Reproductive Health PEEC PCC External Evaluation Committee RHR WHO Department of Reproductive Health and Research SCIH Swiss Centre for International Health SRHR sexual and reproductive health and rights STAG Scientific and Technical Advisory Group UN United Nations UNDP United Nations Development Programme UNFPA United Nations Population Fund UNICEF United Nations Children’s Fund WHO World Health Organization HRP External Evaluation 2008−2012 Chapter 1 Page | 5 1 Background to the Special Programme of Research, Development and Research Training in Human Reproduction The Special Programme of Research, Development and Research Training in Human Reproduction (HRP) was established by the World Health Organization (WHO) in 1972 to coordinate, promote, conduct and evaluate international research in human reproduction. Sixteen years later, in 1988, HRP became a cosponsored programme, with an explicit mandate for: • promoting and supporting research aimed at finding and developing safe and effective methods of fertility regulation, and identifying and eliminating obstacles to such research and development; • identifying and evaluating health and safety problems associated with fertility regulation technology, analysing the behavioural and social determinants of fertility regulation, and testing cost-effective interventions to develop improved approaches to fertility regulation within the context of reproductive health services; • strengthening the training and research capability of developing countries in the field of human reproduction; • establishing a basis for collaboration with other programmes engaged in research and development in human reproduction, which will include the identification of priorities across the field and the coordination of activities in the light of such priorities (1). The original four cosponsors were the United Nations Development Programme (UNDP), the United Nations Population Fund (UNFPA), the World Bank, and WHO, and the Memorandum of Understanding agreed to by these parties (see Annex 2) remains today as the document that guides HRP’s overall purpose as well as its governance. HRP is the only body within the United Nations (UN) system mandated to lead research in human reproduction. It works in close association with countries to provide the answers to critical sexual and reproductive health questions, resulting in the generation of evidence-based research findings and the information needed to achieve universal access to effective services and to enable people to protect and promote their own sexual and reproductive health. Since 1998, HRP has functioned as one of the two components in the Department of Reproductive Health and Research (RHR). The second component is Programme Development in Reproductive Health (PDRH), whose function became to facilitate the application of research results to policy and practice in sexual and reproductive health programmes in countries, thereby raising the value of HRP’s outputs, and increasing the effectiveness of WHO’s work in sexual and reproductive health. HRP’s modus operandi remains largely unchanged since its inception. It develops 5-year strategic plans and biennial workplans and budgets, to implement a prioritized global research agenda, and to provide support to strengthen research capacity in institutions in programme countries. Its budgetary support for these two items remains at 2:1, respectively for research and for capacity strengthening. Its proposed programme of work is reviewed by its Scientific and Technical Advisory Group (STAG), and its Policy and Coordination Committee (PCC) provides oversight for funding, policy and programme management. The content of HRP’s work, however, has changed substantially over the years, from an early focus on developing new methods of fertility regulation and clinical studies on the safety and efficacy of new and existing methods of fertility regulation, to an agenda focused on the broader spectrum of sexual and reproductive health, including: family planning; maternal and perinatal health; sexually transmitted and reproductive tract infections; preventing unsafe HRP External Evaluation 2008−2012 Chapter 1 Page | 6 abortion; and gender, reproductive rights, sexual health and the sexual and reproductive health of adolescents; as well as research on implementation of the sexual and reproductive health products and tools the Programme produces. HRP’s overriding vision is: . . . the attainment by all peoples of the highest level of sexual and reproductive health. We strive for a world where all women’s and men’s rights to enjoy sexual and reproductive health are promoted and protected, and all women and men, including adolescents and those who are underserved and marginalized, have access to sexual and reproductive health information and services (2). Over the last 20 years, the global landscape on sexual and reproductive health has progressively developed. HRP was a key resource to these processes and then incorporated their outcomes fully into its work. They included: the Programme of action of the 1994 International Conference on Population and Development (ICPD) in Cairo (3), and its various follow-up mechanisms; the Beijing Declaration and platform for action of the Fourth World Conference on Women in 1995 (4), which reaffirmed the reproductive health agenda, and its follow-up mechanisms; the Millennium Development Goals (MDGs) (5), including the new target 5B, on universal access to reproductive health; the UN Secretary-General’s Global strategy for women’s and children’s health (6); the WHO Global reproductive health strategy, adopted at the World Health Assembly in 2004 (7); the report of the WHO Commission on the Social Determinants of Health (8), which focused on health inequities; the WHO Global strategy for the prevention and control of sexually transmitted diseases: 2006 – 2015 (9); and, most recently, the 2012 London Family Planning Summit. Many regional commitments also ensued, for example the Maputo Plan of action on sexual and reproductive health and rights (10) for a continental policy framework for sexual and reproductive health and rights in Africa. By both helping to guide global change and then adapting its work to such change, HRP has continued to demonstrate that its business model functions very well, and that it remains relevant to the needs of programme countries. As well as conducting research, and building capacity to conduct research, the Programme also synthesizes research through systematic reviews of the literature, and develops tools that facilitate access, by countries and by individuals, to the latest research information. Research guides policies and programmes, and nowhere is this more important than in the area of sexual and reproductive health. Without research, policies become obsolescent, and information and service programmes do not provide people with the latest information, or the best technologies and tools to ensure their sexual and reproductive health. 2 Previous external evaluations of HRP HRP has a long-standing culture of regularly submitting its work and functioning to external evaluations, of which the last two are briefly reviewed here. In 2002, an external evaluation, jointly conducted by Management Sciences for Health (MSH) and the Swiss Centre for International Health (SCIH) of the Swiss Tropical Institute, focused on four key issues (10). These were: the relevance and effectiveness of HRP-supported research in reproductive health; the dissemination, global use and impact of the results of HRP’s reproductive health research; reproductive health research-capacity strengthening by HRP and the use and impact of HRP’s work at country level; and the HRP governance process, management, administration and efficiency. Two thematic case-studies were also undertaken, the first on emergency contraception and the second on mainstreaming gender and women’s perspectives. The evaluation examined HRP’s work between 1990 and 2002. HRP External Evaluation 2008−2012 Chapter 1 Page | 7 The overall conclusion of this evaluation was that: HRP clearly met expectations in terms of its core mission to coordinate, promote, conduct and evaluate international research in reproductive health and achieved its major objectives. The Programme maintained its position as the global leader in generating research results and establishing the scientific consensus needed to advance reproductive health policies and practices, especially for developing countries (11). The most recent external evaluation reviewed the period 2003 to 2007 (12), and was conducted by a team of eight international experts during the course of 2006–2007. The overall focus of the evaluation was on the impact of the Programme on global public goods, and this was examined in more detail through five case-studies on: the long-term safety and effectiveness of the copper-releasing intrauterine devices; improving the quality of family planning care in China; medical (non-surgical) abortion; improving maternal and newborn health; and knowledge synthesis and transfer. In addition, the evaluation also reviewed HRP’s follow-up actions to the recommendations of the previous evaluation in the areas of governance, management, administration and efficiency. The overall conclusion of the 2003–2007 evaluation was that: HRP remains a global leader in sexual and reproductive health research and capacity building, with particular relevance to the needs of populations in resource-poor settings. The evidence base resulting from this research has been translated effectively into health policy changes and improved practice standards and has ultimately improved health outcomes. 3 The current external evaluation of HRP The current evaluation was requested by the World Bank at the 71st meeting of the standing committee in June 2011. At this meeting, the cosponsors agreed on draft terms of reference, elaborating an approach that would review the comparative advantage of HRP and its impact in improving outcomes and influencing evidence-based changes in sexual and reproductive health policies and programmes, as well as carrying out a number of case-studies. The standing committee also recommended the establishment of a PCC External Evaluation Committee (PEEC) to include: the chair and vice-chair of PCC at its 24th meeting on 16–17 June 2011, one representative of the HRP financial contributors, the chair of RHR’s STAG, and one representative of the four HRP cosponsors, in order to oversee the process of the evaluation. Terms of reference for the evaluation were subsequently shared with PCC members for feedback, and finalized at the 72nd meeting of the standing committee in December 2011. The current evaluation of HRP covers the period 2008 to 2012, and the full terms of reference can be found in Annex 1. The evaluation reviewed HRP’s overall relevance and effectiveness, particularly in terms of producing global public health goods, and the efficiency and effectiveness of its governance, management and administration (Nicholas Dodd, who also acted as overall coordinator of the evaluation), as well as conducting the four case-studies. These examine the following issues: evidence generation and synthesis to improve family planning, prevent unsafe abortion and prevent and control sexually transmitted diseases and reproductive tract infections (Affette McCaw-Binns and Jasneth Mullings, Chapter 4); research- capacity strengthening and network building (Chisale Mhango, Chapter 5); strengthening implementation research (Fernando Althabe and Daniela Colaci, Chapter 6); and the status of, and opportunities for strengthening, engagement with the private sector and civil society (International Development Team, Pricewaterhouse Coopers SA, Chapter 7). HRP External Evaluation 2008−2012 Chapter 1 Page | 8 The evaluation was carried out through a combination of desk reviews of documentation; interviews with key informants; questionnaires; and site visits. The methodology for the evaluation tried to follow the example of the Programme itself. It attempted to be systematic, and to use an evidence-based approach, relying on quantitative data, supplemented wherever possible by qualitative information as appropriate. Further details of the specific methodologies used for each section of the evaluation, as well as findings, conclusions and recommendations, are to be found in the respective chapters. The evaluation also included the commissioning of a bibliometric analysis, which is discussed further in Chapter 2 of the evaluation report. In June 2012, at the time of the 25th meeting of PCC, the majority of the evaluation team was able to travel to Geneva to have direct consultations with HRP staff, to plan and strategize for the evaluation among themselves, to develop and agree on a timeline, and to interact with the PCC subcommittee (PEEC) charged with overseeing the evaluation, as well as with the wider group of PCC members and observers. At this time, an e-mail was also sent to all RHR staff, inviting their input and suggestions, and, as a result, a number of staff members submitted information on their work and ideas for the evaluation process. A timeline for the evaluation was discussed and agreed, and the consultants subsequently shared chapter outlines for discussion and review in mid-July, and then started working on the content of their respective assignments. Progress reports on the evaluation were submitted to PEEC in August and November 2012. An oral presentation of the evaluation report was made at the 30th meeting of the STAG in February 2013. The full report was subsequently reviewed by PEEC and the standing committee in April 2013 and presented to PCC at its 26th meeting in June 2013. Acknowledgement The evaluation team would like to thank most sincerely all those who gave freely of their time and their knowledge in the process of preparing this report. These include current and former staff of HRP; current and former staff of the cosponsors; current and former members of HRP’s governance committees; directors and staff of research institutions in programme countries that have received support from HRP; all those who responded to questionnaires, including national directors of sexual and reproductive health in programme countries, UNFPA country offices, and numerous civil society organizations. The report benefited enormously from your help and your ideas. Finally, we would like to thank Mike Mbizvo, the outgoing Director of HRP, for his guidance, patience, and constant availability whenever questions needed to be answered, information needed to be provided or advice needed to be sought. HRP External Evaluation 2008−2012 Chapter 1 Page | 9 References 1. Memorandum on the administrative structure of the Special Programme of Research, Development and Research Training in Human Reproduction (HRP). Geneva, World Health Organization, 2012 (http://www.who.int/hrp/governance/HRP_MoU_Admin_Structure_rev_2012.pdf, accessed 24 December 2012). 2. WHO’s strategic vision in sexual and reproductive health and rights. Business Plan 2012–2015. Geneva, World Health Organization, 2010 (WHO/RHR/10.10) 3. UNFPA. Programme of action. Adopted at the International Conference on Population and Development, Cairo, 5–13 September 1994. New York, United Nations, 1995 (http://www.unfpa.org/public/home/publications/pid/1973, accessed 19 December 2012). 4. Beijing Declaration and platform for action of the Fourth World Conference on Women in 1995. New York United Nations, 1996 (http://www.un.org/womenwatch/daw/beijing/pdf/BDPfA%20E.pdf, accessed 19 December 2012). 5. UNDP. Millennium Development Goals (http://www.undp.org/content/undp/en/home/mdgoverview.html, accessed 19 December 2012). 6. United Nations Secretary-General. Global strategy for women’s and children’s health. New York, United Nations, 2010 (http://www.who.int/pmnch/topics/maternal/201009_globalstrategy_wch/en/index.h tml, accessed 19 December 2012). 7. World Health Organization, Department of Reproductive Health and Research. Reproductive health strategy: to accelerate progress towards the attainment of international development goals and targets. Geneva, World Health Organization, 2004 (WHO/RHR/04.8) (http://whqlibdoc.who.int/hq/2004/WHO_RHR_04.8.pdf, accessed 19 December 2012). 8. Commission on Social Determinants of Health. Report by the Secretariat. Geneva, World Health Organization, 2008 (http://apps.who.int/gb/ebwha/pdf_files/EB124/B124_9-en.pdf, accessed 19 December 2012). 9. World Health Organization, Department of Reproductive Health and Research. Global strategy for the prevention and control of sexually transmitted diseases: 2006–2015. Breaking the chain of transmission. Geneva, World Health Organization, 2007 (http://www.who.int/reproductivehealth/publications/rtis/9789241563475/en/, accessed 19 December 2012). 10. Plan of action on sexual and reproductive health and rights (Maputo Plan of Action). Addis Ababa, The African Union Commission, 2006 (http://www.unfpa.org/africa/newdocs/maputo_eng.pdf, accessed 19 December 2012). HRP External Evaluation 2008−2012 Chapter 1 Page | 10 11. UNDP/UNFPA/WHO/World Bank Special Programme of Research, Development and Research Training in Reproduction. External evaluation 1990–2002. Executive summary. Geneva, World Health Organization, 2003 (http://whqlibdoc.who.int/hq/2003/WHO_RHR_HRP_03.14.pdf, accessed 19 December 2012). 12. UNDP/UNFPA/WHO/World Bank Special Programme of Research, Development and Research Training in Reproduction. External evaluation 2003–2007: executive summary. (http://www.who.int/reproductivehealth/publications/reports/RHR_08_4/en/index.ht ml, accessed 20 December 2012). 13. IEGWB Guidelines for global program reviews (http://www.worldbank.org/ieg/grpp/docs/gprguidelines.pdf, accessed 20 December 2012). 14. OECDE DAC Network on Development Evaluation. Evaluating development co- operation: summary of key norms and standards, 2nd ed. Paris, OECD, 2010 (http://www.oecd.org/development/evaluationofdevelopmentprogrammes/dcdndep/ 41612905.pdf, accessed 19 December 2012). HRP External Evaluation 2008−2012 Chapter 1 Page | 11 Annex 1: Terms of reference for the 2008–2012 external evaluation of HRP Background The United Nations Development Programme (UNDP)/United Nations Population Fund (UNFPA)/World Health Organization (WHO)/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP) was established in 1972, and became a cosponsored United Nations (UN) Special Programme in 1988, with a explicit mandate, as endorsed by the 41st World Health Assembly (1988) in resolution 41.9, for: (1) the continued assessment of existing technologies and the acceleration of the development of new technologies in fertility regulation; (2) the building-up of national self-reliance in research on all aspects of human reproduction in developing countries to meet their specific needs in primary health care; (3) promoting scientific and technical cooperation between developed and developing countries, and between developing countries; (4) coordination of the global research effort in the field of reproductive health; (5) promoting ethical practices in the field of human reproduction research to protect the health and rights of individuals in different social and cultural settings. Over the years, the work of HRP has evolved, in order to best respond to this mandate and adapt to evolving needs, global context and commitments. In order to ensure its effectiveness and efficiency in carrying out this mandate, HRP has been subject to periodic independent external evaluations, commissioned by its Policy and Coordination Committee (PCC). These evaluations are carried out in order to ensure the efficiency and accountability of HRP, as well as to advocate for its work and respond to specific requirements of its donors and cosponsors. HRP external evaluation 1990–2002 (presented in 2003) In 2002, a comprehensive evaluation was conducted, covering the period 1990–2002.This evaluation was considered by PCC at its 16th meeting on 30 June to 1 July 2003. The executive summary is available at http://whqlibdoc.who.int/hq/2003/WHO_RHR_HRP_03.14.pdf (10). The 1990–2002 external evaluation was conducted by Management Sciences for Health (MSH) and the Swiss Centre for International Health (SCIH) of the Swiss Tropical Institute. These organizations, working as a consortium, were selected following an international tender process by the External Evaluation Monitoring Team (EEMT), set up by PCC to select the external evaluators, to provide overall guidance to the external evaluation and, in particular, to ensure that the external evaluation report fully addressed the terms of reference given to the external evaluation team. The 1990–2002 external evaluation was a wide-ranging, comprehensive study designed to address four key issues: (1) the relevance and effectiveness of Programme-supported research in reproductive health; (2) the dissemination, global use and impact of the results of the Programme's reproductive health research; (3) reproductive health research-capacity strengthening by the Programme and the use and impact of the Programme's work at country level; and (4) the Programme's governance process, management, administration and efficiency. Conclusions and recommendations made by the external evaluation team were based on document review, citation analysis of selected publications, seven country visits, and HRP External Evaluation 2008−2012 Chapter 1 Page | 12 input from more than 300 informants, of whom 249 provided detailed information through face-to-face interviews and e-mail questionnaires. Two thematic case-studies (one on emergency contraception and one on mainstreaming gender and women's perspectives) were also performed, which provided further in-depth information on specific aspects of the Programme's work. The external evaluation report provided a strong and favourable endorsement of the direction and management of the Programme. The overall conclusion of the external evaluation, as reported in the evaluation report, was that, during the period 1990–2002: HRP clearly met expectations in terms of its core mission to coordinate, promote, conduct and evaluate international research in reproductive health, and achieved its major objectives. The Programme maintained its position as the global leader in generating research results and establishing the scientific consensus needed to advance sexual and reproductive health policies and practices, especially for developing countries (12). The external evaluation also made numerous recommendations, described in the report, to further enhance the performance of the Programme. The Programme reported to PCC on implementation of these recommendations at its 17th meeting on 30 June to 1 July 2004. HRP external evaluation 2003–2007 (presented in 2008) In 2003, the HRP standing committee and PCC asked for an evaluation focusing specifically on the impact of the Programme on global public goods. Public goods are generally defined as those goods that: produce benefits that are non-rival (many people can consume, use, or enjoy the good at the same time) and non-excludable (it is difficult to prevent people who do not pay for the good from consuming it). If the benefits of a particular good accrue across all or many countries, then this is deemed a global or international public good (13). The International Task Force on Global Public Goods has made the above definition operational as follows: International public goods, global and regional, address issues that: (i) are deemed to be important to the international community, to both developed and developing countries; (ii) typically cannot, or will not, be adequately addressed by individual countries or entities acting alone, and, in such cases (iii) are best addressed collectively on a multilateral basis (13). In terms of its mandate and the nature of its outputs, as well as with respect to its modus operandi, the Programme is, without doubt, a major contributor to global public goods; thus, this was the suggested focus of the evaluation. HRP was established by WHO in 1972 to coordinate, promote, conduct and evaluate international research in human reproduction. The UNDP, the UNFPA and the World Bank joined WHO as cosponsors of HRP in “coordination of the global research effort in the field of reproductive health”. As the main instrument within the UN system for research in human reproduction, HRP brings together health-care providers, policy-makers, scientists, clinicians and consumer and community representatives to identify and address priorities for research aimed at improving sexual and reproductive health. Since 1998, HRP has functioned within the WHO Department of Reproductive Health and Research (RHR). HRP investigates the extent and nature of sexual and reproductive health problems, their determinants and the interventions needed for their alleviation or resolution. Its research agenda addresses all of the main challenges in sexual and reproductive health identified in HRP External Evaluation 2008−2012 Chapter 1 Page | 13 international fora, particularly the International Conference on Population and Development (ICPD) in 1994 and the Fourth World Conference on Women in 1995, and their respective 5- year follow-ups. HRP also carries out activities to strengthen the capabilities of low- and middle-income countries to meet their own research needs, and to enable them to participate in global sexual and reproductive health research. HRP promotes the use of research results in policy-making and planning at national and international levels and contributes to the setting of norms, standards and guidelines – including ethical guidelines – in the field of sexual and reproductive health research. In order to foster the achievement of greater equity and reproductive rights, HRP works to ensure that gender issues, especially the perspectives of women, are reflected in both its research and research capability strengthening activities. The evaluation was conducted by a team of global experts selected for each case-study and focused on five Programme achievements that fulfil the criteria of global public goods and lent themselves to an in-depth analysis of inputs, outputs, outcomes and, where possible, impact on sexual and reproductive health status and contribution to achievement of MDGs, including poverty alleviation. The five technical case-studies were: (1) promoting family planning: long- term safety and effectiveness of copper-releasing intrauterine devices; (2) promoting family planning: improving the quality of care in family planning in China; (3) medical (non-surgical) induced abortion; (4) improving maternal and newborn health; and (5) knowledge synthesis and transfer. In addition, a study of HRP’s governance and management was carried out. The evaluation concluded that: HRP remains a global leader in sexual and reproductive health research and capacity building, with particular relevance to the needs of populations in resource-poor settings. The evidence base resulting from this research has been translated effectively into health policy changes and improved practice standards and has ultimately improved health outcomes. The case-studies indicate that HRP is in a good position to continue advancing global public goods in a cost-effective way (12). The resulting evaluation summary and the six case-studies are available at http://www.who.int/reproductivehealth/publications/reports/RHR_08_4/en/index.html (12). Proposal for HRP external evaluation 2008–2012 (to be presented in June 2013) The 2008–2012 HRP external evaluation will aim to provide information on (1) the relevance and fulfilment of HRP's objectives; (2) its efficiency and effectiveness; (3) its comparative advantage; and (4) the impact and sustainability of its work. In doing so, it will provide information that is credible and will enable the continued incorporation of lessons learnt into the decision-making process of both the Special Programme and its constituents. In view of the positive feedback received from PCC on the process followed in the previous evaluation, it is proposed to follow a similar process in 2012–2013, in order to examine in depth a number of the critical programme areas and outputs that have not been examined recently. Method of work It is envisaged that the external evaluation report would consist of an introductory, overview chapter followed by seven chapters that would provide in-depth studies of the selected topics. The introductory chapter would give the background to the Programme as well as a brief overall assessment of the Programme’s contribution to global public good as it relates to sexual and reproductive health and rights (SRHR). It would also evaluate the Programme’s modus operandi, including its governance and the consultative mechanisms it employs for HRP External Evaluation 2008−2012 Chapter 1 Page | 14 defining and prioritizing its work programmes and for the scientific and ethical review of planned, ongoing and completed activities. Furthermore, it would also describe the overall frame of reference that the Programme uses in pursuing its mission and vision, and thus would address such issues as the UN Secretary-General’s Global strategy for women’s and children's health (5), the Millennium Development Goals (MDGs), including poverty reduction (4), the Global reproductive health strategy (6), and the means by which it collects, manages and applies evidence to improve sexual and reproductive health in countries. The chapter would specifically address the following issues: 1. a brief overall assessment of the Programme's contribution to global public good, as well as its comparative advantage, as it relates to SRHR, in terms of its mandate, modus operandi and output. This assessment will include considerations on the uptake and utilization of HRP guidance, tools and strategic approaches: • does HRP continue to provide leadership on SRHR global health matters and to be important to the international community, to low-, middle- and high-income countries? • does it address key SRHR issues that will not typically be adequately addressed by individual countries or entities acting alone? • does it shape the SRHR-HIV research agenda, reflecting key programmatic and policy questions and needs? • does it set norms and standards, and articulate evidence-based policy and practices at the national and global levels? • does it monitor and assess effectively reproductive health trends, and conduct important evaluation? • does it identify, promote and evaluate innovative strategies and technologies? • does it communicate effectively within and beyond WHO? The in-depth studies will focus on the following areas of major interest in relation to the Programme’s mandate and plan of work: 2. overall efficiency and effectiveness of HRP’s governance, management and administration; 3. evidence generation and synthesis to improve family planning, prevent unsafe abortion and prevent and control sexually transmitted and reproductive tract infections; 4. research-capacity strengthening and network building; 5. strengthening implementation research; 6. the status of and opportunities for strengthening engagement with the private sector and civil society. An executive summary will be prepared in order to share the findings in a concise manner among stakeholders, potential partners and other interested parties. It is anticipated that most of the external evaluation can be conducted through desk review of materials, and consultation, by tele- and videoconferencing and other means, with relevant programme staff and key respondents in other organizations (multilateral and bilateral agencies; professional and other nongovernmental organizations), scientists and programme managers at national level, etc. Nevertheless, for each case-study, funds are included in the HRP External Evaluation 2008−2012 Chapter 1 Page | 15 budget to support one visit (3 days) to the Programme in Geneva and one country visit, if deemed essential. It is proposed that each study would be carried out by an independent expert in the field, who would be nominated by and report back to a subcommittee of PCC, entitled the PCC External Evaluation Committee (PEEC). PEEC is comprised of the PCC chair and/or vice chair, the STAG chair and at least two representatives of the standing committee. The logistics of the evaluation, including travel and other arrangements, would be supported by the secretariat. The evaluation will be carried out in accordance with Evaluating development co-operation: summary of key norms and standards (14), which was issued in 2010 by the Network on Development Evaluation of the Development Assistance Committee of the Organisation for Economic Co-operation and Development. HRP External Evaluation 2008−2012 Chapter 1 Page | 16 Annex 2: the Memorandum of Understanding of HRP HRP External Evaluation 2008−2012 Chapter 1 Page | 17 HRP External Evaluation 2008−2012 Chapter 1 Page | 18 HRP External Evaluation 2008−2012 Chapter 1 Page | 19 HRP External Evaluation 2008−2012 Chapter 1 Page | 20 HRP External Evaluation 2008−2012 Chapter 1 Page | 21 HRP External Evaluation 2008−2012 Chapter 1 Page | 22 Page | 23 Chapter 2 Overall assessment of HRP’s relevance and effectiveness Nicholas Dodd Consultant, Sexual and Reproductive Health Page | 24 Page | 25 Contents 1 Introduction . 29 1.1 A note on language . 29 2 Methods . 29 3 Recent challenges . 30 4 The questionnaire . 31 5 Global public health goods . 33 5.1 Priority setting . 33 5.2 Geographical focus . 37 5.3 Coordination of research . 39 5.4 Selected highlights . 40 6 Comparative advantage . 51 6.1 Respondents’ views on HRP’s comparative advantage . 52 7 Norms and standards . 53 8 Monitoring of global trends in sexual and reproductive health . 54 8.1 Global maternal mortality estimates. . 54 8.2 Global estimates of unsafe abortion . 55 9 The sexual and reproductive health/HIV research agenda . 56 9.1 Selected highlights . 56 10 Efficiency and effectiveness of HRP’s communication . 58 10.1 Bibliometric analysis of HRP’s papers . 60 10.2 HRP’s channels of communication . 61 10.3 Changes in RHR over the last 5 years and how these have affected HRP’s research communication and uptake . 62 References . 66 Page | 26 Abbreviations BTN “Beyond the numbers” CDC Centers for Disease Control and Prevention CHNRI Child Health and Nutrition Research Initiative CIRE Continuous Identification of Research Evidence CSO civil society organization CVC Core Voluntary Contribution DMT Decision-making tool for family planning clients and providers EPC “Effective perinatal care” FGM female genital mutilation GAP Gender and Rights Advisory Panel GBV gender-based violence GCP good clinical practice GRR WHO Gender and Reproductive Rights team GTZ German Development Agency HPV human papillomavirus HRP UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction IPPF International Planned Prenthood Federation MCA Department of Maternal, Newborn, Child and Adolescent Health MDG Millennium Development Goal MEC Medical eligibility criteria for contraceptive use MPH Improving Maternal and Perinatal Health (team) MTCT mother-to-child transmission MTSP medium-term strategic plan NASG non-pneumatic anti-shock grament NGO nongovernmental organization OHCHR Office of the High Commissioner for Human Rights PEEC PCC External Evaluation Committee PCC Policy and Coordination Committee PDRH Programme Development in Reproductive Health PPH postpartum haemorrhage RAP regional advisory panel RHL Reproductive Health Library (WHO) RHR WHO Department of Reproductive Health and Research RP2 Research Project Review Panel Page | 27 RTI reproductive tract infection SPP WHO/UNFPA Strategic Partnership Programme SRH sexual and reproductive health STAG Scientific and Technical Advisory Group STI Sexually transmitted infection TDR Special Programme for Research and Training in Tropical Diseases UK United Kingdom of Great Britain and Northern Ireland UN United Nations UNAIDS Joint United Nations Programme on HIV/AIDS UNDP United Nations Development Programme UNECA United Nations Economic Commission for Africa UNESCO United Nations Education, Scientific and Cultural Organization UNFPA United Nations Population Fund UNHCR United Nations High Commissioner for Refugees UNICEF United Nations Children’s Fund UNIFEM United Nations Development Fund for Women USA United States of America USAID United States Agency for International Development WHO World Health Organization Page | 28 HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 29 1 Introduction This chapter reviews the overall relevance and effectiveness of the United Nations Development Programme (UNDP)/United Nations Population Fund (UNFPA)/United Nations Children’s Fund (UNICEF)/World Health Organization (WHO)/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP); its research; its comparative advantage and the value added by its location within WHO; its work on norms and standards; its monitoring of global trends in sexual and reproductive health (SRH); its work related to the SRH/HIV research agenda; and how it communicates its products to its clients. 1.1 A note on language HRP will also be referred to throughout this document as “the Programme”; those countries that are the main intended beneficiaries for HRP’s work will be collectively referred to as “programme countries”; and the various outputs of the Programme, whether they be physical devices or printed materials, will be collectively referred to as “the products” of the Programme. Finally, the term “sexual and reproductive health” is implicitly assumed to always include a component of sexual and reproductive rights. 2 Methods The process for the evaluation began in Geneva, with preliminary consultations and interviews with senior staff of the Programme, as well as WHO staff in other related departments including: the Family and Community Health Cluster; HIV Department; the Special Programme for Research and Training in Tropical Diseases (TDR); the Global Management Cluster; the Partnership for Maternal, Newborn and Child Health; and, the Alliance for Health Policy and Systems Research. The evaluation team also met with the subcommittee of the Policy and Coordination Committee (PCC) charged with overseeing the evaluation (PCC External Evaluation Committee – PEEC). In addition, an e-mail was sent to all staff of the WHO Department of Reproductive Health and Research (RHR), inviting their input and suggestions, and as a result a number of staff members submitted information on their work and ideas for the evaluation process. This was followed by a review of all relevant documentation produced by the Programme over the past 5 years. For Chapters 2 and 3 of the evaluation – the overall assessment of HRP and its governance, administration and management – a 65-point questionnaire was then developed and sent out in full confidence to over 400 key informants. Informant categories included the following: directors of national SRH programmes; PCC members; PCC observers; HRP technical committee members (the Scientific and Technical Advisory Group (STAG); the Gender and Rights Advisory Panel (GAP); regional advisory panels (RAPs); and the Research Project Review Panel (RP2)); current and former headquarters, and regional and country staff of the cosponsors; organizations with an interest either in carrying out SRH research or in using the results of such research; and individual experts in the field of SRH. Although there was insufficient time or resources for full pretesting of the questionnaire, it was reviewed by a number of experts before being distributed. Subsequently, a much shorter questionnaire on the relevance and the use of HRP’s products in national SRH programmes was sent to all UNFPA country offices. The rationale here was that, of all the cosponsors, UNFPA’s mandate in terms of substance coincides most directly with the mandate of HRP. Thus, UNFPA would be in a good position to identify examples of the utilization of HRP’s products in countries. The results are reported in Sections 5 and 10. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 30 A citation analysis was also commissioned as part of the evaluation and the results of this are also summarized in Section 10.1. As the evaluation progressed, it became apparent that it was not always easy to distinguish between the outputs of HRP and the outputs of the Programme Development in Reproductive Health (PDRH). Publications citing achievements of HRP often appeared to include work carried out by PDRH. Operationally, the relationship between HRP and PDRH is very important; in essence, HRP does the research and normative work, and PDRH takes HRP’s products and promotes and applies them in countries through various mechanisms. However, in terms of monitoring and evaluating the work of these two arms of RHR, the distinction between who does what needs to be made clearer. Recommendation HRP needs to clearly identify in its reporting mechanisms the results it achieves, as distinct from the results achieved by PDRH. 3 Recent challenges Every fund, agency or programme goes through periods of instability and change. The way in which they respond is often a test of their fundamentals. Over the past five years, HRP has faced a number of challenges including: • a substantial delay from January 2009 until December 2010 in the appointment of Dr Paul van Look’s successor as director of HRP; • the WHO hiring freeze at a time when a larger than normal cohort of senior HRP staff were retiring; • introduction of a new process for the allocation of WHO core budget resources; • introduction of the new WHO ORACLE management system and the decentralization of some WHO administrative functions; • the proposal to rationalize and possibly aggregate a number of WHO research programmes; • the WHO decision to move Making Pregnancy Safer, formerly part of RHR, not back into RHR but as part of the Department of Maternal, Neonatal, Child and Adolescent Health; • major changes in the long and well-established HRP process for ethical review; • the increasing complexity of the global health architecture and HRP’s need to both adapt to, and service, parts of this new and changing architecture; • the fall of the US dollar against the Swiss Franc; • continuing pressure for HRP to take on a number of new areas of work or increase its work in certain areas – such as research on gender, sexual and reproductive rights, sexuality, adolescents, SRH in emergency and crisis situations, the cost effectiveness of SRH, and the SRH of sexual minorities and people with disabilities, and implementation research, to name but a few. All of these occurred without a corresponding increase in funds, and amidst a global financial crisis followed by the Euro crisis, which led donor countries, foundations and cosponsors to review and reconsider their budgets and aid priorities; • on top of all this, in 2011 the Programme underwent an internal restructuring to streamline its research development and its research-capacity-strengthening HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 31 processes, and it is once again in the process of handing over its leadership to a new director, who was fortunately appointed, on this occasion, early enough to allow a 3– 4-month period of overlap. Throughout this period, HRP has clearly demonstrated that it is a mature and focused programme. The difficulties it had to bear, it tolerated and adapted to; those that could have changed it for the worse, it resisted; and those that were negotiable, it negotiated in such a way as to maintain its overall purpose and mandate. As will be documented in the following sections of this chapter, the Programme continued to produce many important global public goods in the area of SRH between 2008 and 2012, albeit at a somewhat reduced level due to budgetary constraints. This is largely due to three factors: the dedication and excellence of its staff; the leadership and determination of its directors in making the necessary decisions to ensure that HRP continued to move forward; and its fundamentally sound governance and technical oversight systems, based on its Memorandum of Understanding (see Chapter 1 Annex 2). So, ultimately, because of a robust business model, and an ability to adapt to change, HRP was able to weather the storm and to continue to function very effectively. In the words of one its former directors, “HRP continued to do well, in order to do good”. Continued and increased financial and political support, from donors and cosponsors, will ensure in the future that HRP becomes an even stronger force for change in the world, and an essential component in the assistance countries need in order to attain the Millennium Development Goals (MDGs), and particularly, but not exclusively, the targets for MDGs 4 and 5. 4 The questionnaire In order to gather information and opinions from as wide as possible a variety of persons with knowledge of the Programme, a confidential questionnaire was sent to a group of 416 individuals; 166 persons (39.9%) either fully or partially completed the survey, and a further 10 respondents declined to take the survey for various reasons, but most commonly because of a lack of more recent knowledge of HRP’s work. From here on, this group will be referred to collectively as the “respondents”. The percentages of the different categories of respondents are given in Figure 1. The single largest group of respondents was from the category “member of an HRP technical committee (STAG, GAP, RAP or RP2)”. These numbered 67 and represented a response rate of over 50%. The next largest group was for “current or previous staff member of a cosponsor organization”. There were 20 such responses, representing a response rate of just over 25%. Other respondents were representatives from programme countries (9% of respondents) and donors (6% of respondents). HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 32 Figure 1 Categories of respondents – responses to the question “Please indicate the one category that best describes your relationship to HRP” The length of time respondents had been familiar with the Programme is shown in Figure 2. The majority of respondents, 54%, had been familiar with the Programme for 10 or more years, the majority of these for more than 15 years. A further 22% each made up the categories 0–4 years and 5–9 years. Figure 2 Length of time respondents had been familiar with the work of HRP HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 33 The analysis that follows is based not only on responses to the questionnaire, but also on an extensive review of HRP documentation, interviews with programme staff and additional discussions with other stakeholders and interested parties. 5 Global public health goods Does HRP continue to be a relevant and effective instrument for research in SRH? Does it continue to produce outputs that are consistent with its overall goals and objectives? What has been the outcome/impact of these public goods in programme countries? Global public goods can be defined as those “goods” that are freely available to all, and are non-rival in consumption; that is, consumption by one person does not affect the availability for consumption by others. Global public health goods can be in the form of health knowledge and technologies; health policy and regulatory guidance; and public health systems, including any “good” that makes such systems more effective, more efficient or more accessible. Public health goods created by the Programme include: the results of its research published in peer-reviewed journals; guidelines; policy briefs; programmatic and policy documents; technical guidelines; systematic evidence reviews; global trend analyses in the area of SRH; an electronic journal for the dissemination of evidence and guidance; and new methods of fertility regulation and new technologies in other areas of SRH. For the purpose of this evaluation report, these outputs will from now on be collectively referred to as the “products” of the Programme. An additional global public health good created by the Programme is the many worldwide institutions whose research capacities have been strengthened over the years. This enables these institutions to participate and contribute to the implementation of HRP’s global research agenda, as well as becoming a national public health good, contributing to the implementation of national SRH research agendas. As an example, a number of centres in Africa, including in Zambia and Zimbabwe, are now taking part in the multicentre maternal mortality research project. Some of the more mature centres now play a regional role as a resource centre for the research community in their regions, and as a regional ambassador for HRP, charged with disseminating HRP’s products. 5.1 Priority setting Does HRP continue to use sufficiently robust mechanisms to determine its priorities? Relevance and effectiveness require a sound and systematic approach to priority setting. HRP periodically reviews and assesses its priorities, by engaging groups of experts to assist in identifying, categorizing and ranking research issues. This process, along with the WHO Global reproductive health strategy, adopted at the World Health Assembly in 2004 (1), informs the Programme’s medium-term strategies and biennial budgets and workplans, which are then reviewed by STAG and approved by PCC. The most recent exercise was the development of the 2010–2015 medium-term strategy and involved consultations at both global and departmental levels. At global level, the consultative process involved all regions and identified priorities in the five key thematic areas that form the pillars of the SRH strategy: promoting family planning; improving maternal and perinatal health; preventing unsafe abortion; controlling sexually transmitted and reproductive tract infections; and sexual health, gender, reproductive rights and adolescence. Simultaneously, in collaboration with the Global Forum for Health Research, a consultation was carried out among individuals and organizations, which informed the Medium-Term Strategic Plan (MTSP) 2010–2015. The MTSP was then finalized during two RHR staff retreats. Subsequently, at departmental level, each thematic area of work developed consultative strategies to set more HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 34 operational priorities within the framework set by the Global reproductive health strategy (1) and the MTSP. The Promoting Family Planning team used a process based on the Child Health and Nutrition Research Initiative (CHNRI) priority-setting methodology, in order to identify areas of research for improving the quality of, access to and use of family planning; for new family planning technologies; and for reducing unmet need for family planning. Researchers, representatives of programme countries, technical agencies, donors, service providers and the private sector all contributed to this process. The results were subsequently discussed with stakeholders, who then assigned weighted scores within various areas such as epidemiological research, social science research, health policy and systems research, implementation research and research on contraceptive technology. Occasionally, priorities are triggered by unpredictable events, such as new research findings. A recent example of this was the publication of a study on the relationship between hormonal contraception and HIV transmission, which prompted HRP to convene a technical consultation to review all published research in this area. The meeting resulted in evidence-based guidance on the use of hormonal contraceptives and HIV transmission, as well as recommendations for further research in this area. The Improving Maternal and Perinatal Health (MPH) team organizes a consultation every 4 years, involving major collaborators such as academic institutions, donors, representatives of WHO regional and country offices, and international nongovernmental organizations (NGOs). Participants discuss progress and review a background document, prepared by the MPH team in Geneva, which highlights proposed research initiatives for the future. Space is given for the discussion of additional proposals. These consultations took place in Malaga, Spain in 2008, and in Hua Hin, Thailand in 2012. Maternal and perinatal health normative guideline work is prioritized through a survey of stakeholders, including governments, and WHO and UNFPA regional and country offices. Guideline panels are also used for identifying priority knowledge gaps that require new research. National workshops were conducted in six countries, to determine implementation/scaling-up priorities for evidence-based maternal and perinatal health interventions. The Sexually Transmitted Infection (STI) team developed a log-frame in 2006, through a series of consultations and an STI consultative meeting. This has remained the main instrument that guides the STI workplan. A consultation to review with stakeholders the STI research strategy is planned for early 2013. STI normative work is guided by the STI global strategy (2) and by regional meetings. The Gender and Reproductive Rights team (GRR) receives guidance on priority issues, primarily from GAP, but also from historical reviews and expert meetings, and sometimes from STAG itself. GAP has played a key role in guiding the work of GRR. They call for reviews and prioritization of the work, as was the case with human rights some years ago, and, more recently, GAP specifically requested a review of the work on female genital mutilation (FGM). GAP also asked for prioritization of specific issues, such as gender-biased sex selection and sexual health/sexuality. An expert group meeting subsequently made recommendations on how GRR could best contribute in this area. The Preventing Unsafe Abortion team relies on expert groups with specific expertise on monitoring, guidelines and research. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 35 Proposals for research priorities from each of the five thematic areas are also guided by a log- frame before receiving further guidance from the respective RAPs and STAG. As requested by PCC, these proposals also include three separate levels of priority (V: vital; E: essential; and I: important), based on criteria such as: comparative advantage, public health impact, ongoing implementation and dedicated funding. This enables an even more detailed review according to priorities within each area of work, and is also used to guide the disbursement of programme funds by the secretariat as costs and income fluctuate over time. The priorities of the Programme are also affected by recommendations from its technical oversight mechanisms, STAG and GAP. A review of their recommendations over the last 5 years reveals a more or less continuing pressure on the Programme to do more and add to its agenda, but generally in the absence of either new or additional funds or identification of areas of work that should receive less attention. Only one recommendation in 5 years encouraged the Programme to discontinue work in a specific area. This puts pressure on the Programme to do more with no additional funds, and may thus diffuse rather than focus HRP’s work. Respondents were asked about the influence of programme countries and donors on HRP’s priorities (see Table 1). Table 1 The influence of programme countries on HRP’s priorities Influence Percentage of respondents 1 – very much 7.0 2 44.3 3 28.7 4 12.2 5 – not at all 7.8 Total (n = 115) 100.0 Only 7% of respondents felt that programme countries have a strong voice in determining HRP’s research priorities, and only just over 50% gave a rating of “1” or “2” in answer to this question. When it comes to the influence of donors, 25% of respondents felt that they had too much influence over HRP’s agenda, though 69% felt that the influence of donors was about right. Overall, over 70% of respondents gave a rating of “1” or “2” in answer to the question: “Do the research priorities and programmes supported by HRP address SRH issues most likely to assist programme countries to achieve MDG targets?”. Table 2 shows that less than 20% of respondents considered HRP’s priority-setting mechanisms to be “very strong”, though a further 43% gave this question a rating of “2”. In providing further comments on priority setting, a number of respondents felt that priorities were too donor driven or driven by the interests of individual staff; that STAG should play a stronger role in ranking priorities and carrying out in-depth reviews of specific areas of work; and that HRP’s agenda was still too biomedical and needed to move towards implementation research. Overall, there was an impression that, with the loss of steering committees and specialist panels, the process of priority setting could be strengthened and more focused on a smaller number of critical policy- and programme-relevant questions. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 36 Table 2 Strength of HRP mechanisms for determining its research priorities _Strength Percentage of respondents 1 – very strong 17.9 2 43.4 3 24.5 4 12.3 5 – very weak 1.9 Total (n = 106) 100.0 One question asked of respondents was concerned with missed research opportunities. Almost all respondents replied to this question, indicating a wide variety of subjects, some very specific and some very general. Many informants responded in terms of areas that should receive more emphasis rather than in terms of missed opportunities. Three particular issues were cited more often than any other. These were implementation research, research on adolescents and research on the social determinants of SRH. Respondents also noted research on abortion as a continuing critical area of work for the Programme, and one where its comparative advantage was paramount. Conclusion HRP cannot be expected to research or solve all issues related to SRH. In general, it appears that different thematic areas within the Programme use different methods to identify priorities, and some of these are more rigorous and more inclusive than others. These processes may need to be strengthened to enable the Programme to focus on a smaller number of critical policy- and programme-relevant questions. Respondents gave the general impression that programme countries should have more influence on the setting of HRP’s research priorities and donors a little less, but that, overall, priorities were generally in line with what needed to be done. However, a number noted that it was important for the Programme to keep its ear to the ground and listen and take note of what the real issues were. A number of respondents indicated that the Programme might need to focus more on research that is likely to have an impact in the short term (for example STI prevention, and its work on magnesium sulfate for pre-eclampsia and eclampsia), while maintaining at a more moderate level other areas of work that could have a major impact, but in the much longer term (for example, male contraception). Respondents identified implementation research, research on adolescents and research on the social determinants of SRH as three areas to which the Programme might wish to give greater attention. Recommendations HRP needs to strengthen and take a more uniform approach to its priority-setting process, in order to identify those key research questions and knowledge gaps in SRH that are most likely to have an impact in programme countries. Criteria should include: a priority issue for countries furthest from the MDGs; likely impact; implementability; sustainability; practicality; cost; risk; comparative advantage of HRP; and lead time. In its overall programme of work, HRP should consider giving higher priority to implementation research, research on adolescents and research on the social determinants of SRH. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 37 5.2 Geographical focus Does HRP’s work focus sufficiently on attainment of the MDGs and other global targets, and on the research needs of the least developed countries in overcoming barriers to improving access to SRH information and services? One often-cited barrier to undertaking research in the poorest countries is the lack of both human and institutional resources. However, through its research-capacity-strengthening grants, and its Biostatistics and Data Management unit, HRP has been instrumental in developing methodologies to undertake research in resource-poor settings, supporting the process of design, implementation, monitoring, data management and processing, and publication, while ensuring that good clinical practice (GCP) guidelines, standard operating procedures, and data-quality standards are maintained. Two such recent examples are: • a study on the “Effect of non-pneumatic anti-shock garment (NASG) in the treatment of postpartum hemorrhage”. The study was a cluster randomized trial conducted in 38 rural clinics in Zambia and Zimbabwe, designed to test the effectiveness of the NASG to reduce maternal mortality among women arriving in clinics in hypovolaemic shock due to severe postpartum hemorrhage (PPH). The device acts by channelling blood to the brain, heart and lungs, and has the potential to keep women alive during transportation from lower-level facilities or from the community to referral hospitals; • ”a demonstration project for the implementation of the WHO antenatal care model in Mozambique”. This is a study being conducted in 10 large hospitals across the country that provide antenatal care. It is testing the impact of an intervention to improve the quality of antenatal care by improving the detection, treatment and prevention of major health conditions, such as hypertension, anaemia, HIV/AIDS, malaria, and congenital syphilis, during pregnancy. The first phase of this research identified deficiencies in the supply chain as the main limiting factor for the delivery of antenatal care with an adequate level of quality. An interventions package is now being deployed, which consists of an innovative, sustainable and simple system of commodities, kits and checklists to strengthen the supply chain and empower the nurses that provide antenatal care services. The commonalities between these two projects are that the research questions are highly focused on problems relevant to resource-poor settings, and could not be implemented, or would not provide a relevant answer if undertaken in more developed settings. Robust research designs and methods have been deployed using innovative tools and technologies to assure compliance with GCP, including data quality assurance, patient’s safety and confidentiality, and adequate trial monitoring. The studies were conducted within the health system, and under conditions that reflect the routine delivery of care. In promoting and introducing its products, HRP continues to be open to collaboration with all countries in all regions, on the basis of the past work it has undertaken and future opportunities. In overcoming barriers to improving SRH policies and programmes in countries, the Programme has developed specific methodologies, such as the WHO Strategic Approach, and specific collaborations such as the Strategic Partnership Programme (SPP) with UNFPA. For example, the SPP introduced guidelines that HRP had developed in family planning, maternal and newborn health and STIs/reproductive tract infections (RTIs), using a systematic approach. Within the SPP, a number of countries were defined as “countries of intensive focus”, and these were closely followed up in adaptation and implementation of guidelines after their systematic introduction. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 38 In order to consolidate this process, HRP identified a number of countries for “strategic focus”. The criteria used to select these countries included: high levels of maternal mortality and unmet need for family planning; a broad vision of reproductive health in country programming; commitments made by the government to the United Nations (UN) Global strategy for women's and children's health (3), especially related to family planning; the existence of resources/champions to lead and collaborate joint efforts; countries of intensive focus in the UNFPA/WHO SPP; RHR/HRP's comparative advantage of long-term collaboration; and the potential to scale-up with availability of in-country resources. As a result, six countries from the African Region, and three each from other regions fulfilling the above criteria were selected. These are: • African Region: Benin, the Democratic Republic of Congo, Guinea, Kenya, Nigeria, Zambia; • South-East Asia Region: Bangladesh, Myanmar, Nepal; • Western Pacific Region: Cambodia, the Lao People's Democratic Republic, Viet Nam; • European Region: Republic of Moldova, Tajikistan, Ukraine; • Eastern Mediterranean Region: Afghanistan, Pakistan, Yemen; • Region of the Americas: Peru, the Plurinational State of Bolivia, Guatemala. In comparison, the MDG “countdown to 2015” lists 73 countries with the highest burden of maternal mortality, including the 49 least developed countries, and the H4+1 initiative has identified, as their primary focus, a subgroup of 25 of these countries with the highest rates of maternal mortality. This listing also reflects, to a large degree, those countries with high unmet demand for family planning, poor access to SRH services and high rates of HIV infection. Only 9 of the 21 strategic focus countries of HRP are on the H4+ priority list of 25 countries. However, of the six countries in Africa, five are on the list; of the three in the Eastern Mediterranean Region, two are on the list; and for both the Western Pacific Region and the South-East Asia Region, only one out of the three are on the list. Conclusion It does appear that HRP focuses its work on a set of strategic countries, and has developed methods to undertake research in low-resource settings that ensure quality of research design, implementation and good clinical practice. However, it should align its focus countries more to those targeted by other global initiatives for SRH. Recommendation For HRP to maximize its potential impact, it needs to strengthen its focus on research questions that will benefit the least developed countries and those furthest from the MDG targets, and on undertaking this research wherever possible in these countries. All proposed work should include a clear statement of how it contributes directly or indirectly to the achievement of MDG targets 4, 5 and 6 or any post-2015 global targets. This statement should be used by STAG as a major indicator of the relevance of the proposed research. 1 UNFPA, the United Nations Children’s Fund (UNICEF), WHO, the World Bank and the Joint United Nations Programme on HIV/AIDS (UNAIDS), have joined forces as Health 4+ (H4+) to support countries with the highest rates of maternal and newborn mortality. The H4+ partners support emergency obstetric and neonatal care needs assessments and help to cost national maternal, newborn and child health plans, mobilize resources, increase the number of skilled birth workers, and improve access to reproductive health services. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 39 5.3 Coordination of research Are coordination mechanisms for research both with outside partners and within WHO sufficiently strong? Globally, HRP does not attempt to map the SRH research landscape in any systematic fashion, and neither does any other organization. It is generally agreed that this would be costly and would need to be updated every few years, and that most organizations would not use the information in planning their own research priorities. Its value added would therefore be minimal. The more informal strategy used by the Programme – maintaining a broad overview of the major areas of work being pursued by the global SRH research community through its many formal and informal contacts with them – appears to generally avoid unnecessary and costly duplication of efforts. A good example of this is microbicide research and development, where other organizations are making major investments, and the Programme has decided not to give this priority, instead focusing on the special regulatory and introductory considerations needed for these products. However, in the context of monitoring the implementation of the WHO Global reproductive health strategy (1), which is done by RHR every 2 years, and reported to the World Health Assembly, data are collected to ascertain the extent to which selected research products of HRP are being used or have been integrated in health systems, in countries. Examples include the use of magnesium sulfate, focused antenatal care, emergency contraception, and early detection of cancer of the cervix with acetic acid-aided visual inspection. As can be seen from Table 3, over 80% of respondents gave HRP a rating of “1”, “2” or “3” for the way in which it coordinated its work with research going on in other institutions. Comments by respondents ranged widely, from “excellent” to “a lost opportunity”; there was an overall feeling that more coordination was needed, but that it was now even more difficult because of the greater number of actors in the field and would therefore require more resources. Table 3 Respondents’ views on the effectiveness of HRP in coordinating research with other institutions Effectiveness Percentage of respondents 1 – highly effective 16.1 2 39.3 3 27.7 4 15.2 5 – not at all effective 1.8 Total (n = 112) 100.1a a This percentage over 100% is due to rounding. Within WHO, coordination of research is perhaps most important with TDR and the Department of Maternal, Newborn, Child and Adolescent Health (MCA). It is notable that, towards the end of 2009, there was more formal collaboration among research entities based in WHO, as well as those in Geneva outside WHO. Monthly meetings of these groups shared research in progress, and a key product was the joint publication in PLoS Medicine on defining research to improve health systems, which discussed implementation, health systems and operations research (4). In addition, HRP, TDR and the Alliance for Health Policy and Systems Research have a joint mechanism for coordination of implementation research through the Implementation Research Platform. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 40 TDR is undergoing a major reorientation towards implementation research, and if HRP also decides to move in this direction, then increased and perhaps more formal coordination mechanisms with TDR will be needed. MCA has within its mission statement the following item: “Generate and synthesize evidence and define norms and standards for maternal, newborn, child and adolescent health”. Its work includes basic research, clinical trials and implementation and health-systems research in this area. In HRP, the MPH team has as one of its key objectives: “. developing, assessing and implementing effective interventions” and “… addressing barriers to improving access to quality maternal and perinatal health care …”. Conclusion Given that organizations undertaking research on SRH seem often to have their own agendas, sometimes driven by their donors, a global platform for sharing information may be a sufficient alternative to a more formal attempt at coordination. There would seem to be considerable opportunity for duplication of effort between HRP and MCA, particularly in the areas of maternal and perinatal health and adolescents, which perhaps require stronger efforts in coordinating the planning and implementation of research activities between these two groups. With increasing focus on implementation research, TDR and HRP may need to develop stronger mechanisms for coordination. Recommendations There is a need for a more formal mechanism for coordination of research between HRP and MCA, particularly in the areas on maternal and perinatal research, and research on adolescent SRH; and between HRP and TDR on implementation research. HRP should consider developing an e-platform to enable organizations engaged in research on SRH to share information on their current work and future plans. 5.4 Selected highlights What are some of the major global public goods produced by HRP between 2008 and 2012? What follows is a brief review of HRP’s work in each of its main substantive areas since 2008. (Institution strengthening is not included here, since it is the focus of one of the four individual case-studies undertaken for the evaluation – Chapter 5, Research-capacity strengthening and network building.) In addition, a further case-study examines the Programme’s record on evidence generation and synthesis to improve family planning, prevent unsafe abortion and prevent and control STIs and RTIs (Chapter 4). 5.4.1 Promoting family planning HRP carried out the research and the systematic reviews of clinical and epidemiological evidence needed to develop a guideline entitled Medical eligibility criteria for contraceptive use (MEC) (5). This offers guidance on the safety of using 19 different methods of contraception for women and men with specific characteristics or medical conditions. It is one of HRP’s four cornerstones of family planning guidance, the others being: Selected practice recommendations for contraceptive use (6); Family planning: a global handbook for providers (7); and, a Decision-making tool for family planning clients and providers (DMT) (8). The MEC won the first prize in the Obstetrics and Gynecology category of the 2011 British Medical Association Book Awards. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 41 In order to implement this guidance, the MEC were re-formatted as a simple “wheel” (the MEC wheel) (9), in association with the INFO project at Johns Hopkins University, the Partnership for Communication for Family Health in Jordan, and the University of Ghana Medical School. The MEC wheel converts the recommendations into a single hand-held tool that can be used directly by service providers when counseling clients on choice of contraceptive method. The MEC wheel has already been adapted, translated into 24 languages and printed and distributed in all regions. There is evidence of its use in over 80 countries, and this is clearly a minimum figure, since it is not based on a universal survey. The information comes from WHO sales figures (over 135 000 copies in 38 countries); replies from respondents (28 additional countries), and the UNFPA survey (14 additional countries). The MEC wheel is also available in a computer-based electronic version for training or demonstration, and as a mobile phone app, which allows family planning providers even greater access than through traditional paper or electronic formats. The MEC are subject to continuous updating through a systematic mechanism known as CIRE (Continuous Identification of Research Evidence). A weekly review of the family planning literature, carried out jointly by HRP and the Centers for Disease Control and Prevention (CDC) in Atlanta, Georgia, United States of America (USA) enables early identification of any new evidence or issues that would affect the recommendations. HRP’s work in the area of family planning also includes initiatives in the area of infertility. These range from research to simplify clinic methods for managing infertility in low-income settings, to regular updating of the WHO laboratory manual for the examination and processing of human semen (10). This publication is a laboratory guide used, for example, in the clinical evaluation of infertility. The manual, released in its fifth edition in 2010, draws on the expertise of a wide group of HRP’s collaborators in the field of male reproductive health, and is the document most frequently downloaded from the WHO web site. It is already available in six languages (Chinese, English, German, Italian, Japanese and Turkish), and additional translations are under way. Over 4500 copies of the fifth edition have already been sold, and more than 500 copies distributed free of charge in programme countries. As of the end of 2012, downloads from the HRP web site number 27 500. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 42 Boxes 1 and 2 present utilization headlines for these publications. Box 1. Utilization headlines for MEC, the MEC wheel and the decision-making tool In Mexico, the Ministry of Health disseminated 10 000 copies of the MEC wheel (9) and evaluated its use in clinical settings. Preliminary results from the sample of 161 doctors interviewed found that 80% said they had used the wheel in the last 2 weeks and, among users, 84% found it “easy or very easy” to use, and 91% found it “useful or very useful” in their work. In the Americas, in collaboration with the Centro Latinoamericano de Perinatologia/Salud de la Mujer y Reproductiva, 5000 Spanish language copies of the DMT (8) and Family planning: a global handbook for providers (7) were distributed among 18 countries. As a result, a number of countries, including Cuba, Guatemala and the Plurinational State of Bolivia, either updated existing or implemented new family planning service norms. In Argentina, Brazil, Paraguay, Peru and Venezuela, the MEC (5) and the MEC wheel (9) are being reprinted and used. In Argentina and Venezuela, the governments are underwriting the costs, and in other countries support is being provided by UNFPA. The DMT (8) and Family planning: a global handbook for providers (7) were used as the basic documents in a workshop for 15 English- and Dutch-speaking Caribbean countries. Participants, who came from ministries of health and NGOs providing family planning services, developed draft workplans for the use of these guides in their national programmes. Chile has used HRP products in the development of its national norms on fertility regulation, for the provision and use of contraceptives in the national health system. The MEC (5) have been used to develop national family planning service-delivery standards, and preservice and in- service family planning training for all levels of providers in the United Republic of Tanzania. In the Philippines, the UNFPA country office financed the adaptation, translation and printing of 40 000 copies of the MEC wheel (9). In a number of countries in Asia, notably Bangladesh, China, Indonesia, Mongolia, Myanmar, Solomon Islands, Tonga, Vanuatu and Viet Nam, HRP’s family planning guidance tools have been used to formulate/update reproductive health policies, update standards and national guidelines, improve training curricula, conduct training activities, develop advocacy materials and promote service-delivery improvements. Many of these came about as a result of the WHO/UNFPA SPP, which, until 2008, provided HRP with additional funds for the introduction, adaptation and dissemination of its evidence-based guidelines in countries. The Mongolian version of the DMT has been widely distributed throughout the country and is regularly used in family planning clinics. Myanmar simplified the DMT (8), which is now used by community health workers. In Nepal, the Ministry of Health has expanded the use of the DMT to 40 villages in four districts. The DMT (8) has been translated into Lao and is being used in orientation workshops for service providers. The MEC (5) have been used to revise clinical family planning protocols in the States of Uttar Pradesh and Bihar in India. Sri Lanka has adapted the MEC wheel (9) for use in its national programme. The MEC (5) and the DMT are both used in Iran to improve the quality of family planning information and care. The MEC (5) have been used to develop and/or revise national guidelines in Latvia, Romania, Turkmenistan and Uzbekistan. Armenia, Azerbaijan, Kyrgyzstan, Latvia, Lithuania, the Russian Federation and Serbia, among several other countries in the European Region, have translated and printed the MEC wheel (9), and then used it for improving the quality of family planning services at primary health care. The MEC (5) is used in many high-income countries, such as the United Kingdom of Great Britain and Northern Ireland (UK) and USA, to inform national guidelines and decisions made by drug regulatory authorities. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 43 Box 2. Utilization headlines for other HRP family planning products As a result of HRP’s research, Afghanistan changed its policy to enable trained nurses and midwives to insert intrauterine devices, thus greatly expanding the availability of this method. The Sichuan Family Planning Research Institute in Chengdu, China, which received long-term institutional strengthening grants from HRP, established non-scalpel vasectomy in China and is now recognized as the national reference centre for training and dissemination, and as an international training centre. Using evidence-based family planning guides and tools developed by HRP, a master training of family planning trainers was carried out in Iraq with collaboration from the Ministry of Health in Jordan. In developing national guidelines and standards, Turkey uses the guidance and recommendations from WHO/HRP’s publications on family planning, and, more generally, uses HRP’s products as a reference point on all other SRH interventions. The International Planned Parenthood Federation (IPPF) works with its member affiliates in all countries to promote the adaption and adoption of HRP guidance on family planning, as well as broader guidance on all SRH issues. Member affiliates extend and expand this process by working with the national authorities to promote the use of HRP’s products. A survey of programme countries in 2011 revealed that around two thirds of those responding had included emergency contraception as one of the family planning methods available in public programmes. Over 80% of respondents felt that the MEC (5) had been used to strengthen SRH policies and/or programmes. Conclusion HRP’s MEC (5), the MEC wheel (9) and the DMT (8) are being widely used in many programme countries throughout the world, and are undoubtedly having a positive impact on quality of the SRH of individual women, men and adolescents. Other HRP products are also expanding the availability of family planning technologies. Recommendation To further strengthen its effectiveness, the Programme should examine if and how the CIRE approach could be used for the other thematic areas of HRP’s work. 5.4.2 Maternal and perinatal health In the area of maternal and perinatal health, HRP has addressed some key questions of high relevance between 2008 and 2012, and as a result has produced a number of global public health goods, many of which are already being used in programme countries. Management of the third stage of labour HRP undertook research on the management of the third stage of labour, which demonstrated that in situations where injectable uterotonics (primarily oxytocin) were available, the use of misoprostol did not have any additional benefits in preventing PPH. Additional research in this area showed that controlled cord traction was safe in hospital settings but could be omitted, if injection of oxytocin was available, in situations where skilled birth attendants were not present. After publication of the results in peer-reviewed journals, the WHO guideline on PPH treatment and management has been updated and is in press, and further dissemination of the guidance has started. A meeting cosponsored by the United States Agency for International Development (USAID) and the Maternal and Child Health Integrated Project in Dhaka, Bangladesh, with over 300 participants from various programmatic backgrounds, presented the results and their implications for national programmes. USAID is also issuing a technical note to all its country offices and contract agents, to change their approach on management of the third stage of labour. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 44 The recommendations on misoprostol were subsequently incorporated into the 17th edition of the WHO Model list of essential medicines (11), a reference guide widely used by programme countries in determining national drug formularies. Optimizing maternal and newborn care HRP undertook the research and analysis that enabled the development of evidence-based recommendations to facilitate universal access to key, effective maternal and newborn interventions through optimizing the roles of health-care workers. These recommendations are intended for health policy-makers, managers and other stakeholders at regional, national and international levels and can be adapted to specific national contexts. The guideline: WHO Recommendations for optimizing the delivery of key maternal and newborn health interventions (in press) makes a total of 128 recommendations, 25 of which are for lay health workers, 27 for auxiliary nurses, 22 for auxiliary nurse-midwives, 17 for nurses, 13 for midwives, 8 for associate clinicians, 8 for advanced associate clinicians and 1 for non-specialist doctors. Two launch meetings have already taken place in 2012, the first with WHO Regional Office for Africa, in Addis Ababa, and the second at the International Federation of Gynecology and Obstetrics conference. The Reproductive Health Supplies Coalition is considering the production of a policy brief, and supporting further dissemination. This guideline provides the scientific basis for various task-sharing and task-expansion activities by different categories of health-care workers in different clinical settings. It will ultimately result in greater access to appropriate and safe maternal and newborn care for women. Maternal “near-miss” HRP developed a standardized definition of maternal “near-miss” (12), and created criteria for its identification, in order to improve monitoring of maternal care. This allows assessments of the management and response to complications occurring during pregnancy and the intrapartum period, and thus serves as a barometer of the maternal care system. By standardizing the definition, the robustness of the evidence base is improved, and programme managers are better able to monitor and assess performance in health-care facilities. The Programme also undertook a multicountry survey on maternal and newborn health (13). This study, conducted in over 350 health facilities in 29 countries aims to evaluate the quality of care in health facilities, through examination of the prevalence of maternal “near-miss” cases. The findings of the study will enable a more comprehensive dialogue with policy- makers, programme managers, professional associations and civil society, to promote best practices, improve quality of care and achieve better health for mothers and children. A facility-level monitoring tool with indicators has already been published. HRP carried out the research showing the effectiveness of magnesium sulfate in the treatment of pre-eclampsia and eclampsia, and subsequently ensured that it was included in the WHO Model list of essential medicines (11). Other important evidence-based guidelines on maternal and perinatal health include: WHO Recommendations for induction of labour (14); WHO Recommendations for prevention and treatment of pre-eclampsia and eclampsia (15); and Managing complications in pregnancy and childbirth (16). All of these, as well as all other HRP guidelines and systematic reviews, are available through the Reproductive Health Library (RHL) both online and as a CD-ROM. Finally, HRP has initiated collaboration with the inventor of the Odon Device, a low-cost, easy- to-use technological innovation to facilitate operative vaginal delivery and minimize trauma to HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 45 the mother and baby. The device has great potential for use by mid-level health-care providers in low-resource settings, and is currently in early clinical trials in Argentina and South Africa. Box 3 presents utilization headlines for maternal and perinatal care products. Box 3. Utilization headlines for maternal and perinatal care products In order to have a criterion-based clinical audit method for improving the quality of maternal care, the HRP near-miss monitoring tool is being used in Peru as a national policy for the surveillance of severe maternal morbidity. Using the results of HRP’s research, national-level interventions for the improvement of maternal, newborn and child health were introduced in Brazil, Chile and Mongolia. Maternal “near-miss” has been adopted for use in Uganda. Using evidence-based guidelines developed by HRP, Myanmar has adopted a new national policy on active management of the third stage of labour. Definition of maternal death, and maternal death audits have contributed to improving maternal care in Mozambique, and in training health-care providers in Pakistan. In Mozambique, during the implementation of a cluster randomized trial, it became apparent that shortages and rupture of stocks of commodities prevented implementation of the model, and this led the Ministry of Health to develop an intervention to address commodity supply issues. Fifteen other countries are implementing the antenatal care model developed by HRP, which halves the expense and the time women spend in accessing services, without compromising the quality of care received. A survey of programme countries in 2011 revealed that 95% of those responding had registered magnesium sulfate in their national lists of essential medicines. As can be seen from Table 4, almost 90% of respondents cited the global estimates as having an impact on programmes and policies. For the other three products, over 60% gave a positive response, with a much higher proportion of “Don’t knows”, most likely because these products are relatively new in the HRP “pipeline”. Table 4 Respondents’ views on the use of maternal and perinatal care products to strengthen sexual and reproductive health programmes or policies Product Percentage of responses Yes No Don’t know Total Global maternal mortality estimates (17) 88.9 2.2 8.9 100.0 Maternal near-miss definition/criteria (12) 63.4 3.0 33.6 100.0 Recommendations on maternal and newborn health interventions (in press) 65.4 3.8 30.8 100.0 Multicountry survey on maternal and newborn health (13) 60.7 5.2 34.1 100.0 5.4.3 Preventing unsafe abortion Every year, approximately 22 million unsafe abortions take place globally, and 98% of these occur in low- and middle-income countries. The global rate of unsafe abortion has remained somewhat unchanged since 2003, though the absolute numbers have increased by 2 million between 2003 and 2008. Complications of unsafe abortion are estimated to result annually in 47 000 deaths, and an additional 5 million women suffer from a subsequent disability. Globally, 1 in 8 maternal deaths are attributable to abortion, and in Eastern Africa this rises to 1 in 5. Technical and policy guidance on safe abortion HRP updated and published in 2012, Safe abortion: technical and policy guidance for health systems (18), a document that provides a set of the latest evidence-based recommendations. The document was developed with inputs from various groups of experts as well as HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 46 information from Cochrane systematic reviews. All available evidence was appraised and graded using a standardized approach. The evidence and recommendations then went through a final review at a technical consultation at WHO. The document is an essential tool for support to countries, including strategic assessments on unintended pregnancy and abortion; development and revision of national standards and guidelines for comprehensive abortion care; and technical opinions on restrictive laws and parliamentary amendments. A review and update will take place in 4 years. In parallel, a companion document, Clinical practice handbook for safe abortion care is being developed by the Programme. Mid-level providers for safe abortion Randomized controlled equivalence trials conducted by HRP showed that trained mid-level health-care providers can administer early surgical abortion (South Africa and Viet Nam) and medical abortion (Nepal) as safely and effectively as doctors. These findings have a major impact on preventing unsafe abortion and related morbidity and mortality. In countries where abortion is not against the law, it potentially expands access to safe abortion through mid-level providers, especially in remote areas where doctors are in short supply, and for poor women who are unable to afford doctors’ fees. Box 4 presents utilization headlines on unsafe abortion. Box 4. Utilization headlines on unsafe abortion Between 2008 and 2011, HRP collaborated with eight countries that wished to better address the issue of unintended pregnancy and abortion. Using the “strategic approach”, a strategic planning and policy and programme implementation tool previously developed by HRP, which includes assessment, introductory and scaling-up phases, these eight countries have now made considerable progress towards the goals they themselves set. Four countries in Africa identified similar barriers to safe abortion and pregnancy prevention. These included poor implementation of existing laws, which limited women’s access; lack of knowledge about the legal status of abortion by both clients and providers; costly, clandestine systems for abortion; inadequate post-abortion care; weak family planning information, education and service systems; and religious, social and cultural barriers. A recent follow-up review concluded that the commitment and actions of governments and civil society stakeholders with regard to reducing abortion-related deaths and injuries had significantly increased. In the Republic of Moldova and Ukraine, HRP’s strategic approach enabled: the development of national standards and guidelines on comprehensive abortion care; revision of preservice training curricula based on the national guidelines; and the development of model services. In Kyrgyzstan, the strategic approach led to development and approval of a new national clinical protocol on comprehensive care for unwanted pregnancies; the introduction of a client-oriented approach in clinics; and, the development, testing, approval and implementation of new training curricula. Using HRP guidelines, the former Yugoslav Republic of Macedonia developed national norms and standards for abortion care in response to a previously undertaken strategic assessment exercise. HRP safe abortion guidance has been used to improve the quality of abortion care in Romania and the Russian Federation. In 2012, Bangladesh adapted the HRP document Safe abortion: technical and policy guidance for health systems (18) as the basis for updating their national menstrual regulation services guidelines. The Programme provided technical assistance to Nepal for scaling up the introduction of methods of medical abortion developed by HRP. Based on the original study carried out in Nepal, South Africa and Viet Nam, an ongoing operations research project is assisting 30 countries to expand access to medical abortion in low-resource settings, again using the strategic assessment approach. The Ministry of Health in Colombia used HRP guidance documents to develop national guidelines on abortion. The Ministry of Health in Argentina is training health-care professionals using the project team that undertook the research related to improving post-abortion care, including use of family planning. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 47 A total of 82.2% of respondents felt that safe abortion guidance had strengthened SRH programmes or policies, and two out of every three felt that research on using mid-level providers for carrying out safe abortion had had a similar outcome. Conclusion This work clearly demonstrates how HRP develops and uses products to facilitate outcomes in programme countries, such as improved standards of care, which have an impact on the health and lives of women. This is also a clear example of the comparative advantage of HRP – its ability to work in sensitive areas with the complete confidence of countries. 5.4.4 Prevention and control of sexually transmitted and reproductive tract infections including gynecological cancers and infertility Sexually transmitted and other reproductive tract infections The HRP publication Sexually transmitted and other reproductive tract infections: a guide to essential practice (19) is a reference manual and a resource to educate and remind health-care workers of the need to consider STIs and RTIs when providing other SRH services. It is used widely for preservice and in-service training; as a source of up-to-date, evidence-based recommendations; and as a starting point for improving policies and programmes for the prevention and management of STIs/RTIs. Rapid test for syphilis Syphilis screening in pregnancy is more effective in preventing stillbirths than any other pregnancy intervention besides comprehensive emergency obstetric care, and costs less per pregnant woman than nearly any other intervention. HRP, in collaboration with TDR, developed a point-of-care rapid test for syphilis that can be administered by any trained antenatal care provider. The test provides an immediate result and thus the possibility of immediate treatment. Evidence has shown that introduction of the rapid test can increase the coverage of syphilis screening in antenatal settings. Research has also included field-studies to better understand implementation, acceptability and scaling-up issues. The Programme is currently developing an advocacy tool for investing in this intervention: Investment case for eliminating congenital syphilis: promoting better maternal and child health and stronger health systems. Preventing cervical cancer HRP has assessed the acceptability and feasibility of implementing a “see and treat” approach to cervical cancer, using visual inspection with acetic acid and cryotherapy. With some basic training and the addition of some minimal equipment and supplies, the test can be administered by nurses and midwives and the treatment can be provided at a single visit. These services are now being scaled up in the six countries where the research was originally conducted, and five of the six have already developed national plans. Other guidelines produced include: technical guidelines on Strategies and laboratory methods for strengthening surveillance of sexually transmitted infections (20); and on Use of cryotherapy for cervical intraepithelial neoplasia (21). HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 48 Box 5 presents utilization headlines on STI/RTI products. Box 5. Utilization headlines for STI/RTI products In four Central Asian countries (Kyrgyzstan, Tajikistan, Turkmenistan and Uzbekistan) and in partnership with UNFPA country offices, guidelines developed by HRP were used for the integration of family planning and management of STIs at primary care level. The Programme provided support to Rwanda, the United Republic of Tanzania and Zambia to strengthen their national programmes for cervical cancer prevention, including the introduction of human papillomavirus (HPV) vaccine. Using the results of its research, the Programme has ensured the updating of the WHO essential medicines list (11), to include the relevant antibiotics for resistant STIs. A total of 77.6% of respondents felt that the HRP manual Sexually transmitted infections and other reproductive tract infections: a guide to practice (19) had been used to strengthen SRH programmes or policies in countries. 5.4.5 Sexual health including adolescence, gender and sexual and reproductive rights A standardized methodology for measuring violence against women HRP contributed to the development of a standardized methodology to assess violence against women, particularly intimate partner violence (physical, sexual, emotional and controlling behaviours) and its health consequences, as well as risk and protective factors. Standardizing such measurement has enabled comparative studies across and between countries, and facilitated the use of such data for policy-making and programming. This training programme aims to promote the integration of gender equality and human rights in reproductive health policy, programmes and research. It has been translated into multiple languages and adapted in different ways. Interagency statement on preventing sex selection The Office of the High Commissioner for Human Rights (OHCHR)/UNFPA/UNICEF/UN Women/WHO statement, Preventing gender-biased sex selection (22) highlights the public health and human rights dimensions and implications of the problem and provides recommendations on how best to take effective action. Box 6 presents utilization headlines for HRP gender and adolescent products. Box 6. Utilization headlines for HRP gender and adolescent products The Global strategy to stop health care providers from performing female genital mutilation (23) is credited with strongly influencing a Kenyan Law against FGM, passed in Parliament in October 2011. The Ministry of Health in Senegal used the results of HRP’s research to develop its first action plan for improving adolescent SRH services. In Panama, the national education policy changed as a result of research undertaken by an HRP collaborating centre. Teachers are now trained to discuss issues of sexuality with students, and pregnant schoolgirls are allowed to continue their studies. Research in Shanghai, China found that young people considered a dedicated web site to be the best way of improving their knowledge about SRH. The web site is now part of the “life education” programme for secondary school students. The tool for examining laws, regulations and policies in relation to SRH and human rights (unpublished) and its adaptation for adolescent SRH was applied in Moldova, Sri Lanka and Tajikistan. Gender training carried out in China, Myanmar and Sudan led ministries of health to examine how to integrate gender and human rights issues into reproductive health policies. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 49 A total of 62.1% of respondents felt that the HRP guideline Gender and rights in reproductive health: a training manual for health managers (24) had strengthened SRH programmes or policies in countries. 5.4.6 The UNFPA country office enquiry In order to assess more objectively and quantitatively both the use of the Programme’s products in countries, and its modes of communication, a short questionnaire was sent to all UNFPA country offices. The rationale for choosing UNFPA was that it is the HRP cosponsor, whose substantive mandate corresponds mostly closely with that of the Programme. Thirty- four replies were received by the deadline, a response rate of just over 30%. The first question enquired about the use of the Programme’s products to strengthen SRH policies and/or programmes in their countries. Table 5 indicates the proportion of respondents who gave a positive response for each of the products listed. Table 5 Proportion of UNFPA country offices that confirmed that the product had been used in their country Product Positive responses (%) “Don’t know” (%) MEC (5) 88.2 12.1 The MEC wheel (9) 55.9 33.3 Kesho Bora study results (25) 8.8 60.6 Maternal mortality estimates (17) 82.4 3.0 Maternal near-miss definition (12) 55.9 30.3 Safe abortion guidance (18) 61.8 18.2 Research on mid-level providers for abortion 20.6 36.4 Optimizing delivery of maternal and newborn health interventions (27) 67.6 27.3 Multicountry survey of maternal and newborn health (13) 35.3 51.5 RHL 55.9 27.3 STI guide to essential practice (19) 76.5 15.2 Gender and rights training manual (24) 52.9 27.3 More than two thirds of respondents gave a positive response to four products – MEC (5); maternal mortality estimates (17); Optimizing delivery of maternal and newborn health interventions (28); and the STI guide to essential practice (19). Over 50% responded positively on 9 of the 12 products. For those products with low ratings, “Don’t know” responses were correspondingly high, Kesho Bora (60.6%) (25); research on mid-level providers for abortion (36.4%); and, the multicountry study on maternal and newborn health (51.5%) (13). Respondents elaborated their responses by giving many instances of where these products had been used to inform the development of national policies and strategies; to guide the development of national norms, standards and clinical protocols; to update training curricula; to strengthen training programmes; to improve the quality of service delivery including counselling; and to launch national initiatives such as on reducing maternal mortality. Two particularly detailed responses from Nigeria and Tajikistan are provided in Boxes 7 and 8. The second question enquired about the effectiveness of HRP in providing guidance on norms and standards. Over 80% of respondents gave a rating of “1”, “2” or “3” for the effectiveness of the Programme’s work in this area. The final questions in the UNFPA survey related to modes of communication, and suggestions for their improvement, and these findings are reported in Section 10 of this chapter, on the efficiency and effectiveness of HRP’s communication. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 50 Box 7. UNFPA Nigeria response Nigeria used the MEC (5) in its review of the national family planning protocol, and the MEC wheel (9) is used by family planning providers. HRP’s products in the area of maternal and neonatal health helped focus greater national attention on maternal mortality, and were used in the development of the national maternal, neonatal and child health strategy, 2012–2017 (27). HRP’s documents, including the one on safe abortion (18) have been used in the development of the country’s reproductive health policy, and for the training of mid-level health workers. The national STI training manual was guided by HRP’s guide to essential practice for STIs (19). The gender training manual (24) is used in the training of health-care providers on how to mainstream gender in the provision and management of reproductive health services. The RHL is a continuing source of information for updating training manuals, and was also used as a resource in the development of the national reproductive health policy. Box 8. UNFPA Tajikistan response The UNFPA country office assisted various capacity-building initiatives for the MEC (5), including training of trainers, and subsequent training of service providers. The MEC wheel (9) was printed and disseminated widely throughout the country. Between 2008 and 2012, the UNFPA country office in Tajikistan conducted a number of national capacity-building activities to implement evidence-based strategies to optimize delivery of key maternal and newborn health interventions, including “Beyond the numbers” (BTN), an audit of maternal near-miss cases, and “Effective perinatal care” (EPC). As a result of these interventions, the coverage of maternal and neonatal care was increased, and the quality of service was improved. BTN started in 2008 with a confidential enquiry into maternal deaths and a “near-miss” analysis, and subsequently a strategic partnership was established between WHO, USAID and the German Development Agency (GTZ), to implement the findings. EPC introduced a set of effective perinatal care technologies through updating training curricula, training of trainers workshops, cascade training, and follow-up monitoring and mentoring. As a result, there was a decrease in the number of deliveries with complications; early neonatal mortality and morbidity decreased; skilled birth attendance increased; facility-based deliveries increased; and bleeding, postpartum infection and haemorrhagic shock declined Using HRP guidelines, the UNFPA country office was also able to provide input to the development of the National Strategy on Safe Abortion, recently approved by the government. National standards on safe abortion were developed and are being implemented. Using HRP guidance on STIs, training modules on STIs were updated and used for basic and in-service training. The Gender training manual (24) was used to conduct a number of training sessions for health managers, and subsequently UNFPA supported the establishment of safe rooms in health-care facilities for victims of gender-based violence (GBV). Conclusion This far from comprehensive review of just a sample of HRP’s products revealed evidence of their use in over 130 programme countries. The UNFPA country office enquiry found that four critical products – MEC (5), maternal mortality estimates (17), optimizing delivery of maternal and newborn health interventions (28), and the STI guide to essential practice (19) – had been used in more than two out of three countries. A more comprehensive survey would surely find many additional examples of country use. It would appear that many countries are making very good use of HRP’s products in strengthening their SRH programmes, but that even more could be benefiting. This will require investment in a new communication and uptake strategy for all the Programme’s products, and investment in an introductory strategy to be able to demonstrate in a limited number of countries how some of HRP’s products can be incorporated into SRH programmes (see Section 10). These “success story” examples can then be used to leverage the larger funds of multilateral and bilateral donors, including the cosponsors and foundations, for use in additional countries for similar purposes. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 51 5.4.7 Overall conclusions and recommendations HRP continues to ensure its relevance by being the unique global resource that generates the research findings, synthesizes the evidence and develops the products to support policy formulation and programme strengthening to improve SRH. HRP’s outputs continue to be consistent with its overall goals and objectives. It continues to provide global leadership on sensitive SRH issues, and it continues to generate global public health goods of the highest quality and utility. Recommendation The Programme should commission a periodic review of the utilization of its products in programme countries, and estimates of their potential or actual impact. Such a review should also identify the source of funds, whether from national governments, cosponsors, other international donors, bilateral donors, foundations, NGOs, etc., that were used to support the introduction. Such a review will demonstrate the value of investing in HRP and thus further strengthen its fundraising ability. 6 Comparative advantage Does HRP have a comparative advantage and does it continue to utilize it? WHO provides a forum that is unique in many ways. It is universally owned by countries; it is inclusive and neutral; it has the ability to work on the widest spectrum of health issues however sensitive, and to convene health authorities and experts to deliberate on health topics; and when WHO speaks, its “imprimatur” or seal of approval is universally recognized. HRP benefits from this mantle and adds to it a commitment to the highest quality of research, science and evidence. Some examples of HRP’s comparative advantage include its work on preventing unsafe abortion, and on adolescent SRH, and violence against women. HRP’s work on unsafe abortion includes methods for non-surgical termination of pregnancy; monitoring global trends on unsafe abortion; advocacy and programme research to increase the use of family planning methods in countries where abortion has traditionally been used as a method of regulating fertility; research on mid-level health personnel as providers of safe abortion services; and dosage studies of mifepristone and misoprostol by vaginal or sublingual administration. In the area of adolescent SRH, HRP’s research continues to legitimize work in this area, and to demonstrate the need to deal openly with the issue. Countries that have collaborated with the Programme in the area of adolescent health have strengthened their capacity to confront and address the issues of SRH among young people. In the area of sexuality education, the Programme has supported the development of models and materials for sexuality education, which can be adapted for use in countries. In the area of violence against women, the Programme has been instrumental in developing consensus on a number of issues, for example, the elimination of FGM (29); the medicalization of FGM (30); violence against women (31); and preventing gender-biased sex selection (32). HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 52 6.1 Respondents’ views on HRP’s comparative advantage On a scale of 1 to 5 (1: highly effective to 5: highly ineffective), the following proportions of respondents gave HRP’s comparative advantage characteristics a rating of “1” or “2”: • neutrality – 78.3%; • convening power – 78.2%; • WHO seal of approval – 84.0%; • ability to address any SRH issue however sensitive – 80.8%; • commitment to the highest standards of research – 76.0%. These figures clearly confirm the confidence respondents have in the comparative advantage of HRP. In its decisions on proposed research, STAG is guided by a number of principles, including quality of proposed research, likely impact, and areas of research neglected by other organizations engaged in SRH research. Application of this last principle additionally strengthens the comparative advantage of HRP. Respondents were also asked if other organizations existed that could fulfil a similar function to HRP. Over 87% of respondents indicated that there was no other such organization. Responses to the question concerning HRP’s convening ability are given in Table 6. Table 6 Effectiveness of HRP in convening groups of experts Effectiveness Percentage of respondents 1 – highly effective 45.3 2 33.3 3 14.5 4 4.3 5 – not at all effective 2.6 Total (n = 117) 100.0 As can be seen from Table 6, an overwhelming proportion of respondents (over 78%) gave a rating of “1” or “2” for HRP’s convening ability. Respondents also felt that HRP could be doing a number of things more or better to use its comparative advantage. These included: strengthening the communication and marketing of its products to programme countries, directly and through international and regional partners, to better translate knowledge and evidence into action; carrying out more implementation research; and strengthening collaboration with research institutions. Overall, respondents felt that preventing unsafe abortion was one very specific area where HRP had a major comparative advantage over other organizations. Conclusion The Programme continues to demonstrate its comparative advantage through its ground- breaking work in areas such as unsafe abortion, adolescent SRH and violence against women. It continues to exploit its neutrality, its inclusiveness and its ability to convene the broadest array of interested parties to discuss and provide guidance on sensitive technical and policy issues in the area of SRH. The value of its guidance and other products is maximized by its position within WHO. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 53 7 Norms and standards Does HRP continue to set policy and programme norms and standards? HRP’s work in the area of norms and standards results in various types of publication, including: technical (clinical) guidelines, programmatic and policy documents and policy briefs. These set global standards for policies, programmes and clinical practice in SRH. In the past 5 years the Programme has produced 86 such documents, including 16 clinical guides, 51 programmatic and policy documents and 19 policy briefs (see Box 9). Box 9. Headline examples of the Programme’s work on norms and standards over the last 5 years • The four cornerstones of family planning, including the MEC (5) • WHO/HRP statement on hormonal contraception and the risk of HIV infection (33) • Fifth edition of the WHO laboratory manual for the examination and processing of human semen (10) • WHO recommendations for prevention and treatment of pre-eclampsia and eclampsia (15) • WHO recommendations for induction of labour (14) • A framework for the standardized classification and quantification of causes of maternal mortality – contributing; underlying; and immediate (34) • WHO guidelines for the management of postpartum haemorrhage and retained placenta (35) • Safe abortion: technical and policy guidance for health systems (18) • Sexually transmitted and other reproductive tract infections: a guide to essential practice (19) • Global reference for fetal- and birth-weight percentiles (36) • The OHCHR, UNAIDS, UNDP, United Nations Economic Commission for Africa (UNECA), United Nations Education, Scientific and Cultural Organization (UNESCO), UNFPA, United Nations High Commisisoner for Refugees (UNHCR), UNICEF, United Nations Development Fund for Women (UNIFEM), WHO interagency statement on Eliminating female genital mutilation (29) • The OHCHR, UNFPA, UNICEF, UN Women, and WHO interagency statement on Preventing gender-biased sex selection (22). In addition HRP, in collaboration with its partners, standardizes SRH terminology, contributes to relevant sections of the WHO model list of essential medicines (11), ensures that essential reproductive health commodities are added to the WHO Essential Medicines Prequalification Scheme, provides global reference standards in the area of SRH, updates relevant sections of the International statistical classification of diseases and related health problems (37), specifically Chapters 14–16 on SRH, including developing standard definitions of items such as a skilled birth attendant, and how to interpret this definition in different country settings. For example, HRP’s input to the WHO model list of essential medicines (11) includes the following: • the use of misoprostol was expanded and clarified, particularly its role in the management of incomplete abortion and miscarriage, in the 16th edition; • in the next edition, issued in 2011 (11), the use of misoprostol was further clarified with regard to PPH and induction of labour, again on the basis of research undertaken by the Programme; • also as a result of an HRP review of evidence and advice from a technical consultation, ethinylestradiol was determined not to be an essential medicine and was removed from the model list in the latest (2011) edition (11). HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 54 Respondents’ views of HRP’s work on norms and standards are presented in Table 7. Table 7 Effectivenes of HRP’s work on norms and standards Effectiveness Percentage of respondents 1 – highly effective 33.1 2 42.1 3 14.9 4 5.8 5 – not at all effective 4.1 Total (n = 121) 100.0 As can be seen from Table 7, overall, on a scale of 1 to 5, where 1 is highly effective, more than 75% of respondents gave a rating of “1” or “2” for the Programme’s work on norms and standards. Many respondents felt that the work undertaken by HRP on the development of evidence- based norms and standards was critical, and an area that was uniquely appropriate for WHO. Respondents also felt that the process of developing guidance was occasionally somewhat cumbersome, and needed to be simplified and expedited whenever possible, without losing any of its rigour. A total of 52.9% of respondents felt that HRP should do more work on norms and standards, while a further 35.3% felt that the proportion of this work was “about right”. A total of 87.2% of respondents felt that the maternal mortality estimates had been used to strengthen SRH policies or programmes in countries, and 67.6% responded similarly on HRP’s work on the criteria/definition of maternal death near-miss. Conclusion HRP continues to be the gold standard for developing, monitoring and updating the evidence- based norms and standards required to guide SRH policies, strategies, programmes and clinical practice. Policy statements, programme guides, clinical guidance, and evidence summaries issued by WHO/HRP are key reference materials for governments when developing or revising SRH policies and programmes. As an entity within WHO, the credibility of HRP among Member States is assured, and its materials thus receive far greater attention, and have a larger global health impact than similar issuances by any other institution. 8 Monitoring of global trends in sexual and reproductive health Does HRP continue to monitor important global trends in SRH? A few examples of HRP’s important work in the area of global trends follow. 8.1 Global maternal mortality estimates. HRP, in collaboration with the WHO Health Information and Statistics Department, UNICEF, UNFPA and The World Bank analyses maternal mortality levels on a routine and continuing basis. The updated maternal mortality estimates (17) allow for trend analysis to determine the progress towards attaining MDG 5a. The methodology and process undertaken for these estimates is intended to ensure both international comparability and ownership by countries, and provides a starting point for building capacity in-country on improved systems for data collection and analysis. The estimates also include an analysis of regional trends. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 55 8.2 Global estimates of unsafe abortion In 2011, HRP published the sixth edition of the global estimates of unsafe abortion and associated mortality (38), a document aimed at policy-makers and programme managers, to continue to raise awareness of the human tragedy of unsafe abortion suffered by women, and particularly poor and marginalized women in low- and middle-income countries. The document highlights the urgency of preventing unsafe abortion and provides policy and programme recommendations for its prevention. The report also concluded that in countries where abortion laws were more liberal, rates of abortion were lower. Infertility due to unsafe abortion or maternal sepsis was identified as the fifth ranked disability, based on prevalence, in low- and middle-income countries for women aged 0–59 years. The Programme was also a key partner of the scientific team that developed preterm birth estimates – the team consisted of experts from HRP, the London School of Hygiene and Tropical Medicine and Save the Children. The input from HRP was particularly important in engaging and assisting countries in providing more data, understanding methods and supporting their capacity in improving measurement of preterm births. The Programme also contributed to the development of publications covering global estimates of stillbirths – National, regional, and worldwide estimates of stillbirth rates in 2009 with trends since 1995 (39) as well as estimates of the global burden of STIs in 2010 (40). The July 2010 progress report on the implementation of the WHO reproductive health strategy (41) noted in a survey of countries that: 85% reported integrating the WHO focused antenatal care approach in national reproductive health prorammes; 80% of the countries had registered magnesium sulfate for use in pre-eclampsia; and 50% had included emergency contraception as part of their family planning method mix. The percentage of countries reporting implementing new cervical screening protocols was somewhat lower (n total, 57 countries reported). The report also noted uneven progress in reduction of maternal mortality; that unmet family planning need was still high; that antenatal care and skilled birth attendance had increased in urban areas, with little change in rural areas; and that less than 50% of men aged 15–24 years reported using a condom when engaging in high-risk sexual behavior. HRP, in collaboration with UNFPA, defined a number of indicators on health outcomes and processes at national and subregional levels, as well as indicators to assess integration of HIV/AIDS and SRH programmes and services (42). From Table 8, it can be seen that over 70% of informants gave a rating of “1” or “2” for HRP’s work on monitoring global trends in SRH. Table 8 Effectiveness of HRP’s work in monitoring global trends Effectiveness Percentage of respondents 1 – highly effective 24.0 2 48.8 3 18.2 4 9.1 5 – not at all effective 0.0 Total (n = 121) 100.1a Conclusion HRP continues to play a major role in the monitoring and assessment of global trends in SRH, and this work is instrumental for evidence-based advocacy, the monitoring of progress HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 56 towards the achievement of the MDGs and other global targets and goals, and the initiation of national campaigns to address specific SRH issues. 9 The sexual and reproductive health/HIV research agenda What is HRP’s involvement in the SRH/HIV research agenda? There are clear bidirectional links between SRH and HIV-prevention policies and programmes. These can be used for example to: promote male and female condom use to prevent STIs, including HIV infection, and unintended pregnancy; decrease mother-to-child transmission (MTCT) of HIV; improve access to and uptake of key HIV and SRH services; improve access for people living with HIV to sexual and reproductive health care that protects their rights and respond to their needs; reduce HIV-related stigma and discrimination; increase the response to and prevention of GBV; improve coverage of underserved and marginalized populations; and decrease duplication of efforts, and thus competition for scarce resources. HRP has been actively pursuing collaborative work in this area over the last 5 years, including development of policy and programmatic guidance to address the SRH of people living with HIV; planning a multicountry study on the fertility intentions and unmet need for family planning among women living with HIV; and implementation research on integrated approaches for strengthening SRH and HIV services (see Chapter 6). 9.1 Selected highlights 9.1.1 Rapid assessment tool for sexual and reproductive health and HIV linkages: evidence review and recommendations HRP collaborated with IPPF, UNFPA, WHO, UNAIDS, the Global Network for People Living with HIV/AIDS, the International Community of Women with HIV/AIDS, and Young Positives, to develop a rapid assessment tool for SRH and HIV linkages at policy, system and service levels. The tool has promoted and enabled dialogue and collaboration between the HIV and SRH areas, which are often separated in national programme contexts. The tool has now been used in over 45 countries and the process, results and outcomes have been summarized in individual country briefs, which are then used as the basis for further action. The Kesho Bora Study The Kesho Bora Study (25) was a landmark piece of research led by HRP, using its own funds and leveraging additional funds from partners. It showed that giving mothers a combination of antiretroviral drugs during pregnancy, delivery and breastfeeding cut HIV infection in infants by 42%. The findings of the research increase the range of treatment options available to mothers living with the virus, and offers them hope that, if they so wish, they have a greater chance of breastfeeding their babies without passing on HIV. As a result of the study, Malawi and South Africa have adopted a new regime for preventing MTCT of HIV, and the findings have changed WHO’s recommendations on infant feeding and led to new drug-combination approaches in global efforts to eliminate MTCT of HIV. Collaboration with the Global Fund HRP continues to advocate for and provide assistance to programme countries in preparing proposals to the Global Fund to Fight AIDS, Tuberculosis and Malaria, particularly in the areas of: unmet needs and opportunities for linking SRH and HIV prevention and care; and opportunities to address violence against women and girls. The proportion of proposals with HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 57 an element of SRH has increased from under 40% in the first round to 90% and 65% respectively in the last two rounds. Collaboration with the WHO HIV/AIDS department Current and future work of the WHO HIV/AIDS department includes specific deliverables that require contributions from HRP. These include: a generic protocol for operational research to understand the performance and impact of use of dual point-of-care tests for HIV and syphilis; guidance on the evaluation, specifications and procurement of female condoms; tools for monitoring condom quality during storage; technical guidance on microbicides; clinical guides and tools for family planning for persons living with HIV; clinical guides and tools for counselling on testing for HIV in family planning clinics; policy and programme guidance for prevention of MTCT of HIV; guidance on infant feeding and transmission of HIV; guidelines on the prevention and management of STIs; an updated package for HIV treatment, prevention and care among young people; and evidence, tools and guidelines to address gender-based inequalities in HIV response for most-at-risk populations. Collaboration with UNAIDS Collaboration between HRP and UNAIDS includes areas such as: a discussion paper on male involvement in the prevention of mother-to-child transmission of HIV, which also indentified gaps in knowledge and areas for further research; the technical meeting on hormonal contraception and HIV risk (33), which provided clear guidance on this issue; development of a counselling tool on reproductive choices and family planning for people living with HIV (43); and guidance on issues such as male circumcision and HIV (44). As shown in Table 9, over 85% of respondents gave a rating of “1”, “2” or “3” for HRP’s effectiveness in contributing to the SRH/HIV research agenda. Comments from respondents suggested a need for HRP to be more proactive in this area (for example, as it had been over the issue of HIV infection and hormonal contraception); to forge stronger links with the WHO HIV/AIDS department and with UNAIDS (the latter perhaps now facilitated by UNAIDS becoming a permanent member of PCC); and for a greater focus on barriers and implementation issues relating to the integration of SRH and HIV programmes at country level. There was also some indication that the regional panels are increasingly generating more research on SRH/HIV issues. Table 9 Effectiveness of HRP in contributing towards the SRH/HIV research agenda Effectiveness Percentage of respondents 1 – highly effective 21.6 2 43.1 3 21.6 4 12.1 5 – not at all effective 1.7 Total (n = 116) 100.1a a This percentage over 100% is due to rounding. Conclusion The Programme continues to play an important role in shaping and implementing the SRH/HIV research agenda and should continue to strengthen links with HIV/AIDS research partners. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 58 10 Efficiency and effectiveness of HRP’s communication HRP communicates with its clients in a number of ways and through a number of mechanisms. Most of these involve its publications, which can be classified into nine separate categories as follows: • policy briefs, highlighting the policy implications of new research findings, such as Programmatic and research considerations for hormonal contraception for women at risk of HIV and women living with HIV – policy Implications (45); • monitoring and evaluation reports, such as Trends in maternal mortality: 1990 to 2010, WHO, UNICEF, UNFPA and The World Bank estimates (17); • advocacy documents, such as Providing the foundation for sexual and reproductive health. A record of achievement (46); • peer-reviewed articles, including systematic evidence reviews; • fact sheets, statements and information notes, such as Emergence of multi-drug resistant Neisseria gonorrhoeae – threat of a global rise in untreatable sexually transmitted infections (47); • newsletters, such as the Reproductive Health Update; • multimedia (videos, exhibits, posters, apps), such as the RHL DVD2, and HRP videos on YouTube; • clinical guidelines, such as: WHO recommendations for prevention and treatment of pre-eclampsia and eclampsia (15); • programme and policy documents, such as Beginning with the end in mind: planning pilot projects and other programmatic research for scaling up (48). For the main six categories, the Programme has produced 477 such documents over the past 5 years, of which further details are given in Table 10. By far the largest output of HRP is its peer-reviewed articles, and these increased from 52 in 2008 to 127 in 2011 and 80 in 2012. Respondents cited peer-reviewed publications and clinical guidelines as the most effective channels for communicating HRP’s results. Table 10 Numbers of communication documents produced by HRP, 2008 to 2012 Document type Number produced 2008 – 2012 Clinical guidelines 16 Programme and policy documents 51 Policy briefs 19 Monitoring and evaluation reports 19 Advocacy documents 15 Peer-reviewed articles 361 Total 481 2 The Reproductive Health Library is an electronic journal, cumulative over time, which publishes evidence syntheses on a variety of key SRH topics and includes comments on the application of the evidence to low-resource settings by experts from programme countries. It is also a conduit for dissemination of all the guidelines, norms and standards developed by RHR/HRP. The RHL CD- ROM was published in April 2011 with 193 Cochrane reviews. Since then, the online version has been expanded with two to three new and updated titles per month. RHL continued to rise in rank within the family of WHO web sites: of the 211 WHO unique web addresses, RHL moved from rank 15th in terms of number of sessions per month in 2010 to 7th in November 2011. At the end of 2012 it ranked 11th, but remained the second most visited site outside of WHO corporate pages. There are approximately 17 000 visits per month and over a million page views per month. The most downloaded document in 2011 was the SUPPORT project summary on the effectiveness of task-shifting programmes on maternal and child health outcomes, with 7800 downloads. The RHL Facebook social networking site was launched in late 2011 to increase exposure to a younger generation of SRH workers and is visited more than 50 000 times each month. Additionally, a YouTube channel was created in April 2012 and has received in excess of 65 000 video views per month since its launch. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 59 A somewhat subjective review of the titles of the peer-reviewed publications would appear to indicate that these are generated by both the Programme’s global research agenda and the support it provides to research-capacity strengthening. Some research papers are highly country specific, and others address issues that are clearly of more interest to the individual researchers than to the global agenda. The undertaking of research is an essential element of research-capacity strengthening, but such research cannot always be expected to reflect global research needs. Recommendation In future reporting, HRP should distinguish between peer-reviewed articles generated through its global agenda, and those generated from research-capacity-strengthening activities. This would provide more transparency and permit a greater understanding of the impact of the Programme’s work at both global and regional levels. HRP continues to undertake high-impact randomized trials, the type of research that answers key questions and provides the core evidence for clinical practice guidelines. This research is vital in determining the standards for the practice of evidence-based medicine and public health. A few examples of such high-impact work undertaken by the Programme since 2008 are given in Box 10. All of these led to new recommendations for clinical practice, which are being adopted by countries. Box 10. Headline examples of HRP’s high-impact research • Kesho Bora Study Group, de Vincenzi I. Triple antiretroviral compared with zidovudine and single-dose nevirapine prophylaxis during pregnancy and breastfeeding for prevention of mother-to-child transmission of HIV-1 (Kesho Bora study): a randomised controlled trial. Lancet Infectious Diseases, 2011, 11(3):171–180. Erratum in: Lancet Infectious Diseases, 2011, 11(3):159. • Meirik O et al; WHO Research Group on Postovulatory Methods of Fertility Regulation. Complications of first- trimester abortion by vacuum aspiration after cervical preparation with and without misoprostol: a multicentre randomised trial. The Lancet, 2012, 379(9828):1817–1824. • Althabe F et al; Guidelines Trial Group. A behavioral intervention to improve obstetrical care. New England Journal of Medicine, 2008, 358(18):1929–1940. • Gülmezoglu AM et al. Active management of the third stage of labour with and without controlled cord traction: a randomised, controlled, non-inferiority trial. The Lancet, 379(9827):1721–1727. • Warriner IK et al. Can midlevel health-care providers administer early medical abortion as safely and effectively as doctors? A randomised controlled equivalence trial in Nepal. The Lancet, 2011, 377(9772):1155–1561. • Widmer M et al. Misoprostol as an adjunct to standard uterotonics for treatment of post-partum haemorrhage: a multicentre, double-blind randomised trial. The Lancet, 2010, 375(9728):1808–1813. • Meirik O et al; IUD Research Group at the UNDP/UNFPA/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction. The frameless copper IUD (GyneFix) and the TCu380A IUD: results of an 8-year multicenter randomized comparative trial. Contraception, 2009, 80(2):133–141. • von Hertzen H et al; WHO Research Group on Post-ovulatory Methods of Fertility Regulation. Two mifepristone doses and two intervals of misoprostol administration for termination of early pregnancy: a randomised factorial controlled equivalence trial. British Journal of Obstetrics and Gynaecology, 2009, 116(3):381–389. • von Hertzen H et al; WHO Research Group on Post-ovulatory Methods of Fertility Regulation. Comparison of vaginal and sublingual misoprostol for second trimester abortion: randomized controlled equivalence trial. Human Reproduction, 2009, 24(1):106–112. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 60 10.1 Bibliometric analysis of HRP’s papers To provide more evidence of the quality and impact of the HRP’s work, the evaluation team requested the Programme to commission a bibliometric analysis of its peer-reviewed publications. The analysis was contracted out to Thomson Reuters in the UK and the report is briefly summarized in this section. Chapter 8 provides more information about the analysis. Citations to prior work are a normal part of publication, and reflect the value placed on the work by later researchers. Highly cited work is recognized as having a greater impact and being highly correlated with other qualitative evaluations of research performance. Citation analysis has to be used with some caution; it is more rigorous for science and technology and less so for social sciences and the humanities. Citations generally increase over time, so very recent publications will generally have fewer citations. Normalization is usually done by reference to relevant global averages for the particular field. The Thomson Reuters analysis was able to identify and match 1842 HRP publications in its database, of which the majority (89.6%) were peer-reviewed articles and reviews. The most frequent type of journal in which HRP research is published is journals focusing on contraception, obstetrics and gynaecology, and andrology. Obstetrics and gynaecology accounts for the highest share, at 40.9% of all papers published, and the citation impact for these articles is twice the world average. HRP articles published in journals dealing with general and internal medicine (such as The Lancet and The New England Journal of Medicine) and oncology had a citation impact of between two and three times the world average. Overall, the normalized citation impact of HRP publications has risen from an already impressive level of 1.42 during the period 1990 to 2007, to 2.14 between 2008 and 2011, indicating a significant improvement in the impact of the Programme’s research. The proportion of papers that are highly cited (those papers that belong to the world’s top 10% of the most cited papers relative to the journal category and year) also increased between these two periods, from 16.5% to 18.3%. Sixteen of the 20 most commonly used journals for HRP publications are in the top quartile, according to journal impact factor. Acceptance by such journals is an indicator that the publication is more likely to have a greater impact. This provides further evidence of the impact of the Programme. All these indicators of research performance are significantly above world averages and clearly reflect high-quality research, and research that is well regarded among the international research community. In terms of collaborations between countries, the study calculated collaboration indices at two different thresholds: at least one paper per year on average over the period, and at least one paper every two years on average over the period. For the higher-threshold index, there were no collaborations with low-income countries during the period 1990–2007, but for the period 2008–2011, five countries (Bangladesh, Burkina Faso, Nepal, Uganda, and Viet Nam) all published at least four collaborative papers with the Programme. For the lower-threshold index, there was an increased level of collaboration with some middle-income countries in Asia and Latin America over the two periods, but again the major change was the increased involvement of low-income countries, from one in the earlier period (Kenya) to eight in the period 2008–2011 (Bangladesh, Burkina Faso, Cambodia, Ethiopia, Kenya, Nepal, Uganda and Viet Nam). This analysis provides clear evidence of the increased involvement, particularly of low-income countries, in the Programme’s work. The Thomson Reuters analysis also carried out an author analysis. This indicated that over the period 1990–2011, the proportion of papers where all authors were from programme HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 61 countries fell from 23.2% to 13.7%, while at the same time, the proportion of papers where any author was from a programme country increased from 43.4% to 63.9%. A possible explanation for this is that as institutions matured over the years with the support provided by HRP for research-capacity strengthening, they have increasingly been part of the international collaborative research community, taking part in HRP’s global research agenda as well as the international research agendas of other organizations. Finally, since 2008, data have been collected on the countries of residence of first authors, and the analysis showed that during the period 2008–2011, 40% of all papers had a first author from a programme country. This is below the 2011 level for TDR, which currently stands at 61%, and reflects the need for the Programme to strengthen its efforts to involve institutions in programme countries. 10.2 HRP’s channels of communication The questionnaire asked for opinions about the effectiveness of various groups in communicating HRP’s products, using a scale of 1–5 (1: highly active, 5: minimally active). The majority of respondents gave negative ratings. The figures in parentheses indicate the proportion of respondents who gave a rating of “3” or lower in relation to the level of activity in disseminating the products of the Programme: WHO country offices (72.2%); WHO regional offices (66.3%); WHO headquarters (59.1%); regional advisory panels (52.6%); cosponsors (55.2%); and donors (58.2%). All these figures indicate considerable room for improvement on the part of cosponsors and donors in helping to communicate the publications of HRP. Many respondents felt that HRP needed to strengthen the advocacy, communication and dissemination of its products, in particular by developing a strategy that specifically targets end users in programme countries and involves the regional and country offices of WHO, as well as the other cosponsors, donors and civil society organizations (CSOs), amongst others. There was also an indication that the strategy should be linked to a stronger focus in the Programme on programmatic, implementation and social science research to better understand the introductory process and its barriers. Many respondents suggested that cosponsorship should become much more of a “two-way street”. As well as providing HRP with political and financial support, the cosponsors needed to become “champions” for the Programme’s products, playing a stronger role in disseminating the products and integrating them into their own development assistance programmes. Findings from the UNFPA survey, which also sought views on the effectiveness of different communication documents, were somewhat similar. As can be seen from Table 11, clinical guidelines, programme and policy documents, fact sheets and statements, advocacy documents, and policy briefs were the materials UNFPA offices felt were most effective in contributing to the improvement of national programmes. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 62 Table 11 Effectiveness of different types of communication documents Document type Percentage of UNFPA responses giving an effectiveness rating of “1”, “2” or “3” (1: highly effective; 5: not at all effective) Peer-reviewed articles 54.4 Advocacy documents 69.6 Clinical guidelines 75.8 Programme and policy documents 75.7 Training materials 63.6 Newsletters 45.4 Policy briefs 66.7 Global estimates 51.5 Fact sheets and statements 72.8 Conference presentations 54.5 RHL, CD-ROM 57.6 Press releases 57.6 In elaborating on their replies, there were many suggestions to improve communication between HRP and UNFPA country offices and national counterparts, the most common being the need to develop and keep up-to-date lists of key stakeholders at country level, such as national SRH managers, national NGOs, universities, training schools, CSOs and professional associations, and ensure that these groups were systematically kept informed of all new developments and products. Other suggestions included: greater use of social media channels; regional workshops for introductory activities; and web site linking. But the overall clear message was the need for HRP to strengthen its communication strategy to guide the dissemination of its products at country level, to better ensure their use. 10.3 Changes in RHR over the last 5 years and how these have affected HRP’s research communication and uptake HRP creates many valuable global SRH goods, but the ultimate value of these goods is in their utilization to improve the quality of SRH policies and programmes in countries, so that individuals can benefit. There is a continuum from research to action, of which research communication and uptake is an integral and essential component. Up until 2007, HRP had two direct mechanisms for promoting the use of its products in countries. These were: a well-funded Department for Programme Development in Reproductive Health (PDRH), and a well-funded Strategic Partnership Programme (SPP) with UNFPA, in the amount of over US$ 6 million between 2003 and 2007. An external evaluation of the SPP in 2007 (UNFPA unpublished document) found that it had used, and promoted with national authorities, a systematic process for the adaptation, adoption and introduction of evidence-based guidelines in national programmes. The SPP always involved governments, mainly ministries of health, from the outset (thus ensuring national ownership); translated evidence into practice; validated a model for use with other guidelines; and validated a model that could be drawn upon by other organizations for the introduction of their guidelines. With the end of the SPP in 2008, and the funding constraints in PDRH that followed from 2008 to 2011, brought about by changes in WHO core funding policies, the introduction of HRP’s products in countries waned, and the potential impact of its products was compromised. HRP started to lose a substantial amount of its ability to ensure that its products were translated into use at country level. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 63 Trends in income for PDRH are presented in Figure 3. PDRH income rose rapidly from US$ 9.2 million in 2000–2001 to US$ 25.0 million in 2006–2007, an increase of over 170%. During this period, its income consistently exceeded its full budget level. Since 2006–2007, PDRH income has fallen by around 16% to levels of US$ 20.4 million and US$ 21.0 million respectively in the last two biennia. In 2008–2009, income did not reach the contingency budget. In 2010–2011, income was between contingency and full budget levels. Figure 3 Trends in PDRH income Between 2006–2007 and 2010–2011, national governments decreased their contributions by US$ 6.6 million (38%), and UN system agencies, programmes and funds decreased their contributions by US$ 3.6 million (65%). Fortunately, foundations increased their contributions fourfold over the same period, to a level of US$ 8.2 million, to bring some stability to the PDRH funding situation. Major bilateral donors in 2006–2007, such as Denmark, Finland, the Netherlands, Norway, Sweden and the UK, combined to provide US$ 12.0 million of funding to PDRH. Subsequently, as requested by WHO, they included this funding in their core support to the WHO budget. However, they have not seen their previous PDRH contributions reflected in increased WHO Core Voluntary Contribution (CVC) support to PDRH. WHO CVC support to PDRH amounted to US$ 2.5 million in 2008–2009, and US$ 3.5 million in 2010–2011. This represents “lost” funding to PDRH of US$ 9.5 million and US$ 8.5 million respectively in the last two biennia. PDRH’s income declined by 16%; had previous funding mechanisms been maintained, it might have increased over the same period by more than 20%, to a level of around US$ 30 million. Fluctuations in income to PDRH from different donor groups present an additional challenge to the work and management of PDRH, since a large proportion of the new funding in the last 5 years is specified rather than flexible. As can be seen from Table 12, flexible funding to PDRH has fallen dramatically in the last three biennia, from 56.0% of total income to 26.3% of total income. Specified funds now make up almost three quarters of income to PDRH. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 64 Table 12 PDRH flexible and specified income, 2006–2011 Biennium Flexible income Specified income Total US$ million % US$ million % US$ million % 2006–2007 14.0 56.0 11.0 44.0 25.0 100.0 2008–2009 6.5 31.7 14.0 68.3 20.5 100.0 2010–2011 5.5 26.3 15.4 73.7 20.9 100.0 One of the major consequences of this trend is that, over the past 6 years, flexible income has increasingly been used to pay for staff salaries. In 2008, PDRH had 35 posts. In 2012, it was reduced to 22 posts. Since 2010, all flexible PDRH funding has been used to pay staff salaries. With no flexible funds for activities, these staff have continued to implement projects funded by individual donors, rather than doing the core work of RHR – promoting and introducing HRP’s products in countries. A further consequence of the loss of core PDRH funding is that, while up until 2008, the costs of STAG and GAP were shared between HRP and PDRH, since 2009 this has not been possible, and the full cost has been underwritten by HRP. As can be seen in Table 13, the vast majority of respondents gave a very strong indication of the negative impact of the reduction in PDRH funding on the promotion and utilization of HRP’s products. More than 70% gave a rating of “4” or “5”. Table 13 Respondents’ views on how the reduced funding to PDRH has affected the promotion and utilization of HRP’s products Rating Percentage of respondents 1 – not at all 3.6 2 8.3 3 17.9 4 36.9 5 – very negatively 33.3 Total (n = 84) 100.0 Conclusion HRP continues to carry out the research and systematic reviews that enable the production of evidence-based policies, programme interventions and clinical guidelines for SRH. Objective indicators of HRP’s research performance are significantly above world averages and clearly reflect high-quality research, and research that is well regarded among the international research community. However, the Programme needs to do more to increase the direct and lead involvement of programme countries in its work. The Programme also needs to be a global leader in efforts to develop and evaluate more effective ways of communicating and introducing the knowledge it develops into SRH policies and programmes. However, with the end of the SPP and the severe cutback in funding to PDRH, the Programme has lost a substantial portion of its ability to ensure the communication and uptake of its products. Over the past few years, some ad hoc solutions have been used to address this issue, but none of them were universally acceptable or viable in the longer term. It is thus a matter of urgency that donors and cosponsors find a more permanent solution. The responsibilities of HRP are: first to do the research, review the evidence and prepare the products; second, to have an effective communication strategy that ensures that all its products are communicated to all those who need to be aware of them; and third, to have an uptake strategy that enables the Programme to demonstrate how to introduce some of its more critical products into a limited number of programme countries, and to assess their HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 65 potential or actual impact on SRH. In order to be accountable for this work, HRP needs sufficient funds in its budget to implement all these activities. This could be achieved in a number of ways. Donors could return to earmarking contributions to PDRH, thus restoring its “lost” funding, or they could request the Programme to use, say, 20% of the funds for research on “research communication and uptake”. HRP has a long- established practice of spending one dollar on strengthening research capacity for every two dollars spent on research; it may now need to review these proportions and include a percentage of the budget for “research communication and uptake”. The larger task of introduction and uptake falls to national governments, the cosponsors, donors, foundations, CSOs and other. All these groups have internal responsibilities to use the uptake “success stories” demonstrated by HRP, and to channel their funds for the same purpose in the countries where they work, to ensure wider and greater impact of the Programme’s products, as well as greater value of their own investments. Such a process would ensure that the value of HRP’s products is maximized. It would also enable the Programme to more clearly demonstrate the relationship between the funds invested in an area of the work and the results of that investment at country level. It would also be possible eventually to track the cascading effect of the Programme’s work as its products are picked up and used by cosponsors, bilateral donors and others, in their programmes of development assistance. Recommendations There is a need for HRP to develop and invest in a new communication strategy, which explores innovative ways of packaging and disseminating HRP’s research findings and other products for use in strengthening national SRH policies and programmes. The strategy should consider the role of knowledge intermediaries and gatekeepers of change, and that different products will require very different approaches. Subsequent communication workplans should identify clear deliverables and associated indicators. HRP needs to develop, invest in, and implement a strategy for the utilization of its key products into a limited number of countries, to demonstrate their potential or actual impact, and to thereby leverage and guide the use of the funds of national governments, cosponsors, bilateral agencies, CSOs, foundations and others in their support to national SRH programmes. The PCC will need to provide guidance on the source of funding for HRP’s communication and utilization work. HRP donors and cosponsors need to review and strengthen their systems and processes for utilizing HRP’s products in their own programmes of development assistance, in order to maximize the value of HRP’s global goods. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 66 References 1. World Health Organization, Department of Reproductive Health and Research. Reproductive health strategy: to accelerate progress towards the attainment of international development goals and targets. Geneva, World Health Organization, 2004 (WHO/RHR/04.8) (http://whqlibdoc.who.int/hq/2004/WHO_RHR_04.8.pdf, accessed 19 December 2012). 2. Global strategy for the prevention and control of sexually transmitted infections: 2006– 2015. Breaking the chain of transmission. Geneva, World Health Organization, 2007. 3. United Nations Secretary-General. Global strategy for women’s and children’s health. New York, United Nations, 2010 (http://www.who.int/pmnch/topics/maternal/201009_globalstrategy_wch/en/index.h tml, accessed 19 December 2012). 4. Remme JHF et al. Defining research to improve health systems. Plos Medicine, 2010, 7(11):e1001000. 5. Department of Reproductive Health Research, World Health Organization. Medical eligibility criteria for contraceptive use, 4th ed. Geneva, World Health Organization, 2009. 6. World Health Organization, Department of Reproductive Health and Research. Selected practice recommendations for contraceptive use, 2nd ed. Geneva, World Health Organization, 2004. 7. Johns Hopkins Bloomberg School of Public Health/Center for Communication Programs and World Health Organization. Family planning: a global handbook for providers. Geneva, World Health Organization, 2011. 8. World Health Organization, Department of Reproductive Health and Research. Decision-making tool for family planning clients and providers. A resource for high- quality counselling. Geneva, World Health Organization, 2005. 9. World Health Organization, Department of Reproductive Health and Research. Medical eligibility criteria wheel for contraceptive use. 2008 update. Geneva, World Health Organization, 2009. 10. World Health Organization, Department of Reproductive Health and Research. WHO laboratory manual for the examination and processing of human semen, 5th ed. Geneva, World Health Organization, 2010. 11. WHO model list of essential medicines, 17th ed. Geneva, World Health Organization, 2011. 12. World Health Organization, Department of Reproductive Health and Research. Evaluating the quality of care for severe pregnancy complications: the WHO near-miss approach for maternal health. Geneva, World Health Organization, 2011. 13. WHO multicountry survey of maternal and newborn health 2010–2012. Geneva, World Health Organization. 14. World Health Organization, Department of Reproductive Health and Research. WHO recommendations for induction of labour. Geneva, World Health Organization, 2011. 15. WHO recommendations for prevention and treatment of pre-eclampsia and eclampsia. Geneva, World Health Organization, 2011. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 67 16. World Health Organization, Department of Reproductive Health and Research. Managing complications in pregnancy and childbirth. A guide for midwives and doctors. Geneva, World Health Organization, 2007. 17. WHO, UNICEF, UNFPA and the World Bank. Trends in maternal mortality: 1990 to 2010. WHO, UNICEF, UNFPA and the World Bank estimates. Geneva, World Health Organization, 2012. 18. World Health Organization, Department of Reproductive Health and Research. Safe abortion: technical and policy guidance for health systems, 2nd ed. Geneva, World Health Organization, 2012. 19. World Health Organization, Department of Reproductive Health and Research. Sexually transmitted and other reproductive tract infections: a guide to essential practice. Geneva, World Health Organization, 2005. 20. World Health Organization, Department of Reproductive Health and Research. Strategies and laboratory methods for strengthening surveillance of sexually transmitted infections. Geneva, World Health Organization, 2012. 21. World Health Organization, Department of Reproductive Health and Research. Use of cryotherapy for cervical intraepithelial neoplasia. Geneva, World Health Organization, 2011. 22. World Health Organization, Department of Reproductive Health and Research. Preventing gender-biased sex selection. An interagency statement, OHCHR, UNFPA, UNICEF, UN Women and WHO. Geneva, World Health Organization, 2011. 23. UNAIDS, UNDP, UNFPA, UNHCR, UNICEF, UNIFEM, WHO, FIGO, IOM, MWIA, WCPT, WMA. Global strategy to stop health care providers from performing female genital mutilation. Geneva, World Health Organization, 2010 (WHO/RHR/10.9). 24. World Health Organization, Department of Reproductive Health and Research. Transforming health systems: gender and rights in reproductive health. A training manual for health managers. Geneva, World Health Organization, 2001 (WHO/RHR/01.29) 25. Kesho Bora Study Group, de Vincenzi I. Triple antiretroviral compared with zidovudine and single-dose nevirapine prophylaxis during pregnancy and breastfeeding for prevention of mother-to-child transmission of HIV-1 (Kesho Bora study): a randomised controlled trial. Lancet Infectious Diseases, 2011, 11(3):171–180. Erratum in: Lancet Infectious Diseases, 2011, 11(3):159. 26. UNDP/UNFPA/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP). Mid-level health-care providers are a safe alternative to doctors for first-trimester abortions in developing countries. Geneva, World Health Organization, 2008 (WHO/RHR/HRP/08.15). 27. Federal Ministry of Health. Saving newborn lives in Nigeria: newborn health in the context of the Integrated Maternal, Newborn and Child Health strategy, 2nd ed. Abuja, Federal Ministry of Health, Save the Children, Jhpiego, 2011. 28. Optimizing health worker roles to improve access to key maternal and newborn health interventions through task shifting. Geneva, World Health Organization, 2012. 29. OHCHR, UNAIDS, UNDP, UNECA, UNESCO, UNFPA, UNHCR, UNICEF, UNIFEM, WHO. Eliminating female genital mutilation. An interagency statement. Geneva, World Health Organization, 2008. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 68 30. Global strategy to stop health-care providers from performing FGM. Geneva, World Health Organization, 2010 (WHO/RHR/10.9). 31. World Health Organization and London School of Tropical Medicine. Preventing intimate partner and sexual violence against women: taking action and generating evidence. Geneva, World Health Organization, 2010. 32. World Health Organization Department of Reproductive Health and Research. Preventing gender-biased sex selection: an interagency statement. Geneva, World Health Organization, 2011. 33. World Health Organization, Department of Reproductive Health and Research. Hormonal contraception and HIV. Technical statement. Geneva, World Health Organization, 2012 (WHO/RHR/12.08 ). 34. The WHO application of ICD-10 to deaths during pregnancy, childbirth and puerperium: ICD MM. Geneva, World Health Organization, 2012. 35. WHO guidelines for the management of postpartum haemorrhage and retained placenta. Geneva, World Health Organization, 2009. 36. Mikolajczyk RT et al. A global reference for fetal-weight and birthweight percentiles. The Lancet, 2011, 377:1855–1861. 37. International statistical classification of diseases and related health problems, 10th revision. Geneva, World Health Organization, 2010 (http://apps.who.int/classifications/icd10/browse/2010/en, accessed 1 January 2013). 38. World Health Organization, Department of Reproductive Health and Research. Unsafe abortion. Global and regional estimates of the incidence of unsafe abortion and associated mortality in 2008, 6th ed. Geneva, World Health Organization, 2011. 39. World Health Organization, Department of Reproductive Health and Research. National, regional, and worldwide estimates of stillbirth rates in 2009 with trends since 1995. Geneva, World Health Organization, 2011 (WHO/RHR/11.03). 40. World Health Organization, Department of Reproductive Health and Research. Prevalence and incidence of selected sexually transmitted infections. Chlamydia trachomatis, Neisseria gonorrhoeae, syphilis and Trichomonas vaginalis. Methods and results used by WHO to generate 2005 estimates. Geneva, World health Organization, 2011. 41. World Health Organization, Department of Reproductive Health and Research. Reproductive health strategy: to accelerate progress towards the attainment of international development goals and targets. Geneva, World Health Organization, 2004 (WHO/RHR/04.8) (http://whqlibdoc.who.int/hq/2004/WHO_RHR_04.8.pdf, accessed 19 December 2012). 42. World Health Organization, Department of Reproductive Health and Research, UNICEF. National-level monitoring of the achievement of universal access to reproductive health: conceptual and practical. Considerations and related indicators. Geneva, World Health Organization, 2008. 43. Reproductive choices and family planning for people living with HIV: counselling tool. Geneva, World Health Organization, 2006. 44. Consultation to review manufacturing, clinical and regulatory requirements for male circumcision devices to support programme expansion in high HIV incidence settings in Africa. Meeting report, Nairobi, Kenya 11–12 March 2009. Geneva, World Health HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 69 Organization, 2009 (http://www.who.int/hiv/pub/malecircumcision/ meetingreport_mar09/en/, accessed 29 January 2013). 45. World Health Organization, Department of Reproductive Health and Research. Programmatic and research considerations for hormonal contraception for women at risk of HIV and women living with HIV. Policy Implications. Geneva, World Health Organization, 2012 (WHO/RHR/12.09). 46. UNDP/UNFPA/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP). Providing the foundation for sexual and reproductive health. A record of achievement. Geneva, World Health Organization, 2008 (WHO/RHR/HRP/08.13). 47. World Health Organization, Department of Reproductive Health and Research. Emergence of multi-drug resistant Neisseria gonorrhoeae – threat of a global rise in untreatable sexually transmitted infections. Geneva, World Health Organization, 2011 (WHO/RHR/11.14). 48. World Health Organization, Department of Reproductive Health and Research – ExpandNet Beginning with the end in mind: planning pilot projects and other programmatic research for scaling up. Geneva, World Health Organization, 2011. HRP External Evaluation 2008−2012 (DRAFT) Chapter 2 Page | 70 Page | 71 Chapter 3 Efficiency and effectiveness of HRP’s governance, management and administration Nicholas Dodd Consultant, Sexual and Reproductive Health Page | 72 Page | 73 Contents 1 Introduction . 75 2 Funding and fundraising . 75 2.1 Overall funding . 75 2.2 Designated funding . 76 2.3 Leveraged funding . 78 2.4 Income from royalties . 78 2.5 HRP fundraising initiatives and achievements, 2008 to 2012 . 78 3 Financial management . 80 4 Cosponsorship . 80 5 Governance. 82 5.1 The Policy and Coordination Committee . 82 5.2 The Scientific and Technical Advisory Group . 84 5.3 The Gender and Rights Advisory Panel. 86 5.4 The Research Project Review Panel . 87 5.5 Costs of HRP governance . 89 6 Management and administration . 91 6.1 Managing for results . 91 6.2 Managing research grants . 93 6.3 Managing research . 94 Reference . 96 Annex 1: Trends in HRP income from royalties, 2000–2011 . 97 Annex 2: Summary of HRP resource-mobilization activities 2008–2012 . 98 Annex 3: HRP governance structure . 100 Page | 74 Abbreviations ERC Ethics Review Committee FEW-OHRP Federal Wide Assurance for the Protection of Human Subjects – Office for Human Research Protections Database GAP Gender and Rights Advisory Panel HRP UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction ICPD International Conference on Population and Development JCB Joint Coordinating Board OWER Organization-Wide Expected Result (WHO) PCC Policy and Coordination Committee PDRH Programme Development in Reproductive Health RAP regional advisory panel RHR WHO Department of Reproductive Health and Research RP2 Research Project Review Panel SERG Scientific and Ethics Review Group SPP UNFPA–WHO Strategic Partnership Programme STAG Scientific and Technical Advisory Group TDR Special Programme for Research and Training in Tropical Diseases UK United Kingdom of Great Britain and Northern Ireland UN United Nations UNAIDS Joint United Nations Programme on HIV/AIDS UNDP United Nations Development Programme UNFPA United Nations Population Fund UNICEF United Nations Children’s Fund USA United States of America WHO World Health Organization HRP External Evaluation 2008−2012 Chapter 3 Page | 75 1 Introduction This chapter does not attempt to review and evaluate all aspects of the governance, management and administration of HRP (the United Nations Development Programme (UNDP)/United Nations Population Fund (UNFPA)/United Nations Children’s Fund (UNICEF)/World Health Organization (WHO)/World Bank Special Programme of Research, Development and Research Training in Human Reproduction). Given the time and resources available, only a number of the more central issues have been examined. The chapter examines the following: funding and fundraising; financial management; cosponsorship; the functioning of the main governance bodies of HRP: the Policy and Coordination Committee (PCC), the Scientific and Technical Advisory Group (STAG), the Gender and Rights Advisory Panel (GAP), and the Research Project Review Panel (RP2); and selected aspects of programme management, including managing for results, managing research grants and managing research. 2 Funding and fundraising HRP’s level of income both reflects and affects its ability to achieve its goals and objectives. Fundraising is one of the prime responsibilities of the director of the Programme, but a responsibility that is subject to many external factors over which he or she has little control. Over the last 5 years, the most important of these has probably been the global financial climate, which has inevitably affected the development budgets of both governments and United Nations (UN) funds, programmes and agencies. 2.1 Overall funding shows trends in HRP’s income between 1996 and 2011, and the relation between its income and its approved budgets at both full and contingency levels. Income to the Programme declined each biennium between 1996 and 2005, and then increased by 80% in the next biennium, 2006–2007, to over US$ 45 million. In the last two biennia, income declined by around 10%, but remained relatively stable at a little in excess of US$ 40 million. Current income projections for the 2012–2013 biennium predict an increase to over US$ 45 million. Figure 1 HRP biennial income, and full and contingency budget levels, 1996–2011 (US$ millions) HRP External Evaluation 2008−2012 Chapter 3 Page | 76 Between 1996 and 2005, income consistently fell below the approved budget at both full and contingency level, and only in the 2006–2007 biennium did income exceed the budget. In the last two biennia, income has remained in between the full and contingency budget levels, though in reality closer to the contingency budget. In the current biennium, 2012–2013, the income and full budget gap is projected to continue at a shortfall of around 10%. Figure 2 shows trends in HRP income by major donor groups – governments; cosponsors; foundations; and civil society/private sector/other. Governments continued to provide the major portion of HRP’s income. Over the period 1996 to 2011, governments provided an average of 58% of HRP’s income. This has increased more recently, and in the 2010–2011 biennium it reached 72% of total income. Funding from cosponsors has declined over the last 15 years by around 50%, from US$ 14 million in 1996–1997 to around US$ 7 million in 2010– 2011. Contributions from foundations, though somewhat variable on a year-to-year basis, have become a significant source of HRP’s income. Figure 2 HRP biennial income by major donor grouping, 1996 to 2011 (US$ millions) 2.2 Designated funding The Programme prepared and the PCC approved guidelines for the acceptance of designated funding in 1998, and trends in core and designated funding from 2000 to the present are shown in Figure 3. Over the years, designated funding has continue to increase, and continues to be an important source of HRP’s income, currently running at around 20% of total income. In the 2000–2001 biennium, HRP had an income of US$ 33.3 million, of which US$ 858 248 (2.6%) was designated funding. After deducting costs for staff salaries1, general technical 1 In the 2006–2007 biennium, the Programme had 53 posts. In the 2010–2011 biennium, this had increased by only one to 54 posts. The ratios between professional and support staff have also changed positively, from being almost 1:1 up until 2005 to a ratio of around 2:1 in the 2010–2011 biennium. The Programme kept its staffing level more or less in line with its income over the period of the evaluation. However, the 53 posts in the 2006–2007 biennium cost the Programme US$ 13.4 million, whereas the 54 posts in the 2010–2011 biennium cost the Programme US$ 25.2 million, an increase of almost 90%. Approximately one third of this increase was due to changes in the exchange rate between the US dollar and the Swiss Franc, and two thirds was due to increases in salaries, post adjustment, and an 8.5% increase in WHO overhead payments with the introduction of the “post occupancy” charge. Currently, the Programme has some 20 vacancies. Seventy-five per cent of respondents gave the Programme a rating of “3” or better on its effectiveness in managing its human resources (scale 1 to 5 where 1 is highly effective and 5 is not at all effective). Many noted that the Programme is not completely independent in this regard, and is bound to the policies and procedures of WHO. HRP External Evaluation 2008−2012 Chapter 3 Page | 77 activities and programme management (US$ 13.0 million), the Programme had around US$ 20 million for research activities, of which more than 95% was undesignated. In the 2010–2011 biennium, HRP had an income of US$ 41.7 million, of which US$ 7.6 million (18.2 %) was designated. After again deducting costs for salaries, general technical activities and programme management (US$ 28.0 million), the Programme had around US $ 13.7 million for research activities, of which US$ 5.1 million (37.2%) was designated. Figure 3 HRP designated and core funding 2000–2011 (US$ millions) Thus, the total funds available for HRP research activities have decreased by 31.5% over the last 10 years; undesignated funding for research activities has decreased by 58 per cent, and designated funding for research activities has increased by a factor of almost six. In 2000–2001, every staff dollar implemented US$ 1.5 of activities; in 2010–2011, every staff dollar implemented US $ .50 cents of activities. Conclusion Core funding and core funds available for research have both declined significantly. The importance of core funding should not be underestimated, since it enables the HRP research agenda to be driven by needs and priorities that are independently and consensually identified; by the products that countries need to improve the quality of their sexual and reproductive health policies and programmes; and by scientific and technical assessments of their feasibility and likely impact. The unavoidable increases in the costs of staffing compounded by the weakening of the US dollar against the Swiss Franc have reduced the efficiency of the Programme over the last ten years. The increased proportion of designated income has also caused the Programme additional work in terms of developing proposals, and then monitoring and reporting on them. To regain its efficiency, the Programme needs an increase in undesignated funding, and needs to contract out a greater proportion of its work (see also section 6.3). Recommendation All donors to HRP should reflect on the importance of providing the Programme with undesignated funding, and, wherever possible, provide undesignated funds on a multiyear basis. Where this is not possible, the current practice of providing designated funds for specific HRP External Evaluation 2008−2012 Chapter 3 Page | 78 items of HRP’s already approved workplan and budget should continue. The Programme should explore the possibility of additional funding from new foundations located outside the United States of America (USA). 2.3 Leveraged funding Table 1 shows trends in leveraged funding for the last six biennia. Leveraged funding includes those projects in which HRP develops a partnership involving cost sharing or contributions in kind, or those projects that are brokered in their entirety by another agency. Data on HRP research projects and other activities that have involved this type of partnership are collected each year and the total value of the HRP investment in these projects since 2000 has been US $13.8 million. It is estimated that the Programme’s partners contributed an additional US$ 51.9 million to the same projects, thus representing a “leveraging” of almost US $4 for every dollar invested by the Programme. Table 1 Trends in HRP leveraged funding, 2000 to 2011, US$ millions Biennium HRP Funding Leveraged funding Ratio 2000/2001 2 317 410 9 676 068 4.2 2002/2003 3 723 679 12 926 904 3.5 2004/2005 847 350 3 492 713 4.1 2006/2007 1 355 100 6 144 003 4.5 2008/2009 1 908 685 7 095 800 3.7 2010/2011 3 691 670 12 535 804 3.4 Total 13 843 894 51 871 292 3.7 While direct support to HRP is critical, the catalytic role that the Programme can play by providing support to initiatives in sexual and reproductive health that are funded or implemented by partners is also important, and demonstrates an added value for every dollar invested in HRP. 2.4 Income from royalties HRP income from royalties continues at a little under 3% of total income, as can be seen from the table in Annex 1. 2.5 HRP fundraising initiatives and achievements, 2008 to 2012 HRP developed its first resource-mobilization strategy in 2006, made possible through a capacity-strengthening grant from The David and Lucile Packard Foundation. A professional fundraising consultant was hired to advise on the strategy, which included the development of a standardized communication and follow-up approach with donors; donor roundtables and regular briefings; developing marketing materials and key messages; and strengthened resource-mobilization capacity of all HRP staff. The process culminated in an all-day retreat for professional staff of the WHO Department of Reproductive Health and Research (RHR), to discuss application of the strategy in their day-to-day work. This initiative, which also resulted in a full-time staff position for fundraising, laid the groundwork for HRP resource mobilization up until the present. 2008–2012 was a period of both global fiscal crises and WHO financial reform, both of which brought threats of reduced income to the Programme. Despite the fact that the number of donors to the Programme declined between 2009 and 2011, HRP was able to maintain its overall level of support. For 2012, three new donors are expected to raise the total number of donors to the Programme from 22 to 25. HRP External Evaluation 2008−2012 Chapter 3 Page | 79 The Programme is also currently hoping to consolidate a number of resource-mobilization initiatives. For example, the governments of Norway and Flanders have committed a substantial increase in 2012; in 2012 the US Government is providing a core contribution for the first time since 2005; the governments of Switzerland and the United Kingdom of Great Britain and Northern Ireland (UK) have both indicated a willingness to discuss increased as well as multiyear contributions in the future; and other key donors, including the Bill and Melinda Gates Foundation, David and Lucile Packard Foundation, an anonymous donor, and the Swedish Government, have committed to maintain or possibly increase their support. The current financial position for the Programme seems very positive, and income for the 2012– 2013 biennium is predicted to reach a level in excess of all previous biennia. In addition, in 2011 PCC proposed the establishment of a resource-mobilization subcommittee. This was endorsed by PCC in 2012, and is expected to further support the Programme’s resource-mobilization work, and perhaps initiate other activities. A more detailed summary of all HRP’s resource-mobilization activities can be found in Annex 2. Respondents’ views on funding and fundraising are shown in Table 2. Over 40% of respondents gave a rating of “1” or “2” for HRP’s work on strengthening its fundraising since 2008, and over 70% gave a rating of “3” or better (scale 1 to 5 where 1 is highly effective and 5 is not at all effective). In commenting on their ratings, a large number of respondents indicated that the single most important way for the Programme to secure additional funding was to document and communicate the utilization, and thus potential impact, of its products in programme countries, and particularly how these help achieve global goals in sexual and reproductive health. In addition, respondents advised that HRP should clearly identify those products that would have a significant potential impact in programme countries, but that could not be developed owing to insufficient funding. This would highlight the advances and opportunities that would be lost whenever income fell below full budget level. Table 2 Respondents’ views of HRP’s work on strengthening fundraising since 2008 Rating Percentage of respondents 1 – highly effective 10.9 2 30.4 3 30.4 4 26.1 5 – not at all effective 2.2 Total (n = 46) 100.0 A number of other respondents felt that the Programme needed to continue its outreach and engagement with foundations and global health initiatives to seek additional funds. As previously mentioned in Section 10 of Chapter 2, the vast majority of key respondents felt that the reduction in funding of the Programme Development in Reproductive Health (PDRH) has had a very negative effect on the promotion and use of HRP’s products, and that this has only been exacerbated by the termination of the UNFPA–WHO Strategic Partnership Programme (SPP) in 2008. Both these factors have somewhat reduced the value of the Programme’s products, and this is an issue that must be addressed as a matter of urgency. Conclusion The Programme should be commended on its resource-mobilization efforts. During a period of global financial crisis, when many organizations have experienced severe funding cuts, the progamme’s resource-mobilization initiatives have enabled it to maintain income at over US$ 40 million for the last two biennia. The projected income for the current (2012–2013) HRP External Evaluation 2008−2012 Chapter 3 Page | 80 biennium is expected to surpass all previous biennia. While designated funding has become a significant proportion of total income, core funding is crucial for the Programme to continue to be driven by independently identified needs and priorities. Recommendation HRP needs to continue to build on the success of its resource-mobilization work and strengthen it further by demonstrating and communicating the utilization of its products in programme countries, their potential impact, and how this helps the achievement of global targets in sexual and reproductive health. 3 Financial management As demonstrated in Section 2, there have been major fluctuations in HRP’s income over the past 5 years. These fluctuations are compounded by the timing of the receipt of funds, with much income being received towards the end of each year. A number of respondents felt the Programme should follow up more frequently with donors, in order to expedite their contributions as early as possible during the year. PCC generally views HRP’s management of funds very positively, and more than 67% of respondents gave a rating of “1” or “2” for the efficiency of the Programme’s management of its financial resources, and a further 25% gave a rating of “3”. Some respondents felt that more communication on financial matters should take place between PCC meetings (though this is already being done with the standing committee), and that HRP needed to continue to be very transparent in distinguishing between financial reporting of HRP and PDRH. Conclusion The Programme has continued to manage its financial resources in an efficient and effective manner. 4 Cosponsorship HRP cosponsorship is defined in the Memorandum of Understanding (see Chapter 1 Annex 2) as an expression of commitment rather than a legally binding relationship. The original cosponsors were UNDP, UNFPA, WHO and the World Bank, and these have remained so since HRP became a cosponsored programme in 1988. However, two important developments have taken place since 2008. First, UNDP increased its involvement as a cosponsor, playing a more active role in both the standing committee and PCC, and returning as a financial contributor with a grant of US$ 120 000 covering the years 2009 to 2011. UNDP continues to be an important global promoter of both the links between sexual and reproductive health, HIV/AIDS and development, and the importance of the sexual and reproductive health targets to the overall achievement of the Millennium Development Goals (MDGs). As the lead UN coordinating agency at country level, UNDP has an increasing responsibility to provide the platform for strengthening coordination of the UN system in providing assistance to national sexual and reproductive health programmes, utilizing the evidence base provided by HRP. Secondly, the Programme has successfully attracted a new cosponsor, the United Nations Children’s Fund (UNICEF). This was agreed at the PCC meeting in June 2012, and hopefully will result in future financial contributions. It also strengthens the sphere of partnership and influence of the Programme, and further expands opportunities for the promotion and use of HRP’s products in programme countries, especially in the context of the one UN. UNICEF officially became a cosponsor in December 2012. HRP External Evaluation 2008−2012 Chapter 3 Page | 81 On a related topic, the Joint United Nations Programme on HIV/AIDS (UNAIDS), unable to become a cosponsor because it is itself a cosponsored programme, nevertheless was accepted by PCC in 2012 as a new permanent member of PCC, joining the International Planned Parenthood Federation in that capacity. This development strengthens HRP’s potential synergies in both basic and programme research where sexual and reproductive health and HIV overlap. Also, again, it strengthens partnerships at the country level for the application of research findings and technical guidance. The cosponsors of HRP make up the standing committee, and this normally meets three times a year. A typical schedule would be once at the time of the PCC meeting, once by video- or teleconferencing, and once at the headquarters of one of the cosponsors. This enables the costs of the standing committee to be very well controlled, cosponsors pay for themselves, and the cost to the HRP secretariat is a mere 1% of total governance costs. However, more importantly, the meetings at cosponsor agency headquarters give the opportunity for the HRP director to interact with executive directors and other senior staff of the cosponsors, as well as to provide information on the work of the Programme and its outputs, at open cosponsor staff meetings. This will be a continuing opportunity for both updating on new developments, and reinforcement of the existence and importance of HRP’s work, particularly in view of the normal cycles of cosponsor staff rotations. These meetings further represent an opportunity to discuss stronger cosponsor engagement in advising on future research priorities; stronger support from cosponsors for use of the Programme’s products in countries; and future financial contributions from cosponsors, which have waned considerably in recent years. As can be seen from Table 3, the World Bank has been the most consistent and strongest financial support of HRP over the last decade, providing US $22 million or 49.0% of the total support from the four cosponsors. Next is UNFPA, providing US$ 14.2 or 31.6% of cosponsor support. Between them, these two cosponsors provide over 80% of the financial support from cosponsors, while WHO provides almost all of the remaining 20 per cent. Table 3 Trends in funding by HRP cosponsors, 2000–2011 Biennium Cosponsor contribution (US$ millions) UNDP UNFPA WHO World Bank 2000/2001 0 3.0 1.3 4.0 2002/2003 0 2.6 1.1 3.0 2004/2005 0 3.7 2.1 4.0 2006/2007 0 2.2 1.6 4.0 2008/2009 0.06 1.0 1.5 3.5 2010/2011 0.06 1.8 1.3 3.5 Total 2000–2011 0.12 14.2a 8.6 22.0a a All figures have been rounded up. Overall totals in the final row are correct, though they do not correspond exactly with totals derived from the figures in the body of the table because of rounding errors. While a number of respondents to the questionnaire felt that the political and financial support provided by cosponsors was sufficient, a larger number felt that it could be much improved. This group felt that financial support to the Programme from cosponsors should be increased, that cosponsors should also become stronger champions for HRP, and that new partnerships such as the previous SPP with UNFPA should be developed to add more value to the Programme’s products. Respondents felt that the joint UN statements initiated by the Programme, but issued under the names of the cosponsors, and sometimes additional UN funds agencies and programmes, was a clear example of the added value of cosponsorship. HRP External Evaluation 2008−2012 Chapter 3 Page | 82 On the possibility of additional cosponsors, responses were somewhat mixed. A number of respondents cautioned against further increasing the number of cosponsors because it would cost the Programme more time investment in exchange for perhaps little additional income and possibly greater influence on the research agenda. However, the majority was in favour of increasing the number of cosponsors, providing they were willing to commit funds and the necessary time to help guide the Programme. Most respondents in this group indicated that UN Women and the UN High Commission for Refugees could be the next two most logical organizations for the Programme to approach. Finally, respondents felt that the Programme needed to further strengthen and continue its direct communication with the executive directors and senior management of the cosponsors, in order to promote all aspects of cosponsorship: funding, political support and partnerships. Conclusion Cosponsorship continues to be considered a very important element of the fundamental structure of HRP. However, it needs to be reinvigorated, as much by the cosponsors themselves as by the Programme. Recommendation HRP and the cosponsors need to strengthen their engagement, developing clear plans and mechanisms to use the programmatic experience and networks of the cosponsors to help identify key research questions and needs for policy, programmatic and technical guidance, and to use their programmes and networks to promote and expand the use of HRP’s products in countries. A progress report should be presented to PCC after 2 years. The Programme should, somewhat cautiously, explore additional cosponsors. 5 Governance This section of the evaluation examines the functioning of PCC, STAG, GAP and RP2. A review of the regional advisory panels (RAPs) can be found in the case study on research-capacity strengthening in Chapter 5. A later section of this chapter reviews the costs of HRP governance, and a schematic chart of the HRP governance structure can be found in Annex 3. The governance of HRP is laid down in its Memorandum of Understanding, which defines the composition and role of PCC and STAG (see Chapter 1 Annex 2). PCC provides overall programme monitoring, strategic guidance and financial oversight, while STAG provides overall scientific and technical guidance on the Programme’s work. These are supplemented by two other important mechanisms, GAP and RP2. GAP advises the Programme on issues of gender equity and equality and human rights, and ensures a gender orientation in all its work. Reports of GAP are presented at both the STAG and the PCC meetings. RP2 reviews every research project proposed for support, and no project can be funded unless RP2 provides a positive scientific, technical and ethical review. 5.1 The Policy and Coordination Committee Membership of PCC is determined by the Memorandum of Understanding. All major policy, financial and strategic decisions are either made by or endorsed at PCC meetings. A clear valued-added mechanism for the governance of HRP is that the Programme maintains a system whereby it documents its follow-up actions to all recommendations made by PCC, and the subsequent meeting always provides the opportunity for PCC to examine and further review the Programme’s responses. (This mechanism also applies to STAG and GAP.) HRP External Evaluation 2008−2012 Chapter 3 Page | 83 PCC is made up of various categories of members. Category 1 members of PCC are the 11 largest donors to the Programme in the previous biennium; category 2 members are 12 representatives from programme countries selected by the WHO regional offices; and category 3 members are two additional members elected by a ballot of category 1 and 2 members every 3 years. Up to the present, category 3 members have always b

View the publication