Getting the Numbers Right- A Guide to USAID-Developed Contraceptive Forecasting Tools

Publication date: 2009

Getting the Numbers Right A Guide to USAID-Developed Contraceptive Forecasting Tools November 2009 Abstract: Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools aims to assist USAID health officers, USAID Cooperating Agencies, and USAID partners identify appropriate tools for forecasting contraceptive demand and need. The Guide describes the following four USAID-funded tools: the FamPlan module of Spectrum, developed by the POLICY Project; CastCost, developed by the Centers for Disease Control and Prevention; Reality √, developed by the ACQUIRE Project; and PipeLine developed by the DELIVER Project. The Guide contains a narrative description of each tool, including the goals, uses and requirements of each tool, a situational analysis section to help determine which tool(s) to use in what situations, and a comprehensive table that allows side-by-side comparison of the features of each tool. Each of the tools has distinct characteristics and strengths in terms of ease of use, requirements, output and purpose. This Guide will assist readers identify the tool that best fits their specific needs and context. Acknowledgements: We thank Trisha Long, John M. Pile, Tim Johnson, Susanna Binzen, Karen Foreit, Suneeta Sharma, and Dana Aronovich and for providing information on their organizations’ tools for use in the guide. We thank Leila Yosef for assistance with editing and formatting. And, we especially thank Dana Aronovich for assisting USAID with the development of a strategy and tool for developing this guide. Recommended citation: Bagga, Sharlene; Simpson, Tara; Schaffner-Gurney, Ruth & Pilz, Kevin. 2009. Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools. Pilz, K. (Ed.). Washington D.C. U.S. Agency for International Development. For more information or assistance with this guide, please contact: U.S. Agency for International Development Bureau for Global Health Office of Population and Reproductive Health Commodities Security and Logistics Division Email: csl@usaid.gov i Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools Table of Contents: Acronyms page iii Introduction page 1 FamPlan Module of Spectrum page 4 CastCost Tool page 6 Reality √ page 8 PipeLine page 10 FAQ page 12 Tool Comparison Chart page 14 ii Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools Acronyms: AIDS – Acquired Immunodeficiency Syndrome CA – USAID Cooperating Agency CDC – Centers for Disease Control and Prevention CPR – Contraceptive Prevalence Rate CPT – Contraceptive Procurement Tables CYP – Couple-Years of Protection DHS – Demographic and Health Survey FAQ – Frequently Asked Questions HIV - Human Immunodeficiency Virus IDB – International Data Base JSI – John Snow, Inc. MOH – Ministry of Health MWRA – Married Women of Reproductive Age NGO – Non-Governmental Organization RHS – Reproductive Health Survey SPA – Service Provision Assessment TFR – Total Fertility Rate UN – United Nations UNFPA – United Nations Population Fund USAID – U.S. Agency for International Development WRA – Women of Reproductive Age iii Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools Introduction: The purpose of this guide is to help USAID health officers, USAID Cooperating Agencies (CAs), and USAID partners [e.g. Ministries of Health (MOH), United Nations (UN) agencies, and non- governmental organizations] identify appropriate tools for use in forecasting contraceptive demand and need. Though this tool only addresses tools developed by USAID, it may also be useful for organizations not affiliated with USAID. Quality forecast data are critical for planning procurements of contraceptives, advocating for funding to meet family planning program needs, planning program growth, and setting programmatic goals; however, not all forecasting tools are equally appropriate for each distinct objective. This guide will assist USAID staff and partners to select the correct tool or combination of tools. The tools included in this guide, developed by partners with USAID funding, are: • FamPlan module of Spectrum, developed by the POLICY Project (Futures Group International); • CastCost, developed by the Centers for Disease Control and Prevention (CDC); • Reality √, developed by the ACQUIRE Project (EngenderHealth); • PipeLine developed by the DELIVER Project (John Snow, Inc.). The first three tools listed above rely on demographic and other survey data to provide demographic forecasts of contraceptive demand. The fourth tool, PipeLine, relies on historical logistics data to estimate future contraceptive requirements. Specifically, this guide contains the following information: • a narrative on each of the four contraceptive demand forecasting tools including the goals, uses and requirements of each tool (pages 4-11); • FAQ/situational analysis to help decide which tool(s) to use (pages 12-13); • a detailed comparison chart that allows side-by-side comparison of the features of each tool (pages 14-19). The essential differences between these tools is briefly summarized in the Quick Key table on page 3, which differentiates between the tools based on purpose, forecast type, ease of use, and robustness of forecasts. This short table simplifies the distinctions among the four tools, but it provides a quick key for basic understanding of each. Much greater detail on each of the tools is given in the subsequent sections. An important consideration for the user of any of these tools is that the quality of the projections (i.e., the output) depends absolutely on the quality of the input data entered and the validity of the assumptions made. To generate reliable projections for a program, users should have good knowledge of the program and be aware of aspects of the program that may change and impact contraceptive demand or use. Users should also understand any weaknesses of the data they enter, and of assumptions implicit in the data, and understand how those weaknesses may impact the projection of future needs. For all tools, the projections generated should be 1 Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools 2 considered carefully - with respect to input data quality, assumptions made, expected programmatic changes, and programmatic goals – and adjusted accordingly before being utilized; this is especially true when the output will be used for procurement purposes, but also true when it will be used for program planning and advocacy purposes. Users who are not familiar with technical aspects of forecasting should consult some of the many resources available that provide guidance on forecasting methodologies, including considerations about data quality and assumptions.1 1 For example, see the following resources, which are available at: http://deliver.jsi.com/dhome/resources/publications/guidelines: USAID | DELIVER PROJECT, Task Order 1. 2008. Quantification of Health Commodities: A Guide to Forecasting and Supply Planning for Procurement. Arlington, Va.: USAID | DELIVER PROJECT, Task Order 1. Family Planning Logistics Management (FPLM). 2000. Contraceptive Forecasting Handbook for Family Planning and HIV/AIDS Prevention Programs. Arlington, Va.: FPLM/John Snow, Inc., for the U.S. Agency for International Development. Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools Quick Key to the Four USAID-Developed Contraceptive Forecasting Tools: Forecasting Tool: Characteristics: FamPlan CastCost Reality √ Pipeline Appropriate to forecast future commodity costs for advocacy and planning purposes? Yes Yes Yes Only over short term Appropriate to forecast future commodity quantities for procurement purposes? Yes over the short term, when historical logistics data are not available. When historical logistics data are available, FamPlan is an excellent tool for validating logistics-based forecasts. Not as the primary tool, but well-suited to validating logistics-based forecasts. Not as the primary tool. Can be used to validate other logistics forecasts, but this is not its primary intended purpose. Yes, with sufficient historical logistics data, but Pipeline outputs must to be adjusted for program factors and should be validated with other methodologies. Forecast type Demographic Demographic Demographic Logistics Software type Database-driven with windows interface Microsoft Excel spreadsheet Microsoft Excel spreadsheet Database-driven with windows interface Input data requirements Multiple demographic and population variables Fewer demographic and population variables Fewest demographic and population variables Historical logistics data, manually adjusted for reporting rates and stockouts Ease of use for computing novices More challenging to use Easy to Use Easiest to use More challenging to use Output data robustness Most robust and most detailed output among demographic tools Moderate to less robust, easy to interpret Less robust, easy to interpret Robust, if appropriate assumptions and adjustments are made 3 Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools FamPlan Module in Spectrum Overall Description: This relatively sophisticated tool is designed to aid in program planning at the national level. Developer: Futures Group International’s POLICY Project To Access the Tool: You can access the tool for free online on the USAID | Health Policy Initiative web site under the Software tab: http://www.healthpolicyinitiative.com/index.cfm?id=software&get=Spectrum Main Point of Contact: For more information, for assistance with this tool, or to offer your feedback on the tool, please contact Suneeta Sharma at ssharma@futuresgroup.com or policyinfo@healthpolicyinitiative.com. Main Goals of Product: FamPlan can be used to estimate the service delivery burden and the cost that will be incurred to expand family planning programs and achieve national goals. It also projects the impact of increased or modified contraceptive use on fertility and other demographic indicators such as infant and child mortality. By comparing the outputs of different scenarios, the user can plan for service expansion to meet program objectives and evaluate alternative methods for achieving specific goals. Languages: English, French and Spanish Intended Users: Ministry of Health staff, national-level policy makers, donors and CAs. Pre-requisites to Using the Tool: Users of the tool should be comfortable with a database- driven program that uses a windows interface. The tool requires a reasonable degree of sophistication and prior experience (or orientation) to manage efficiently. An overview training is ideal to take advantage of the full utility of the tool. There is a three-part online tutorial to assist users in entering the demographic information portion of the tool at: http://www.healthpolicyinitiative.com/index.cfm?id=demProjE. Advanced users may be able to simply download the tool and use it after reviewing its user’s guide, available on-line at: http://www.healthpolicyinitiative.com/index.cfm?id=software&get=Spectrum. Input Data Sources Needed: • Demographic and Health Survey (DHS) or Reproductive Health Survey (RHS); FamPlan allows users to automatically upload the relevant DHS or RHS data. • National Population Policy or Family Planning Strategic Plan. Input Data Requirements: • Base year population projection. The user can either input UN Population Division demographic data, supplied with the tool, or build a population projection by hand. 4 Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools • Contraceptive prevalence rate • Method mix • Source mix • Cost per user • Proportion of women in union • Duration of postpartum infecundability • Sterility rate • Abortion rate Projection Method: FamPlan is built on the proximate determinants of fertility: marital status, contraceptive use, breastfeeding and abortion. It allows the user to choose among five types of national family planning goals: to reduce unmet need for contraception, to achieve desired fertility, to meet contraceptive prevalence or total fertility rate goals, or to meet specified expenditure levels. In addition, the user can manipulate contraceptive method and source mix to reflect introduction of new methods, targeted promotion of long-acting methods, cost- recovery initiatives, increased private sector participation through social marketing, or other family planning program priorities. Output Data: For each scenario tested, FamPlan produces estimates of commodity needs, users, acceptors and costs by contraceptive method and source. Other outputs include contraceptive prevalence rate (CPR) and fertility impacts such as total fertility rate (TFR), births and abortions. The user can display multiple scenarios to examine the effects of different assumptions. Utilization of Output Data: The outputs produced by FamPlan are typically used to estimate the service delivery burden and costs to expand a family planning program to achieve national family planning goals. The model can be used to explore program alternatives including changes in method mix and source mix (e.g., public sector, private sector, and subcomponents). FamPlan can also be used for advocacy purposes. Strengths: The FamPlan tool is the most robust tool for creating forecasts based on demographic data. The FamPlan user can manipulate the inputs in several different ways to see potential program outcomes, thus making this an excellent tool for program planning at the national level. It can be learned readily especially by those familiar with similar programs, downloadable on your computer, and free. UN Population Division estimates and projections as well as DHS data are automatically supplied by the tool itself, thus saving the user time. The demographic data that are entered for FamPlan can be used for other models in the Spectrum Policy Modeling System, which focus on a wide range of public health issues including HIV/AIDS and maternal and child health. Limitations: FamPlan is a sophisticated tool that can be difficult to pick up and use with little prior orientation. FamPlan can only be used for program planning at the national level or other macro level such as urban/rural (every level of disaggregation requires its own set of inputs). 5 Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools CastCost Overall Description: This relatively user-friendly tool is primarily used to estimate quantity and cost of contraceptives that will be needed in a country for each of the next five years. It is known as the Contraceptive Forecast and Cost Estimate Spreadsheet, or CastCost. Developer: Centers for Disease Control (CDC), Division of Reproductive Health To Access the Tool: CastCost is free to the public. Please email drhinfo@cdc.gov for a copy of the tool and its user’s guide. CastCost is available on CD-ROM and will soon be available at www.cdc.gov/reproductivehealth on the Contraceptive Logistics and Forecasting page. Main Point of Contact: For more information, assistance with this tool, or to offer your feedback while using this tool in the field, please contact Susanna Binzen at syb7@cdc.gov or Tim Johnson at jtj2@cdc.gov, or via telephone at +1(770)488-5200. Main Goals of Product: CastCost estimates the quantity and cost of contraceptives needed for the next five years, based on survey data or equivalent information. It can also be used to test scenarios of different method prevalences, or of different contraceptive unit costs from different suppliers, and can be an advocacy tool. Languages: Currently only English. Will soon be translated into Spanish. Intended Users: Primary audience: Ministry of Health personnel, procurement unit, program managers, policy makers, budget personnel, contraceptive security teams in countries graduating from USAID family planning assistance. Secondary audience: CAs, donors, and non- USAID countries. Pre-requisites to Using the Tool: Familiarity with Microsoft Excel and with the CastCost user’s guide that accompanies the tool is likely sufficient for use. A shortened tool overview is currently being developed for print and electronic distribution. However, in the meantime, Susanna Binzen at syb7@cdc.gov or Tim Johnson at jtj2@cdc.gov are available to give a 5-10 minute overview to new users. Input Data Sources Needed: • Contraceptive prevalence data from Reproductive Health Surveys or from Demographic and Health Surveys – ideally current and past data. • Demographic data from the U.S. Census Bureau International Database (IDB, www.census.gov/ipc/www/idb/), the Population Reference Bureau (www.prb.org), the United Nations World Population Prospects Database, or local sources. Input Data Requirements: • Number of women ages 15-44 (or 49) • Annual rate of population increase • Percentage of women of reproductive age who are in union 6 Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools • Current contraceptive prevalence rate by method • Estimated contraceptive prevalence rate (CPR) by method 5 years from the starting date of the projection • Source of method/supply Projection Method: CastCost forecasts are calculated using spreadsheet-generated projections of the number of women in union, the user’s estimates of contraceptive prevalence for each method 5 years forward, and couple-years of protection (CYP) factors. Historical prevalence data from multiple previous years (up to five) can be entered, in which case CastCost will produce a linear projection of prevalence by method to aid the user in making estimates. Users should also consider the following factors in estimating a reasonable projected prevalence rate: program history such as past shortages or stock outs, and future program plans. Output Data: CastCost produces graphs of contraceptive prevalence trends by method and projection tables of the number of users and of contraceptive units. The tool also produces annual quantities and costs for all methods for each of the five years in the projection, or by individual method, for each sector (private, public, all, and subcomponents). Utilization of Output Data: The outputs produced by CastCost can be used for a variety of functions at different staff levels. Program planners can use the output data to test scenarios involving different method mixes and different prevalence rates, or to plan for budgets and commodity storage. Program managers can use the output data to do budget planning and planning for commodity storage. Logistics managers can use the output data to estimate the quantities that will need to be managed and to validate their logistics-based forecasts. Contraceptive security teams can use the output data for estimating and budget planning for when donations cease. Strengths: As CastCost is Microsoft Excel spreadsheet-based, it is relatively easy to learn to use. Its requirements for input data are also relatively light. The tool comes with a helpful user’s guide. It can be used at the national and sub-national levels (either by sectors, institutions, pharmacies, hospitals), or by regions/districts if survey data are available at those levels. It allows projections based on previous data from multiple years. CastCost automatically calculates commodity needs by sector (private sector, public sector, and their subcomponents as available in RHS/DHS data). It also allows the user to add in contraceptive methods and donors not included as a default in the program. Limitations: The user must have adequate information available to estimate future contraceptive prevalence rate, as the input value will greatly influence the tool’s projections. Projections provided by CastCost incorporate fewer demographic variables and use some simplifying assumptions, the projections may therefore may be less robust than those of more sophisticated tools, such as FamPlan. 7 Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools Reality √ Overall Description: This highly user-friendly tool is designed to generate contraceptive forecasts for evidence-based advocacy and planning. The Reality √ tool helps users plan based on informed estimates of contraceptive need. It can be used as an important advocacy tool, helping users set reasonable family planning program goals by illustrating the inputs required to achieve those goals. Developer: EngenderHealth’s ACQUIRE Project. To Access the Tool: The Reality √ tool and the user’s guide can be downloaded for free on the RESPOND Project web site, www.respond-project.org, and at the ACQUIRE Project Digital Archive, http://www.acquireproject.org/archive/html/3-program-effectively/tools.html. The tool is also available on CD-Rom. Main Point of Contact: For more information or for assistance with this tool, or to offer your feedback while using this tool in the field, please contact: John M. Pile at JPile@engenderhealth.org. Main Goals of Product: The main goal of Reality √ is to provide a user-friendly advocacy and program tool that encourages using evidence-based information. The tool can be used to set realistic family planning goals, plan for service expansion to meet program objectives, and evaluate alternative methods for achieving specific goals. A key feature of the tool is that it enables users to quickly test future scenarios for a program, including whether current goals are achievable or feasible. It can help managers better understand the costs of continuing reliance on a particular method in a program as well as the potential benefits of expanding method mix to promote use of more effective contraception. Languages: Currently available in English. French version under development. Intended Users: Ministry of Health personnel, health planners and administrators at the national, regional, provincial and district government levels, and family planning programmers at donor agencies and CAs. Pre-requisites to Using the Tool: Familiarity with Microsoft Excel and with the Reality √ user’s guide that accompanies the tool is likely sufficient to use this tool. The user’s guide is available on-line at http://www.acquireproject.org/archive/html/3-program-effectively/tools.html, and on the RESPOND Project web site www.respond-project.org. An animated demo of the tool will be available in early 2010. The tool is best introduced and used in a workshop setting to increase the users’ capacity to understand and use the tool, and to generate relevant questions and scenarios to examine with the tool. Input Data Sources Needed: • CPR data derived from Demographic and Health Survey (DHS) or Reproductive Health Survey (RHS) or National Family Planning Service Statistics 8 Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools • Population data on women of reproductive age or married women of reproductive age derived from the United Nations World Population Prospects Database (provided on Reality √ CD-Rom), national census, or other population database [e.g. U.S. Census Bureau International Database (IDB) www.census.gov/ipc/www/idb or the Population Reference Bureau www.pbr.org]. Input Data Requirements: • Contraceptive prevalence rate (CPR) • Number of women of reproductive age (WRA) or married women of reproductive age (MWRA) • Number of service delivery sites and contraceptive methods they offer (optional) Projection Method: Reality √ uses a linear projection methodology based on current and past contraceptive prevalence (i.e., two data points) to project trends and future goals. Reality √ users input estimates of projected prevalence and/or projected method mix, and the tool provides guidance on factors to include in estimating these values. Output Data: Reality √ produces estimates of CPR, annual number of users, number of adopters (new users), annual number of implant removals, number and cost of commodities, estimated client load per month/per site, and CYP. Projections are displayed in both numeric format and in charts/graphs. Utilization of Output Data: The worksheets and trend information generated by Reality √ are primarily used to estimate the service delivery burden and the inputs required (including costs and training requirements) to expand a family planning program to achieve national or other goals. The model can be used to explore program alternatives and to advocate for resources to meet family planning needs. For example, an MOH official could use the projections to advocate for more family planning or long-acting and permanent method supplies, or to help set realistic family planning goals at the national, regional or district levels. Strengths: As Reality √ is a Microsoft Excel spreadsheet-based, it is relatively easy to learn to use. Its requirements for input data are minimal. Reality √ can be used at a number of levels – national, regional, district, hospital and even clinic levels. It allows the user to quickly generate alternative scenarios (e.g. shifts in method mix) and assess the potential impact of these scenarios in terms of cost, number of users, caseload, etc. Limitations: Conducting analyses at levels below the general population level may be difficult since required input data is often not easily available. Reality √ does not automatically calculate contraceptive need disaggregated by specific sectors. Adding contraceptive methods not included in the model is cumbersome. Reality √’s projections may be less robust that those of the other tools because Reality √ incorporates fewer demographic variables; for example, projections are based on current CPR and past CPR from one year only (future versions will be able to utilize past CPR data from multiple years). As the projection methodology is linear, long-term projections (e.g., more than 5-10 years) are less reliable. 9 Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools PipeLine Overall Description: This relatively sophisticated tool is designed to help plan procurement activities. PipeLine produces a specific procurement plan entailing when quantity of contraceptive methods should be procured, utilizing forecast information input by the user from any forecasting tool. PipeLine’s trend analyzer feature can be used to estimate future commodity needs based on historical logistics data. Developer: John Snow, Inc’s (JSI) DELIVER Project To Access the Tool: PipeLine can be downloaded for free at http://deliver.jsi.com/dhome/resources/tools/softwaretools/pipeline and is also available on CD- Rom at no cost by mail. The user’s guide is also available here: http://deliver.jsi.com/dhome/resources/tools/printedtools/scessentialsprintedtools. Main Point of Contact: For more information or for assistance with this tool, or to offer your feedback while using this tool in the field, please contact: Trisha Long at trisha_long@jsi.com or askdeliver@jsi.com. Main Goals of Product: PipeLine is not designed primarily as a forecasting tool, but as a procurement planning tool that identifies what quantity of commodities should be procured when. However, PipeLine’s trend analyzer feature can be used to estimate future commodity needs based on historical logistics data. Languages: English, Spanish, French, Arabic, and Portuguese Intended Users: Logistics and program managers in a wide variety of settings including Ministries of Health, donors, and procurement units. Pre-requisites to Using the Tool: Users of the tool should be comfortable with a database- driven program that uses a windows interface. The tool requires a reasonable degree of sophistication and prior experience (or orientation) to manage efficiently. A half-day to full-day training is ideal. Advanced users may be able to simply download the tool and use it after reviewing its user’s guide which is available on the CD-Rom with the software or on DELIVER’s web site: http://deliver.jsi.com/dhome/resources/tools/softwaretools/pipeline. Basic understanding of logistics concepts will also be important for appropriate use of the tool. Input Data Sources Needed: The required logistics data are often available from a logistics management information system. Program data are required to make appropriate adjustments and can be gathered from key informants or policy documents. Input Data Requirements: • For the forecasting function, the user will need at least two years of historical data on facility-level consumption (i.e., the number of commodities dispensed to users). When 10 Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools consumption data is not available, issues2 data can be substituted in some cases. • The historical logistics data must be adjusted so that it approximates the total consumption (i.e., number of commodities dispensed to users) supported by the program for each method. Examples of required adjustments include adjusting for stockouts or non-reporting facilities that artificially reduce historical consumption values, or adjusting for one-time outreach campaigns that temporarily increased consumption. Projection Method: PipeLine generates a linear regression graph based on historical logistics data that projects expected future consumption. Output Data: Based on the historical logistics data and projected consumption trends, PipeLine generates graphs and tables with estimates of future consumption for each product. PipeLine also generates procurement plans – based on any forecast consumption data the user enters - that include cost information, national stock status reports, and reports showing shipments from various suppliers. Utilization of Output Data: The future consumption projections estimated by PipeLine, based on historical consumption trends, should be adjusted to account for future programmatic expectations and goals before use. Once adjusted, the forecasts can be used to develop procurement plans or to plan for programs and supply chains. PipeLine will produce procurement plans using any user-provided forecast consumption data. PipeLine can be used to identify future supply issues such as funding gaps. In addition, PipeLine generates Procurement Table Reports and Shipment Summary Reports, which can be used by USAID Missions to produce Contraceptive Procurement Tables (CPTs) for placing contraceptive orders with USAID/Washington. Strengths: PipeLine is a very robust tool for estimating future commodity needs based on historical logistics data. This tool is an excellent resource for planning immediate and short- term contraceptive needs and can be invaluable in procurement planning. Limitations: PipeLine is a sophisticated tool that can be difficult to use with little prior orientation or without understanding of basic logistics concepts. PipeLine forecasts are only reliable where input data accurately reflects true consumption of contraceptive methods. As the tool does not automatically account for supply stock outs or poor reporting, the user must have sufficient knowledge and technical capacity to manually adjust for these limitations. PipeLine forecasts should be adjusted based on programmatic information and validated with other forecasting methodologies (i.e., demographic or service-based forecasts) before making final decisions on procurement quantities. PipeLine can be used only at the program level and not at any lower levels of the supply chain. The tool does not forecast number of users or future CPR. 2 Product issued from one level of the health system (e.g., from a warehouse) to another (e.g., to a service delivery site or lower level warehouse). When consumption data is not available, issues data can sometimes be used in logistics-based forecasts as a rough estimate of facility-level consumption. The input data should be the issues from the lowest level of the system where data is available, so that it most accurately reflects consumption. 11 Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools Frequently Asked Questions Which USAID-developed tool should I use… Q1. If I would like to estimate the quantities of contraceptive methods I need to procure or order for the next year or two, which tool is best to use? To determine the number of units of each contraceptive method to order for one to two years into the future, the most appropriate tool is PipeLine. When estimating contraceptive requirements through PipeLine, you should input historical consumption or issues data, adjusting for artificial variation due to stock-outs that limited consumption, one-time outreach activities that created a short-term increase in consumption, or reporting rates of less than 100%. These factors must be accommodated before data is entered into PipeLine to create accurate forecasts. PipeLine can then analyze the trends for contraceptive requirements, and estimate future needs. It is a good idea to complement PipeLine projections with demographic projections; FamPlan and CastCost are particularly well-suited for validating logistics-based projections, and Reality √ may also be used. Note that demographic projections often estimate greater requirements than logistics-based projections. Once different projections are compared and reconciled, adjustments should be made based on expected program changes for coming years before procurement quantities are finalized. If you are forecasting for long- acting and permanent method use, Reality √ will provide additional output data on needed supplies and equipment that the other tools do not provide. Q2. If I am a program manager or advocate and would like to forecast the number of contraceptives needed and their cost over the longer-term for program planning or advocacy purposes, which tool would be best to use? The three tools that are best suited for longer-term projections for planning and advocacy purposes are FamPlan, CastCost and Reality √. These tools use demographic data to project number of contraceptive users over the longer term, and based on that information calculate the number of needed commodities and their cost. This method is especially relevant for illustrating future demand. These three tools thus provide data vital to advocacy efforts, including possible costs of meeting currently unmet need for family planning. In determining whether to use FamPlan, CastCost and Reality √, please consider the following three factors: 1) FamPlan is a more sophisticated tool that requires more input data and greater technical capacity on the part of the user, but – for those reasons – can also provide the most robust estimates of new clients needed or other desired output data; 2) Where limitations on user capacity, available input data, or other factors prohibit FamPlan use, Reality √ and CastCost can provide good estimates of the needed number of new clients. These two tools allow the user to easily and quickly test the impact of different scenarios. Reality √ requires the least input data and has the simplest interface. CastCost allows the user to enter more demographic variables (including CPR data by method from a series of previous DHSs) as the basis for the projection, and thus may provide more robust estimates. 3) FamPlan and CastCost are best suited for use at the national program planning level while Reality √ can be used at any level – for example, at the national level, regional or district level, hospital level, or even the clinic level. 12 Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools 13 Q3. I need to create contraceptive procurement tables to report planned shipments to USAID. Which tool is the best to use to create the tables? Contraceptive Procurement Tables (CPTs) are procurement plans, which require a forecast of near future commodity requirements. PipeLine generates contraceptive procurement tables utilizing forecast information entered by the user; thus these forecasts can be generated using any methodology or tool. When forecasting for procurement purposes, logistics based forecasts (such as those which can be generated using PipeLine) should be adjusted based on program knowledge and validated with forecasts generated by other methodologies (i.e., demographic or service-based forecasts). See Q1 for more information. Q4. If I would like to estimate the number of new clients needed in order to increase the Contraceptive Prevalence Rate by a certain percent per year, which tool would be best to use? FamPlan and Reality √ are designed to automatically calculate the estimated number of new clients needed each year to meet specific CPR targets. Both of these tools, as well as CastCost, can estimate the client growth, by method, that you would need to meet a specific CPR target. See the answer to Q2 for description of the differences between FamPlan module in Spectrum, CastCost and Reality √. Q5. In the country where I work, the public sector is the main source of contraceptives and its market share is growing. Which tool can help me examine how improved market segmentation (involvement of the private sector) would affect the commodities required by the public sector, and the financing required? CastCost and FamPlan are designed to automatically generate sector-specific projections. These tools can also provide additional outputs that may help determine how private sector involvement might affect the commodities and financing required for public sector services. Q6. One of our main donors has informed us that they are phasing out their donations of contraceptives over the next few years. Which tool can help us understand how this will impact our program, our future contraceptive procurements and how much funding we will need to fill the gap? PipeLine will provide this information for short-term procurement and shipment planning. The other tools will provide longer-term projections of commodity and financial requirements. Q7. Our National Contraceptive Security Committee would like to understand the financial implications of reducing unmet need by a certain percent per year to advocate for additional resources from our government and donors. Is there a tool that can help us do that? Only the FamPlan tool is designed to generate projections specifically based on meeting a percentage of unmet need. If the Contraceptive Security Committee is primarily concerned with the financial implications of increased numbers of users, Reality √ and CastCost can also be used. Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools 14 USAID-DEVELOPED CONTRACEPTIVE FORECASTING TOOLS DETAILED COMPARISON CHART Factor PipeLine CastCost Reality √ FamPlan/Spectrum Purpose Primarily a procurement planning tool that identifies what quantity of commodities should be procured when. However, PipeLine’s trend analyzer feature can be used to estimate future commodity needs based on historical logistics data. To support planning and advocacy for family planning/RH programs, and to generate estimates of the quantity of contraceptives that will be needed for each of the next five years and their likely cost. To support planning and advocacy for family planning/RH programs, and to show requirements in terms of commodities and service delivery to achieve goals at a national, regional or lower level. To support planning and advocacy for family planning/RH programs, and to show requirements in terms of commodities and service delivery to achieve national goals. Intended Users Logistics and program managers in a wide variety of settings, including ministries of health, donors, and procurement units. MOH, procurement unit, program managers, policy makers, Ministries of Finance and contraceptive security teams in graduating countries. CAs and donors would be secondary users. MOH personnel, health planners and administrators at the national, provincial and district levels, and family planning programmers at donor agencies or CAs. MOH program managers, national policy makers, donors and CAs. Intended Level of Use National level National and sub-national (private or public sector, institutions that supply contraceptives, pharmacies, hospitals, etc.) or by region/province if appropriate input data are available. National, regional, provincial, district or clinic site levels Primarily national, but can also be used for urban/rural and by different sectors. Inputs: Sources of Data Used Logistics Management Information Systems and other sources of logistics data Any source of forecast data can be used for procurement planning. Reproductive and demographic surveys (current and past, if available), Census Bureau International Database (IDB) or other population data sources. Prevalence data from DHS, RHS and/or family planning service statistics and population data from UN World Population Prospects Database, census or other population database. Optional: Service delivery site data from national health statistics or Service Provision Assessments (SPA). DHS, national population policy Data Inputs Required Historical consumption data adjusted for reporting, stockouts, and other deviations from true consumption. Unit cost data are needed for cost estimates. Forecast data are manually entered for the procurement planning function. Number of women of reproductive age, rate of population increase, CPR by method, source of supply. Can add commodity cost information, and adjust CYP factors. CPR, projected population data, and number of service delivery sites. Other factors that can be adjusted include commodity cost, discontinuation rates, and CYP factors. The default settings for these latter variables are international estimates which can be replaced with national or local data if available. Demographic data based on the UN Population Division estimates and projections. Additional data requirements include CPR, method mix, source mix, cost per user, proportion in union, duration of postpartum infecundability, sterility and abortion rates. Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools 15 Factor PipeLine CastCost Reality √ FamPlan/Spectrum Assumptions Used for Calculations/ Estimates Based on consumption entered by users, the tool assumes that the data accurately reflects true consumption. CYP factors, estimated prevalence for 5 years in the future, population growth rates, past CPR by method if available. Calculations based on past CPR trends and/or linear progression towards future goal and accounts for method discontinuation. The model uses the proximate determinants of fertility framework to relate TFR, CPR and method mix. User adjustable CYP factors Yes Yes Yes Yes Differentiates male/ female condom prevalence? No Yes, although it focuses on prevalence as reported among women. No No Are projections made using method mix or prevalence by method? Not applicable Prevalence by method Prevalence by method, but can be adapted to maintain/achieve particular method mix. Input data are entered as method mix (sum of all methods = 100%). Total CPR may be entered as a target or derived from TFR. Future projection is for prevalence by method or for method mix Not applicable Prevalence by method. Future prevalence is determined by the user. Past prevalences (when available) are entered to compute prevalence by method and a 5-year estimated projection. Program history (such as past shortages or stockouts) and future program plans can also factor in to the projections. Prevalence by method. Projections are based on the expertise of individual users. A user can examine past trends as a foundation for projecting into the future. Information that can inform goal setting includes: Annual increase or decrease, CPR, Contributing factors, Policy Shifts, Allocation of resources, Family Planning Priorities, Strength of Advocates, Method myths, Campaigns, Stock-outs, New Methods and Other factors. Future prevalence can be an input goal or can be calculated as the level required to achieve another national goal, including TFR, wanted TFR, or reductions in unmet need or expenditure level. Future method mix is input by the user. It can reflect new program initiatives, desired method mix, stated intentions or other factors. Cost sources, adjustability, including user adjustability Users input prices per unit of the product. Freight costs of shipments are estimated by percentages input by users. Unit prices are easily changed. Costs from UNFPA (low and high ends of the range) and USAID are provided by the tool, the user can choose which source to use and can add other sources, and can easily change prices. International commodity costs are readily adjustable. A default set of prices is based on UNFPA, 2005 and USAID, 2007. To capture additional costs, such as shipping or wastage, amounts input could be marked up by a certain percentage, based to local program estimates. FamPlan projects total family planning program costs. It could be used to project just commodity costs by ignoring all other costs. The unit costs are easily changed. Does the tool allow for contraceptive commodity costs to change over time? Yes, it projects future costs for shipments based on prices input by the user, which can be made effective on a specific date to accommodate price-changes over time. Yes, the user can enter estimated future unit costs for any of the methods included. No, it assumes that future unit costs will not change from current unit costs; i.e., a single unit cost is used for all years of the projection. Yes, the unit costs are easily changed and can be specified by year and type of outlet (e.g. public, NGO, private, etc.). Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools 16 Factor PipeLine CastCost Reality √ FamPlan/Spectrum Outputs: Outputs, results, and reports generated. The tool generates graphs showing consumption trends, using historical logistics data, that estimate the future requirements for each product. PipeLine also generates procurement plans – based on any forecast data that the user enters - that include cost information, national stock status reports, and reports showing shipments from various suppliers, and USAID Contraceptive Procurement Tables (CPTs). Produces tables that show the projected number of users and of contraceptive units, and trend graphs of contraceptive prevalence by method, as well as the estimated quantities and costs for all methods together by year (for each of the five years of the projection), or by individual method. CPR, annual number of users, adopters, implant removals, and commodities by method, commodity costs, site monthly case load and CYPs. The output data is presented in both charts and graphs. Excel graphs are simultaneously generated along with the projections. TFR, CPR, number of users and number of acceptors by source and method, total costs by method and source. Also other fertility impacts and vital events. The output data are presented in both charts and graphs. Use, users, and questions answered by outputs/ results/ reports Users of PipeLine can use its outputs estimate future commodity requirements, to monitor supplies, and to communicate supply issues or future gaps in supply to stakeholders. PipeLine can answer: *What quantity of product X will my program likely need next year? *What is the level of stock on hand in a given month for product X? *When should the next shipment be planned (and for what quantity) to maintain a consistent supply? *How much will a shipment cost? *What is the total cost of supplying product X over a year to a specific supplier? *What is the supply shortfall or surplus for a given time period? *How many couple-years of protection does a given method of contraception provide, based on its forecast consumption? These outputs can be used by program planners to test scenarios (different method mixes, prevalences, etc.), by program managers to plan for budgets and commodity storage, by logistics managers to see the quantities that will need to be managed, by budget staff to calculate estimated costs, and by contraceptive security teams to estimate and plan for the required budgets once donations cease. CastCost can answer: *What quantity of each contraceptive will we likely need for each of the next 5 years? *What quantity will we need if the prevalence increases for a certain method? *How much will it cost if we procure them from other sources? Results can be used for advocacy or planning purposes as well as to help set realistic national family planning goals into the future. Reality √ can answer: *How do past prevalence trends affect future trends? *Are goals achievable and what resources would be required to achieve them? *What inputs would be required to achieve a certain level of prevalence? *What will future commodity needs be? *What will be the impact of reducing discontinuation on the numbers of users and commodities needed? *What level of input would be necessary to meet current unmet need? The model is generally used to estimate the service delivery burden and the costs to expand the family planning program to achieve the national goal. The model can be used to explore program alternatives in developing a plan. Alternatives include method mix, source mix, and changes in other proximate determinants of fertility. Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools 17 Factor PipeLine CastCost Reality √ FamPlan/Spectrum Extent at which outputs show: A. Reliable years of projection B. Individual year reports C. Quantities by brand/ formulation or only by method D. Quantities by client source E. Quantities for subpopu- lations within the total projection area A. 1-2 years for procurement purposes. For new or rapidly growing programs, projections should be closely monitored and adjusted. B. Yes C. User determines whether to enter products by brand/formulation or by methods. D. No E. No. Forecasts are made at the program level. Input data that reflect a certain level (whether national, regional, or district) will result in outputs that reflect that level. A. 5 years including current year B. Yes C. By method; could be adjusted for brand. D. Yes E. Yes, provided you have sub- population survey and demographic data. A. Projections are generated for up to 25 years. Projections beyond 5-10 years are less reliable B. Yes C. By method. If CPR information is available by brand/formulation the tool can be used to project future commodity needs/costs. D. Projections can be done for any single sector, but the tool does not simultaneously project for multiple sectors. E. Yes, at any level, provided you have sub-population survey and demographic data. A. The model can project up to 100 years into the future. This is a strategy model, not an operational one. Projections are based on alternative assumptions. B. Yes C. By method only D. Yes E. No Use of the Tool: Training Requirement Users of the tool should be comfortable with a database-driven program that uses a windows interface The tool requires a reasonable degree of sophistication and prior experience (or orientation) to manage efficiently. A half-day to a full-day training is ideal. Advanced users may be able to simply download the tool and use it after reviewing its user’s guide which is available on the CD-Rom with the software or on DELIVER’s web site: http://deliver.jsi.com/dhome/resourc es/tools/softwaretools/pipeline. Basic understanding of logistics concepts will also be important for appropriate use of the tool. Familiarity with Microsoft Excel and with the CastCost user’s guide that accompanies the tool is likely sufficient to use this tool. A shortened tool overview is currently being developed for print and electronic distribution. In the meantime, Susanna Binzen at syb7@cdc.gov or Tim Johnson at jtj2@cdc.gov is available to give a 5- 10 minute overview to new users. Familiarity with Microsoft Excel and with the Reality √ user’s guide that accompanies the tool is likely sufficient to use this tool. The user’s guide is available on-line at http://www.acquireproject.org/archiv e/html/3-program- effectively/tools.html, and on the RESPOND Project web site www.respond-project.org. An animated demo of the tool will be available in early 2010. The tool is best introduced and used in a workshop setting to increase the users’ capacity to understand and use the tool, and to generate relevant questions and scenarios to examine with the tool. Users of the tool should be comfortable with a database-driven program that uses a windows interface. The tool requires a reasonable degree of sophistication and prior experience (or orientation) to manage efficiently. An overview training is ideal to take advantage of the full utility of the tool. There is a three-part online tutorial to assist users enter demographic information at: http://www.healthpolicyinitiative.com /index.cfm?id=demProjE. Advanced users may be able to simply download the tool and use it after reviewing the user’s guide, available online at: http://www.healthpolicyinitiative.com /index.cfm?id=software&get=Spectru m. Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools 18 Factor PipeLine CastCost Reality √ FamPlan/Spectrum Users’ manual provides step by step instructions Yes Yes Yes Yes Instructions for use embedded in the tool Yes. Electronic help files are available by pressing F1, and the status bar across the bottom of the screen provides additional details. Yes No No Previous use/testing of the tool The tool is use in USAID | DELIVER PROJECT offices, as well as at external organizations, ministries of health, and others. It has been subjected to significant peer review and use in a wide variety of situations, at first with family planning products, and now with numerous products, including lab commodities, essential drugs, HIV test kits, anti-retrovirals, anti- malarials, and insecticide-treated bed nets. Has completed testing phase, and has been tested in various programs by logistics advisers. Tested globally both internally and externally. A number of EngenderHealth staff tested the tool before it was used in workshops in Tanzania, Uganda, and Bangladesh. MOH officials (at multiple levels) in Uganda and Bangladesh have used the tool in a workshop setting. Colleagues at USAID and other CAs have reviewed/tested the tool. Has been used in nearly 100 countries around the world and is taught in many universities in the US and overseas. It has been peer reviewed. Feedback Users are generally satisfied with the capabilities, and particularly like its simple interface and reports. Its simplicity and functionality in low- resource situations is one of its main strengths. Users find the tool easy to use and think the information it produces is useful. Users have found it an easy to use tool and have found it useful for examining future family planning scenarios. Users like the ease-of-use, extensive manual and focus on planning to reach national goals. Where, how and with what results/ impact the tool has been used Successfully used in more than fifteen countries around the world to monitor supplies, create supply plans based on a forecast, track commodities, and communicate gaps in supply to stakeholders. The impacts have been better visibility of logistics data to program managers and stakeholders, and more consistent supplies of products through better planning. Used by CDC staff in several countries for forecasting exercises, as well as internally to do analyses that form part of the Reproductive Health Survey dissemination seminars. Used in Yemen by an outside advisor as part of forecasting. Workshops have been conducted in Uganda and Bangladesh and projections using Reality √ have been presented in other countries including Nepal, Ethiopia, Tanzania and Kenya. The projections have frequently resulted in MOH rethinking whether or not future family planning goals were reasonable; the goals set initially by MOH officials were frequently not achievable as they were set very high. In Tanzania, USAID | DELIVER PROJECT staff are working with ACQUIRE staff to use Reality √ for their contraceptive procurement table (CPT) exercise for planning commodity purchases. Earlier versions of the model were first developed in the 1980s. FamPlan has been used in nearly 100 countries to develop RH plans. It has influenced the choice of goals and understanding of effort required to achieve those goals. Getting the Numbers Right: A Guide to USAID-Developed Contraceptive Forecasting Tools 19 Factor PipeLine CastCost Reality √ FamPlan/Spectrum What the tool provides that is not already available PipeLine was specifically designed to be used in situations such as developing country health programs and ministries of health, which may not have the benefit of complete data sets and strong infrastructure. It is simpler and more user-friendly than many other forecasting tools. It is a tool for putting survey data to use for improving program management. Builds on existing national-level family planning projection tools to project CPR and user, commodity, and service needs at the regional, district, and site levels. The tool was designed with the end user in mind, making it easy to generate data for decision making. Consistent approach to estimating feasible RH goals and the effort required to achieve them. Access and Dissemination: Where and how a user or the public can obtain the tool Freely available for download from the USAID | DELIVER PROJECT website. It is also available for free on CD, by mail from the USAID | DELIVER PROJECT. Currently, it has to be requested, but it is free. Soon it will be available for free on the CDC web site and will accompany electronic versions of Reproductive Health Survey reports and data. Freely available for download from EngenderHealth RESPOND Project website and ACQUIRE Project Digital Archive. CD-Rom also available from EngenderHealth and the RESPOND project. It can be downloaded from the Health Policy Initiative’s web site for free. Modifications planned/ ability to modify tool Future updates as recommended by our users (both internal to the project and external) are being considered. The current version, version 4.0, was released in Fall 2007. Users are free to suggest updates to the USAID | DELIVER PROJECT. Final modifications have been made. Minor modifications or improvements are still possible. It is possible that it could be modified in response to user requests. There is consideration for adding an Unmet Need component, and possibly modifying it for use in refugee/crisis situations. Prior to the release of Version 1 modifications were made based on feedback from Tanzania and Uganda. Version 2 (Summer 2010 release date) will include separate calculations of training needs, expand on the cost projections, and include estimates for annual removals for implants. Future revisions to improve the tool, as users and programmers find new and innovative ways of using the data generated, are possible. The tool is in its final form and has been in use for many years. It is modified to respond to changes of value to many users. It is not usually modified for specific applications. Current or planned production, publication, and dissemination of the tool Version 4.0 is being actively used and disseminated. Version 5 will be released in 2010. Once completed, the tool will be translated into Spanish and both versions will be available on the web. It will be disseminated to USAID CA logistics staff. It may be introduced to other staff working on contraceptive security issues, Contraceptive Security Teams, and UNFPA staff. It may be included in a survey capacity building project activity. Training/orientation or technical assistance can be provided to interested groups and individuals. Reality √ Version 1 was officially released in December 2007 and the ACQUIRE Project has published color, hardcopies of the User’s Guide. The tool has been disseminated at numerous conferences. There has been global interest in implementing use of the tool in a number of countries. The RESPOND Project will continue to support Reality √ dissemination and use, as well as future modifications. The model has been available as part of the Spectrum family of models since 1995. It has been downloaded thousands of times. The POLICY Project and HPI have used it in numerous countries and conducted many training sessions. Universities, other CAs and national programs use it without technical assistance.

View the publication