Measuring Family Planning Logistics System Performance in Developing Countries

Publication date: 2008

LOGISTICS BRIEF Measuring Family Planning Logistics System Performance in Developing Countries Effectively measuring supply chain performance is essential for strengthening and improving system performance. Workers unload health commodities at a district store in Bangladesh. As the performance of a health logistics system improves, so does product availability and family planning use. D E LI V E R P ro je ct , 2 00 4 This publication was produced for review by the U.S. Agency for International Development. It was prepared by the USAID | DELIVER PROJECT, Task Order 1. Contraceptive security has been achieved when individuals can choose, obtain, and use quality contraceptives whenever they need them. The family planning (FP) logistics system makes contraceptives available at service delivery points (SDPs) per clients’ need. Therefore, an efficient and well-functioning logistics system is essential for FP programs to be successful. The performance of the logistics system must be regularly measured to improve and strengthen it. But, how can one be certain that the system is being measured? The Logistics System Assessment Tool (LSAT), developed and used by the USAID | DELIVER PROJECT, measures logistics system performance. This study provides insight into whether the LSAT is an effective measure of logistics system performance by developing an index from the items the tool measures and then by testing the reliability and validity of the index. FAMILY PLANNING SYSTEMS AND THE LOGISTICS SYSTEM ASSESSMENT TOOL The FP logistics system comprises the different supply chain management functions that organizations and people perform (figure 1). All of the functions of the logistics system are interdependent. For example, a system in which stakeholders accurately estimate the type and quantity of products that their clients need, but that lacks an efficient inventory control system, is unlikely to consistently and economically meet customer demand. The complexity of the FP logistics system arises when multiple organizations are involved. The various functions of the supply chain are often carried out by different international and national agencies and require external assistance. Figure 1. The logistics cycle shows the different functions of the logistics system that influenced each other. Source: John Snow, Inc./DELIVER 2004. LSAT is a monitoring and evaluation tool that quantifies the functional level of logistics systems for FP programs in less developed countries (JSI/DELIVER 2004). The tool uses a battery of questions (or items) to obtain information from in-depth interviews of FP managers and policymakers. It then scores 11 functional aspects or components of FP logistics system: organization and staffing, logistics management information systems (LMIS), product selection, forecasting, procurement, inventory control, warehousing, distribution, organizational support, product use, and financing. METHODOLOGY The LSAT Index is constructed by aggregating the logistics system component scores to quantify the overall performance of the FP logistics system of a program. Eight of the 11 logistics system component scores are used to construct the FP logistics system performance index, or the LSAT Index. Because of missing values, the component score for product use is not used. The component scores for organization and staffing and for organizational support are also excluded because they do not represent direct logistics activities. The LSAT Index and its component scores range from 0 to 100, with a higher score indicating a better-performing logistics system. Data from 12 countries are used for this study. The reliability and validity of the LSAT Index to measure logistics system performance are determined next. Reliability—whether the LSAT Index score remains consistent over repeated assessments of the same FP program under identical conditions—was measured using split-half, internal consistency, and item analysis (for details, see Karim, Bieze and Chimnani 2008). Next, the LSAT Index is assessed for construct validity using factor analysis. The construct validity is the extent to which the items of the index measure a single attribute (or construct), which is, in this case, the FP logistics system performance. Factor analysis is a statistical technique used to identify how well the component scores are correlated with the latent construct (that is, the FP logistics performance). Finally, the LSAT Index is assessed for predictive validity. The expected outcome of a well- functioning logistics system is commodity availability at the SDPs. Because the LSAT Index measures the level of performance of the FP logistics system, one would expect that a higher score would result in better availability of contraceptives at the SDPs. Accordingly, the correlation between the LSAT Index and contraceptive availability at the SDPs is assessed. Contraceptive commodity availability is assessed from public sector facility surveys conducted within two years of the LSAT assessment. The number of facility surveys varied from 65 to 200. The indicators for contraceptive prevalence included (a) availability of contraceptive method mix (that is, percentage of facilities with oral pills, male condoms, and injectable contraceptives available at the time of the visit) and (b) the average duration of stockouts of contraceptives during the past six months. RESULTS Table 1 presents item scores for the eight components of the FP logistics system in 12 MEASURING FAMILY PLANNING LOGISTICS SYSTEM PERFORMANCE IN DEVELOPING COUNTRIES 2 Table 1. Item scores of the eight components of the FP logistics system in 12 countries. Country LSAT year LMIS Product selection Forecasting Procurement or obtaining of supplies Inventory control Warehousing and storage Transportation and distribution Finance Bangladesh 2005 91.7 — 100.0 100.0 91.7 85.7 100.0 87.5 Bolivia 2003 88.9 — 0.0 0.0 54.5 64.3 0.0 50.0 El Salvador 2003 80.0 57.1 85.7 89.7 84.0 100.0 100.0 37.5 Ghana 2004 60.4 — 94.4 97.2 54.2 82.1 43.8 100.0 2006 100.0 66.7 100.0 95.0 58.3 92.9 91.5 80.0 Honduras 2005 66.1 66.7 42.9 75.0 41.7 86.7 68.8 52.9 Malawi 2002 63.6 — 66.7 100.0 53.8 — — — 2006 66.1 71.4 71.4 80.0 77.1 87.5 93.8 30.0 Mali 2003 45.5 — 50.0 77.8 58.3 70.0 62.5 100.0 2005 76.4 85.7 71.4 95.0 50.0 96.4 75.0 65.0 Nicaragua 2003 95.8 71.4 91.7 100.0 91.7 100.0 50.0 42.9 Nigeria 2005 33.3 — 16.7 33.3 91.7 82.1 25.0 — Paraguay 2005 76.8 20.0 42.9 70.0 75.0 62.5 37.5 45.0 2006 62.3 57.1 35.7 31.6 72.2 60.0 42.9 68.4 Rwanda 2003 0.0 — 33.3 54.1 18.2 66.5 0.0 12.5 2006 92.0 100.0 78.6 97.5 80.5 83.3 46.7 40.0 Uganda 2003 54.5 — 61.1 61.8 28.6 32.0 33.3 75.0 2006 77.5 — 66.7 85.3 53.5 93.9 38.9 90.6 — = not available. countries. The components indicate different areas that need strengthening. For example, in 2003, the forecasting, procurement, and distribution components in the supply chain of the Bolivia FP program were practically nonexistent. Similarly, in El Salvador, the product selection component lags the other components of the supply chain. The reliability assessment of the LSAT Index is conducted using only the latest data from each of the 12 countries. The reliability analysis indicates that the component score for finance is not a reliable measure of logistics performance, which is Finally, the predictive validity of the index is assessed. Simple correlation between the scores and contraceptive commodity availability at public sector FP SDPs is observed. Figure 2 shows a scatter plot between the percentage of the public sector facilities with method mix (condom, pill, and injectable) available on the day of the visit and the LSAT Index score. As expected, the analysis indicates that the countries with a relatively high-performing FP logistics system (indicated by a relatively high LSAT Index score) Figure 2: Correlation between availability of method mix and logistics system performance contrary to the expectation. Hence, the finance component is omitted, which should improve the reliability of the LSAT index. Construct validity of the LSAT Index is assessed using factor analysis. Factor analysis concludes that all seven items of the remaining components are highly correlated with only a single construct Bangladesh 2005 Ghana 2004 Ghana 2006 Honduras 2005 Malawi 2006 Mali 2003 Mali 2005 Nigeria 2002 Rwanda 2006 Uganda 2003 Uganda 2006 0 20 40 60 80 10 0 M et ho d m ix a va ila bl e on th e da y of v is it (% ) (that is, the performance of the FP logistics system). 20 40 60 80 100 FP logistics system performance (LSAT Index score) Correlation coefficient = 0.84 (p=0.001) MEASURING FAMILY PLANNING LOGISTICS SYSTEM PERFORMANCE IN DEVELOPING COUNTRIES 3 Figure 3: Correlation between average stockout duration for oral contraceptives and logistics system performance Ghana 2004 Ghana 2006 Honduras 2005 Malawi 2006 Mali 2005 Nigeria 2002 Rwanda 2006 Uganda 2003 Uganda 2006 0 50 10 0 15 0 20 0 M ea n du ra tio n of s to ck ou t f or p ill s (in d ay s) 40 50 60 70 80 90 FP logistics system performance (LSAT Index score) Correlation coefficient = -0.93 (p<0.001) are associated with higher contraceptive method mix availability. The validity of the LSAT Index to predict product availability is further confirmed by the analysis in figure 3, which shows the relationship between the FP logistics system performance and the average days of stockout for oral pills during the past six months. Countries with relatively better systems performance have a relatively low duration of stockouts for pills. A similar relationship is observed between logistics systems performance and average days of stockouts for condoms and injectables. The assumption that the availability of contraceptive commodities at SDPs is associated with successful FP programs has been confirmed by numerous studies (Jain 1989; Bruce 1990; Magnani et al. 1999; Chen and Guilkey 2003). Therefore, countries with better contraceptive commodity availability should have a better contraceptive prevalence rate (CPR). Because information on contraceptive product availability is limited to public sector condoms, pills, and injectables, contraceptive availability should influence CPR primarily for temporary methods from public sector sources. Comparing contraceptive prevalence data from demographic and health surveys with contraceptive availability data from this study provides additional support for that conclusion (figure 4). Countries with a relatively high percentage of facilities with available contraceptive method mix are associated with a relatively high CPR for pills, condoms, and injectables from public sector sources. CONCLUSIONS The study concluded that a seven-item index, including LMIS, product selection, forecasting, procurement, inventory control, warehousing, and distribution, is valid and reliable for measuring and appropriately predicting the performance of the FP logistics system. As the performance of the health logistics system improves, so does product availability and FP use. The LSAT component scores are useful in monitoring different functions of the supply chain, and the LSAT Index is good for measuring the overall supply chain. The analysis confirms the important role that strengthening supply chain systems and making products available play in the use of health programs and, consequently, the achievement of desired results in such programs. WANT MORE INFORMATION? A full-length copy of this working paper, “Measuring Logistics System Performance in Developing Countries,” is available on the USAID | DELIVER PROJECT website at www.deliver.jsi.com. REFERENCES Bruce, Judith. 1990. “Fundamental Elements of Quality of Care: A Simple Framework.” Studies in Family Planning 21(2): 61–91. MEASURING FAMILY PLANNING LOGISTICS SYSTEM PERFORMANCE IN DEVELOPING COUNTRIES 4 Chen, Susan, and David K. Guilkey. 2003. “Determinants of Contraceptive Method Choice in Rural Tanzania between 1991 and 1999.” Studies in Family Planning 34(4): 263–76. Jain, Anrudh K. 1989. “Fertility Reduction and the Quality of Family Planning Services.” Studies in Family Planning 20(1): 1–16. John Snow, Inc./DELIVER. 2004. Logistics System Assessment Tool (LSAT). Arlington, Va.: John Snow, Inc./DELIVER for the U.S. Agency for International Development. Karim, Ali Mehryar, Briton Bieze, and Jaya Chimnani. 2008. “Measuring Family Planning Logistics System Performance in Developing Countries,” Working paper, USAID | DELIVER PROJECT, Task Order 1, Arlington, Va. Magnani, Robert J., David R. Hotchkiss, Curtis S. Florence, and Leigh Anne Shafer. 1999. “The Impact of Family Planning Supply Environment on Contraceptive Intention and Use in Morocco.” Studies in Family Planning 30(2): 120–32. The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. USAID | DELIVER PROJECT John Snow, Inc. 1616 North Fort Myer Dr., 11th Floor Arlington, VA, 22209 USA Tel: 703-528-7474 Fax: 703-528-7480 Email: askdeliver@jsi.com Internet: deliver.jsi.com

View the publication