2021 VAN Scorecard and Narrative

Publication date: 2021

O V ER -A LL Average rating on the pain scale to rate experience using the current systems and processes for collaborative FP supply chain mgmt decisions 5.9 4.4 4.1 3.9 3.2 P E O P LE # members organizations in the cross-organizational Steering Committee that meets regularly to advise on rollout - 6 6 6 8 # of multisectoral task forces (by topic area) identified and formed to advance VAN activities - 4 4 4 4 # entities participating in VAN steering committee and task forces - 16 16 18 18 # countries involved - 2 2 4 34 # manufacturers involved - 4 4 4 4 P R O C E SS EF FI C IE N C Y % of respondents reporting “a lot less” or “less” time spent on triangulating data each month to make collaborative supply chain mgmt decisions for pilot countries and products - 0% 21% 44% 53% % of respondents reporting a “lot less” or “less” time spent on communication to review data and make supply chain decisions together for pilot countries and products - 0% 38% 53% 41% % of respondents indicating that it was “somewhat easy” or “very easy” to access the supply chain reports and data analyses needed for review with the FP community 19% 75% 63% 47% 76% % of “past due” orders that do not have associated shipment records - 1% 6% 5% EF FE C T IV EN ES S % of respondents who “agree” or “strongly agree” that they are able to reliably anticipate expected arrival dates of FP commodities 24% 33% 43% 44% 65% % of respondents who expect in the future that the VAN will allow them to make more timely and specific supply chain recommendations and decisions* regarding Nigeria and Malawi compared to before - 85% 77% 87% 94% % of respondents who report that the data they currently have allow them to make timely and specific supply chain recommendations/decisions regarding Nigeria and Malawi* 41% 0% 44% 39% 87% % of respondents who “agree” or “strongly agree” that the status and progress of the collaborative FP supply planning process are transparent and visible at all times 27% 42% 69% 84% 83% % of targeted countries providing complete supply plans at least once per quarter - 67% 100% 85% % of targeted countries providing complete inventory updates - 95% 95% 90% % of supply plans with a projected MOS below Min alert as of the end of the quarter - - 16% 42% % of Action Request tickets resolved in line with the original request for the year-to-date - 45% 50% 95% SC A LE % respondents who report that it is “likely” or “very likely” that with the VAN processes and systems, they will be able to cover more countries than currently possible without increasing total work hours - 63% 75% 80% 88% % respondents who report that it is “likely” or “very likely” that with the VAN processes and systems, they will be able to cover more products than is currently possible without increasing total work hours - 63% 75% 80% 88% C O ST - 97% 87% 92% 92% CSP Online Tool retired - Y Y Y Y PPMR retired - N N Y RHI retired - N Y Y PPT retired - N N Y Pipeline retired - N N N P O LI C Y # versions of official Terms of Use (effective dates) - 1 1 2 4 # of logins (month of survey) - 342 570 814 # official VAN users (accepted the TOU) - 21 58 124 463 # VAN member organizations (accepted the TOU) - 15 95 T E C H N O LO G Y COUNTRIES WITH SUPPLY PLAN DATA 12 PREV 25 CURR 24 SCOPE COUNTRIES WITH INVENTORY DATA**** 37 PREV 37 CURR SCOPE 46 METHODS** 11 PREV 9 SCOPE 13 CURR PRODUCTS** 33 PREV 45 CURR COUNTRIES WITH ORDER AND SHIPMENT DATA*** 105 PREV 136 CURR SCOPE 136 25 SCOPE Baseline Time 1 Time 2 Time 3 Time 4 VISIBILITYV C COST DM DECISION-MAKING PI PROCESS IMPROVEMENT SC SCALE COUNTRIES SP SCALE PRODUCTS SU SYSTEM USABILITY TIME SAVINGSTS TS TS PI V BC BC BC SC SP % of respondents who “agree” or “strongly agree” that current systems (CSP Online, RHI, PPMR) could be retired if the VAN covered all countries/products BC DM DM DM SU SU NOT LIVE NOT LIVE NOT LIVE NOT LIVE NOT LIVE NOT LIVE NOT LIVE NOT LIVE NOT LIVE NOT LIVE C NOT LIVE Baseline and Time 1, 2, 3, and 4 are defined in the VAN Scorecard Narrative. * Aggregate of the following supply chain decisions/recommendations: adjusting orders and shipment timing to reduce stock imbalances; funding proposals to better align demand with limited resources; adjusting supply plans to avoid future shortages, stockouts, and overstocks; and planning production and shipment schedules. **The count of method and product coverage beyond the total universe has to do with approved requests for product expansion. *** The total universe has been updated to reflect the complete coverage of active countries in the RHI dataset as of 2017. Since 2017, the highest coverage has been 136 countries. The complete set of 136 was transferred to RH Viz, so 136 now represents the complete coverage. **** The total universe has been updated to reflect the coverage of 46 countries proposed in the 2020 business case. VAN Scorecard and Narrative 2021 - 2 - Scorecard progress against efficiency, effectiveness, scale, and cost Overview This section uses existing data to analyze scorecard progress (see 2021 VAN Scorecard above). The scorecard is designed to pull together a snapshot view of key indicators across both the objective key performance indicators (KPIs) measured by the platform and the subjective KPIs measured by the longitudinal evaluation survey, as well as other relevant statistics.1 Four time frames are included in this version of the scorecard: 2018, 2019, 2020, and 2021.2 As noted in previous years, certain KPIs and statistics are grayed out for the “Time 1” column, given that the VAN platform was only launched in live production mode on January 22, 2019. Overall, the inefficiency and ineffectiveness pain felt by those using current processes for collaborative FP supply chain processes has reduced consistently each year, from 5.9 to 3.2 on a 10-point pain scale. This positive trend reflects ever decreasing discomfort in collaborative supply chain management since the introduction of the VAN. Notable feedback from survey respondents includes the following quotes: “The VAN has strengthened in-country stakeholder collaboration and will continue to foster that collaboration as we all work together to improve supply chain systems.” (Control Tower Planner) “Points of contact at MOHs no longer have to spend a lot of time liaising with different stakeholders to obtain a picture of orders and shipments for their respective countries; the VAN has simplified it and democratize[d] access to data for decision-making.” (Control Tower Planning Manager) “Supply plan update[s] are made without hesitation since all members visualize the pros and cons of any shipment.” (Planner3) “It is also exciting to see how the VAN has hit its tipping point in momentum, especially in terms of being able to onboard significant numbers of members.” (Control Tower Administrator) People and Policy In terms of people and policy, the scorecard reflects the successful and wide-ranging scope of community involvement in VAN governance and operations. This past year saw VAN country membership increase by 750 percent (from 4 to 34 countries) through a robust, highly collaborative onboarding effort undertaken between donors, implementing partners, and the VAN Management Unit to migrate the Procurement Planning and Monitoring Report (PPMR) data sharing mechanism to the VAN and roll out free Basic membership to countries. On a smaller, but no less impactful scale, the VAN Steering Committee expanded representation from six to eight member organizations, allowing for the Global Health Supply Chain-Procurement Supply Management and Global Health Supply Chain-Technical Assistance Francophone Task Order projects–key technical partners—to formally engage in VAN governance activities. Closely related to the governance of the VAN are the policies that lay the foundation for its operation, and the individuals who agree to abide by those policies. As in prior years, the VAN updated the TOU in response to member feedback and increasing data security needs. During the span between Time 3 and Time 4, two updates of the TOU were launched (version 3.0 in January 2021 and version 4.0 in July 2021), with each version adjusting elements of the VAN legal framework to ensure robust data security. Membership in the VAN is considered “official” when at least one user from a member organization logs into the platform, accepts the TOU and is trained. Official VAN membership and usage continue to grow, with this year seeing an exponential rise in the number of members and individual users. Between Time 3 and Time 4, the number of member organizations who accept the TOU grew 533 percent—from 15 to 95, respectively. Similarly, the number of individual users rose from 124 in Time 3 to 463 in Time 4, a 273 percent increase. As with country membership, this significant rise in members and users is largely due to the dedicated effort to onboard countries as Basic members. 1 Historically, the longitudinal survey is conducted with a small group of respondents (n=19), querying them on an ongoing basis over time in order to accurately compare trends. However, VAN membership and use has grown exponentially over the years, now covering close to 100 organizations and hundreds of individual users. The VAN Management Unit is exploring options to update the survey methodology so responses better reflect the diversity of membership, user experience, and engagement. 2 The data time frame for Time 1 is December 2018 for the survey and platform KPIs and February 2019 for the other indicators. For Time 2, the data time frame is August 2019 for the survey KPIs and the 4th quarter 2019 for the platform KPIs and other indicators. For Time 3, the data time frame is January 2020 for the survey indicators and the 1st quarter 2020 for the platform and other indicators. For Time 4, the data time frame for the survey indicators is April-May 2021, and October 2021 for the platform and other indicators. 3 Formerly termed Country Program Liaison (CPL). VAN Scorecard and Narrative 2021 - 3 - Efficiency The efficiency trend over time suggests that respondents continue to spend less time on triangulating data for collaborative decision- making for pilot countries and products4, but that there may be room to streamline communications once-again. Fifty-three percent of respondents reported less time triangulating data on a monthly basis (compared to 44 percent at Time 3), while 41 percent spent less time communicating back-and-forth to make decisions (compared to 53 percent at Time 3). While respondents felt communication time increased in Time 4, the efficiency reported for this time period is still positive compared to Times 1 and 2. “Since using the VAN the amount of time needed to triangulate data from multiple sources has significantly decreased. It's expected with continuous improvement that the time spent in that aspect will be minimal.” (Control Tower Planning Manager) Improvements were seen this year in users’ ability to access supply chain reports and data analyses as well as the availability of shipment records for past due orders. In Time 4, 76 percent of respondents felt it was easy to access key reports and analyses to enable collaboration, compared to only 47 percent in Time 3, demonstrating the success of efforts to better onboard users to the location of reports and analyses. Furthermore, 5 percent of past due orders did not have associated shipment records, a decrease from 6 percent in Time 3. “I find the VAN very easy to get into and find what I'm looking for.” (Procurer) With the transition of the PPMR, the multi-year effort to merge previously siloed supply chain management functions and activities (e.g., RHI, CARhs, CSP) under VAN management is complete. We anticipate efficiency metrics will continue to increase as newer users regularly access the platform and become accustomed to the networking and analytical capabilities now available to them. Effectiveness The effectiveness metrics saw some impressive gains in the most recent round, with some indicators holding steady. Notable increases include respondents’ ability to use currently available data for decision-making and the number of action requests resolved in the current year. For Malawi and Nigeria, 87 percent of respondents felt they could make timely and specific decisions using the current data, a 48 percent increase compared to Time 3. This demonstrates the success of ongoing Analyst Pool support to fully onboard and engage users in these countries, and their subsequent comfort applying the data analytics to decision-making. Similarly, impressive growth was seen in the number of action request tickets resolved, with 95 percent of tickets resolved in Time 4 compared to 50 percent in Time 3. This reflects the range of changes that took place across the VAN between Time 3 and Time 4, including expanded expertise within the Analyst Pool, more supply plans included in the platform which increases the accuracy of action requests, streamlining data analysis as the PPMR was transitioned, and, importantly, more countries engaging directly with their data and therefore better utilizing the ticket functions. “[The] holistic picture of the FP commodities (stock-on-hand, average monthly consumption, products ordered, expected shipments, arrival dates, queries/tickets & responses/treatment, etc.) provided by the VAN makes for easier considerations and decision making.” (Planner) “The tickets raised and the information [available] in the platform facilitates decision making when it comes to delay orders. We have delayed some orders and advanced others based on the feedback provided by the tool.” (Procurer) Overall, the effectiveness of the VAN is positive and increasing in some key areas over time, according to the metrics. Since the last round, more respondents felt they were able to anticipate the arrival dates of FP commodities (65 percent in Time 4 compared to 44 percent in Time 3), a strong showing for one of the VAN’s key value propositions. Respondents were also more optimistic about the future, with 94 percent anticipating being able to make more timely and specific recommendations and decisions compared to before. Respondents continued to find positive transparency in VAN supply planning processes (83 percent in Time 4 compared to 84 percent in Time 3). This positive perception will continue to feed into planning for the long-term sustainability of the platform and homing in on the networking and collaborative decision-making capabilities within such a transparent data management space. Some fluctuation was seen in the metrics related to supply plans and inventory, with the percent of targeted countries providing complete supply plans dropping from 100 percent in Time 3 to 85 percent in Time 4, and the percent of targeted countries providing complete inventory updates decreasing from 95 percent in Time 3 to 90 percent in Time 4. The slight shifts in each indicator reflect regular fluctuation in reporting patterns from month-to-month, as well as significant growth in the number of supply plans included in the VAN; with 25 supply plans now in the platform, compared to 12 at Time 3. With such growth, it is not unexpected there would be some growing pains. We anticipate positive trends for these indicators in the future as onboarding continues and strong supportive relationships are established with the Analyst Pool. 4 Malawi and Nigeria, and oral contraceptive pills and implants. VAN Scorecard and Narrative 2021 - 4 - Scale A key component of the VAN Business Case is the belief that efficient, effective, coordinated supply chain processes will improve coordinated supply chain management across more countries, more products, and more actors without additional costs. Bringing this coordination to scale requires buy-in from everyone involved and consistent engagement with the reproductive health community. Over time, respondents are gaining confidence in the VAN’s ability to meet the needs of many, with 88 percent of Time 4 respondents agreeing they can cover more countries and more products without increasing costs when working through VAN systems and processes. The steady increase of these indicators demonstrates the effectiveness of VAN outreach, advocacy and member and community engagement over time. The VAN is a complex endeavor, and it is clear that repeated stakeholder engagement drives buy-in over time. “I expect [the] VAN to increase in its utility as data quality improves and more countries are onboarded.” (Procurer) In terms of scaling up data aggregation to cover more countries and more products, the VAN continues to exceed this goal. The rapid scaling of the VAN platform as a data aggregator is evident in the bar graphs at the bottom of the scorecard. Twenty-five country supply plans are now formatted, mapped, and uploaded to the VAN, exceeding the initial scope. Thirteen contraceptive methods and 45 products are now covered in the VAN, exceeding the agreed scope and growing from the previous scorecard (11 and 33, respectively). Overall, VAN users can now track more than 4,000 orders and 5,700 shipments across 136 countries and 45 products. We expect these bar graphs to keep shifting to the right, maintaining the trend of exceeding expectations. Cost A key value-added benefit of the VAN is the potential to merge multiple contraceptive supply chain management platforms into one globally accessible technology, thereby reducing operating and training costs and increasing efficiencies in decision-making and troubleshooting. In the Time 4 survey, 92 percent of respondents continued to feel that current systems (CSP Online, RHI, PPMR) could be retired should the VAN be able to absorb all countries and products covered under those long-standing platforms. While this number stayed the same from Time 3, the Time 4 data was collected in the midst of the transition of the PPMR to the VAN (January to June 2021). Therefore, we are pleased respondents continued to positively perceive the potential cost savings of the VAN via the migration of previously existing tools and expect future survey responses will show higher positivity given the success of efforts to date to merge these functionalities under the VAN. Another key cost benefit of the VAN is cost saving and cost mitigation related to improved supply chain management. The breadth of data and health system strengthening services accessible to VAN members has fostered more efficient RH supply chain management at global and regional levels. In 2020 alone (infographic), VAN Control Tower analysts worked with countries and procurers to mobilize more than $12 million in new orders and cost savings. Despite downward pressure on global markets for RH supplies, VAN support drove more than $9.9 million in new procurement, which helped 13 countries maintain adequate contraceptive stock levels. In addition, the VAN’s newly formed Consensus Planning Group (CPG) saved around $2 million by canceling or postponing five orders in four countries to prevent waste from overstock and possible expiry. The VAN Control Tower analysts also conducted a joint review of 29 country funding request for 2021 with the UNFPA Supplies Program, which helped mitigate any cases where countries had accidentally made duplicate orders or data entry errors; identify urgent orders to prioritize within available funding; and flag adjustments to order quantities to reduce risk of stockout and risk of expiry due to overstock. Finally, VAN data and services have also proved critical to addressing country procurement gaps and rationalizing the allocation of support depending on need and donor presence. In August 2021, the VAN Control Tower analysts helped UNFPA Supplies get the data it needed to address the implications of UK budget cuts on its ability to meet the supply needs of 54 priority countries. VAN Consensus Planning Group meetings throughout July 2021 drew on the collective strength of RHSC partners, including Avenir Health, the Global Financing Facility, UNFPA, and USAID, to understand and respond to the $92 million gap. The data analyses and recommendations have become the cornerstone of recent conversations to define the way forward in response to the crisis. The analyses bolstered USAID’s decision to procure an additional $17.5 million worth of family planning products to help fill the commodity gap, and they were used by UNFPA to secure a $3 million donation of family planning products from Bayer. The VAN is currently being used to help allocate both the additional USAID procurements and the Bayer donations across countries. https://www.rhsupplies.org/uploads/tx_rhscpublications/CPG_2020_FINAL.pdf 2021_VAN_Scorecard_and_Narrative.pdf 2021.10.25_VAN_Scorecard_and_Narrative_FINAL 2021.10.25_VAN_Scorecard_FINAL.pdf 2021_VAN_Scorecard_Narrative_PDF.pdf Scorecard progress against efficiency, effectiveness, scale, and cost Overview People and Policy Efficiency Effectiveness Scale Cost

View the publication