Understanding Your Customers: How AI and Market Knowledge Improve Retail Demand Forecasting

When the COVID-19 pandemic hit the world, retail supply chain executives came to the painful realization that traditional forecasting models based on historical sales data were insufficient in predicting sales and demand. As demand for many items fluctuated (dramatically decreasing for some and doubling or tripling for others), the traditional forecasting models that usually capture mostly seasonal fluctuations didn’t help to accurately predict demand. To better anticipate consumer needs, retailers need to shift their focus to more accurate short-term planning and consider the external data that can impact consumer demand. To ensure greater forecasting accuracy, proactive retailers started incorporating external market data like COVID-19 infection rates, mobility indices (Google, Apple), demographics and macroeconomic information, which are possible drivers that explain demand patterns and ultimately improve forecast accuracy.

Improving forecast accuracy through market knowledge

Retail forecasting processes are rapidly changing to incorporate market knowledge through publicly available external data on consumer demographics and global trade, as well as other leading indicators of demand like news, product reviews, and search engine results. This is possible through Machine Learning (ML), an artificial intelligence technique that has the ability to automate model building, learn from data, and identify patterns to create more accurate forecasting. ML with feature engineering is delivering robust forecasts for different demand patterns at varying levels of granularity and consumer channels and can account for:

• Omnichannel demand including forecasts for consumer fulfillment options such as click-and-collect, ship from store, ship from DC, and in-store purchases.
• Slow movers where the volume of sales for items may be too small to generate a robust forecast. Hierarchical ML algorithms can forecast at an aggregate level and then be applied to lower levels like at store/daily levels.
• Day of week variations for food items with a short shelf life and multiple replenishments each week, incorporating this data into a weekly retail forecasting process.
• Intraday forecasting for items that get replenished several times each day, such as in the bakery section. This requires more granular level forecasts which can be by the hour or shift.

Machine learning algorithms allow retailers to obtain detailed results (down to local stores, for example) and bring in local weather events and road conditions, which affect store footfall and consumer purchases.

These next-generation technologies can take leading indicator data and create a forecast view free of human bias or manipulation. At the same time, the technology constantly learns what leading indicator data best predicts changes for a more accurate, granular forecast, right down to the individual store, item, day/hour, and consumer fulfillment option.

Bringing data science teams into the equation

A key competitive advantage for retailers is the ability to efficiently use data to inform their decision making and planning processes, to break down silos within their business, and to better serve customer needs. Many retailer data science teams have developed cutting-edge algorithms in critical areas such as store and omnichannel forecasting, labor capacity planning, assortment optimization, promotion and price modeling, and out-of-stock analysis.

As such, a retailer’s data science division can help drive innovation when their algorithms and models are deployed as production-grade applications. This needs to be done quickly and efficiently without the need for a large software development team. To do so effectively, data scientists need a platform to productize their models that allows iterations safely and securely, as well as scalability to handle retail volumes. The platform should support enhanced cross-functional coordination with role-based access, scenario management, workflows, and flexible reporting capabilities. In addition, it should promote stakeholder participation by enabling the ability to edit inputs and plans, change algorithm parameters, execute algorithms, and combine human and algorithmic decisions to drive what-if scenarios.

Data is the key to building robust demand forecasting and planning capabilities

As consumer purchasing behavior and demand planning strategies continue to evolve, more advanced math with newer AI/ML algorithms, leveraging external market data and providing a platform to operationalize models developed by in-house data science teams, will continue to be key ingredients to driving more robust retail demand forecasts.

-Vikram Murthi, VP of Industry Strategy, o9 Solutions