Importance of the Right Data in the Demand Planning Process
The operational side on the intelligent enterprise is driven by Sales and Operations planning and in turn S&OP is driven by an accurate demand forecast. While this is simply stated, it can be very challenging to consistently produce good forecasting results. At the core of the challenge lies an important first step of harnessing the right data. Many forecasting and planning projects are abandoned or considered failures due in part to data issues. The right data input to the forecasting and planning process has several important dimensions that must be considered.
What type of historical data should be used? Almost all supply chain forecasting systems use some form of statistical forecasting models driven off of historical data. Many times the tendency is to use product shipment data to predict the future since it is most readily available and perhaps the best understood by operations people driving the process. Unfortunately, product shipment data contains several undesirable components like incomplete or partial order fills, delivery delays, and product substitutions – all of which represent supply chain inefficiencies. Shipment data represents how operations responded to customer demand, not the customer demand itself. Operational systems must build plans off a demand forecast and use shipment data as a measure of effectiveness in meeting those plans. Product order data less any customer returns is the best representation of customer product demand and in most cases should be the historical input for the statistical forecasting process.
How much data should be used? For supply chain forecasting systems the correct answer is perhaps the most surprising to new planners. An accurate, statistically-generated forecast has several elements – seasonality, level and trend. There must be sufficient previous cycles in order to extract all of these elements from the historical data to produce a forecast for future periods. In most cases this means a minimum of 2 past years of historical data and ideally 3 or more would be best. Most operational-driven forecasting systems use monthly summaries of product demand separated either by the manufacturing source or the distribution point. Finally, putting the statistical requirements aside, historical data that does not represent the enterprise today should be eliminated such as premerger or acquisition data.
Does the data need to be cleaned? In nearly all cases data cleansing and transformation is required to facilitate the demand forecasting process. Without the aid of cleansing tools the process can be both time consuming and impractical. The various tools employed are driven off rules created by the implementation teams. Often, these rules get developed using estimates and experimental tests to assess the results of a cleansed data forecast with care taken not to alter the important characteristics of the base input data. The cleansing process is usually completed for all historical data during the implementation process, but the rules developed must be applied in real time as historical data is generated over time. Beyond cleansing, a transformation process often must occur to normalize the input data for units of measure or changes in product sourcing locations in the future. Additionally, for new products that are merely product line extensions, a transformation process can be used to fabricate historical data that will provide the necessary inputs for the statistical forecasting engine.
How to deal with anomalies or events in the data? Data cleansing normally is employed to remove inaccuracies or noise from the input to the forecasting system. Events such as promotions or catastrophes are described as real data and should be handled differently. By finding and tagging these events during the forecasting process they can be used to predict the impact of future occurrences. These events can also be used to assess the lift or cannibalization of a promotion that occurred in the past, helping to plan future promotions. Many times these types of data anomalies are called outliers and sophisticated forecasting systems can detect outliers and correct their impact on future data.
Harnessing the right data for the demand planning process always appears to be straight forward and relatively simple. However, bad data often is the real reason behind a demand planning project failure.