Three Highly Common Mistakes in Forecasting

Forecasting is often greatly misunderstood and also obfuscated by the allure of the latest technology. In Lord of the Rings, Saruman had the best technology available with the palantir (the “seeing stone”). However, his technology didn’t save him from making serious mistakes or from a bad end.

Forecasting is a good thing. If your lead time to your customer is shorter than the time it takes to build or procure your product, you must forecast and stock material or install capacity in anticipation of demand. However, the less forecasting you have to do, the better. Forecasting means uncertainty in demand and uncertainty means variability. From basic operations science concepts, we know that more variability means more buffers (inventory, response time or capacity) and this typically means more cost. Since all demand is not known ahead of time, forecasting is usually required and is a good skill to develop.

Here are three common forecasting mistakes that companies make.

Forecast error is used incorrectly

Mean Squared Error (“MSE”) is the only error measure to use for  calculating safety stock, projected inventory values and  customer service. Stick with this explanation, there’s some math and science involved—there’s no slogan-based solution. First, an example of inventory mechanics. In a perfect world demand and replenishment time are known and don’t vary. So, ordering inventory is easy. For the example illustrated to the right, replenishment time is 10 days and demand is 2/day. Replenishment time demand is (10)(2) = 20. The company would start its sales campaign with 20 in stock, immediately place an order for 20 and place another order for 20 every 10 days. This would result in the next replenishment order arriving just as the last inventory was sold. There would be minimum on-hand inventory with 100% customer service. However, we live in a world of variability so there is variability in both demand and replenishment time.

Variability means that safety stock is needed. Variance of replenishment time demand must be used to calculate safety stock correctly. If variance of replenishment time demand (“VRTD”) increases, things get worse: inventory increases and customer service decreases. Variance of replenishment time is given as:

Note that specifying variance of demand for future demand is impossible since you don’t know future demand. A couple of options to escape this conundrum:

  1. Use variance of historical demand and a variance to mean ratio (“VMR”). See page 123 in Factory Physics for Managers for more discussion.
  2. Use Mean Squared Error of Forecast (“MSE”) based on past forecasting efforts.

For a., the assumption is that the future VMR is the same as the past VMR. Use of the VMR allows adjustment based on changing demand from historical averages.

Option b. is the most authentic approach as you are planning based on the level of error that you have demonstrated in the past. One strong caveat though: if your MSE is greater than historical variance of demand, you are actually making things worse (more inventory, worse customer service) by forecasting rather than using historical demand variance. Another issue with using MSE is that it’s complex to track and calculate.

The forecast error to use for inventory and service optimization is MSE, not Mean Absolute Deviation, not Mean Absolute Percent Error or others. You may use those other error measures to compare performance or for other purposes but for calculating safety stock and to get predictable inventory and service levels, MSE must be used. The reason is quite simple, the MSE calculation is a statistical variance calculation that is equivalent to the variance of demand calculation used in the VRTD calculation. Mean Absolute Deviation and Mean Absolute Percent Error are not the statistical measures of variance required for the VRTD.

Forecasting is not done at the part level

Do customers buy families of parts? Do companies track and control parts by family? No. A customer may use a family of parts to narrow down to which particular part it wants. For instance, a car buyer doesn’t go to a dealer and say, “Give me a Toyota Camry.” The buyer will specify an option package and color. Companies may report activity by families of parts but each part has to be ordered stocked and tracked individually.

Companies often make the mistake of forecasting and tracking forecast error only at the product family level. This makes forecast accuracy results look better because invariably forecast accuracy at the family level is better than at the part level—this is called pooling variability. However, it’s simultaneously a receipt for inventory optimization failure and poor customer service. Smearing policies like peanut butter across large groups of parts is certainly sub-optimal—but it’s easy. This is even more curious since today’s computing power makes it easy to handle large amounts of data.

It’s good to manage parts in large groups once each individual part’s demand and replenishment time statistics have been analyzed. For instance, put high volume, low cost items on automated tracking and ordering control. One planner can manage thousands of these parts. However, each part’s policy should be determined by its individual replenishment time and demand profile.

Forecast error is not tracked at part lead time

It does no good to track the forecast error at 3 weeks for a part that has a 6-week lead time. Often, parts are put in groups irrespective of lead times and the forecast error is tracked at a specific horizon. Calculating forecast error at part lead time makes tracking forecast error more complex but tracking error at part lead time corresponds to the real-world behavior you are trying to control. Again, computing power today should make this relatively easy but many software packages use the forecast horizon approach instead.

How does your company use forecasting effectively?

Ed Pound is Chief Operations Officer of Factory Physics Inc. Ed has worked with major international companies such as Intel, ABB and Baxter Healthcare providing education and consulting in the practical operations science of Factory Physics concepts. Ed’s work has helped companies realize millions of dollars in improvements and make operations and supply chain management easier. Ed is lead author, along with Dr. Mark Spearman and Jeff Bell, of McGraw-Hill’s lead business title Factory Physics for Managers.

For more information on operations science, CSUITE Operations Analytics and Factory Physics services, send an email to Ed at espound@factoryphysics.com

    Comments are closed