Why is forecasting difficult in commodities - Innovation

Forecasting is difficult for any financial asset, but it can be especially difficult for commodities. There are the peculiarities of futures, the potential for large supply shocks, and just the higher volatility associated with the varied interaction of hedgers and speculators, but there is also something that makes medium-term forecasts especially difficult - innovation. Innovation and technical change will impact production sensitivities in a way that makes past information difficult to use. Innovation changes the sensitivities to supply and demand shocks and because innovation is implemented through time, there is not usually a well-defined implementation date.

Innovation changes the costs of product. When innovation lowers the marginal cost of production, supply will not be reduced until prices fall below the marginal cost. We have seen this issue with the price of oil. The marginal cost of production has fallen through technological change in shale drilling and discovery. Hence, prices had to fall further before a firm's production became unprofitable. The result can be deeper price declines and a cap on price increases.

Innovation can change price behavior to outside shocks. In the case of plant genetics and specialized seeds, production can be less sensitive to drought, insects, or blight. The result is that negative shocks do not have the same adverse impact on prices. Crop yields can continue to rise and show less seasonal sensitivity.

Innovation can increase demand. The processing of foods can create more demand for agricultural products which my be less sensitive to price. In the case of natural gas, improvement in power generation can increase demand.

Innovation in regulations impacts supply and demand. Changes in regulation such as a carbon tax or ethanol usage will impact supply and demand. Innovation on gas mileage driven by regulation has changed demand for gasoline. 

The response by some traders could be, so what. Innovation is long-term phenomenon that should not effect prices of forecast of less than a year. Yet, our reading of fundamental data suggests that innovation can make past history less useful and thus make future forecasts more difficult. This is one reason for the focus price relative to fundamental data for these markets.