Saturday, October 31, 2015

Sherman Kent - the godfather of precision in forecasting language

Estimating is what you do when you do not know.
-Sherman Kent

Most in money management have never heard of Sherman Kent. He is a legend within the intelligence community as the person who shaped thinking of intelligence analysis, the methodical examination of information to make risk assessments, for the CIA. Kent worked for years at developing the CIA's capability to make effective intelligence analysis. 

His thinking could easily be applied to investment analysis. What is surprising is the poor use of language with investment analysis. Research reports are written with words that may not have a deep level of precision. Hence, we can have readers come away with different expectations from the same report. 

This poor level of precision can be even worse with the spoken word. Think of the investment committee meeting where everyone gives their opinion on the direction of rates or the stocks market. Members will use words like "probable," "likely", or "going to happen", but there is little agreement or clarity on what are the probabilities with these words. Opinions are given but they may not be enlightening because here is no precision in the forecasts.

Kent focused on clear recision in analysis. The analysis may not always be correct. These are estimates or forecasts with uncertainty,  but there has to be clarity on what is being said and how it is presented. Kent's piece Words of Estimative Probability is a classic on how precision could be used with language. It was not fully implemented; nevertheless, it provides a framework for better precision in language. It weds the measurable with the verbal. 

Below is the matrix he developed to match probabilities with words and a matrix of similar words that can be used for different levels of probability.  

100% Certainty
The General Area of Possibility
93%give or take about 6%Almost certain
75%give or take about 12%Probable
50%give or take about 10%Chances about even
30%give or take about 10%Probably not
7%give or take about 5%Almost certainly not
0% Impossibility

For example:
Almost certainvirtually certain
all but certain
highly probable
highly likely
odds [or chances] overwhelming
we believe
we estimate
50-50chances about even
chances a little better [or less]
than even
Probably not(14)we believe that . . . not
we estimate that . . . not
we doubt, doubtful
Almost certainly notvirtually impossible
almost impossible
some slight chance
highly doubtful

I would not go as far as what Kent is suggesting for analysis, but it certainly is worth a discussion to see how any qualitative judgment can be made more precise. Quantitative analysts should be forced to provide precision in forecasts and a similar standard should be applied for those using qualitative language.

No comments: