Conveying probabilities and uncertainty is difficult especially if numbers are not used. Word choice matters. Words have different meanings and there is the potential for a disconnect between the sender of messages and the receiver. When words are used to convey probabilities, there is significant room for error.
The current Afghanistan situation may be a perfect example of the problem of conveying assessments in words and deciding based on those possibly ambiguous assessments. We don't know the exact information given or the debates that were held concerning risks in Afghanistan, but if those sitting around a conference table used words like "probably not" or "little chance", it is likely that these had a range of probabilities and a range of understanding.
The ambiguity in word choice was a key finding of the CIA analyst Sherman Kent who first studied words of estimative probability (WEP). He tried to get analysts to be more precise in their language in order to minimize ambiguity. He conducted surveys and developed a mapping of words to likelihood. We have written about this extensively in the context of investment committees. For investments there is not the risk of life with word choices for assessments, but the cost for wrong interpretations is real.
Recent research has replicated the work of Kent and has found similar results. For the new update on this work, see the posting by Wade Fagen-Ulmschneider from the University of Illinois. While finding similarity with Kent's work is good, this research shows that there is still a range of meaning for these words. We present the comparison with Kent's work and the range of probabilities for different word choice. Notice that the tail or extreme events have the highest level of ambiguity.
Someone can leave a committee meeting and have a very different interpretation for the likelihood of a given event. There can still be a high degree of ambiguity. Should investment committee members be given a scorecard on the words that could be used? That may not be a crazy idea. The key point is to ask analysts and committee members to offer precision with their estimates. This is not about the second decimal point but exacting specific information on what is being predicted and offering precision with the chances of an event occurring.
Of course, using probabilities has its own set of issues. What does it really mean to say that there is a 10% probability that a government will fail? Of course, there needs to be a time frame, but what is a 1/10 chance in next year really mean? This is the problem of uncertainty and subjective probabilities. We can say with countable data that there was a 10% probability that rates will rise by 75 bps in six months based on historical data, but that is very different than saying that there is a 20% that tapering will be announced at the September FOMC meeting. Learn to be precise in your assessments and that means defining clearly what is being handicapped.
No comments:
Post a Comment