The hazard of experts - How do you avoid this hazard effect with hedge fund managers?

There is a well know cognitive bias called the Dunning-Kruger effect whereby individuals who perceive themselves as experts will have the illusion of superiority concerning their cognitive abilities. They believe their own talk. They are experts, right? 

Some just call this confidence, but it seems to be a trap for investment managers who cannot admit how much they do not know. It is also a trap for investors when they pick managers who seem very confidence in their abilities to make money. You want confident managers, but how confident? Now, more confident traders are less risk averse, but they may also be prone to bigger errors and are more likely to hold onto losers since they will not admit failure. The confidence of toady could be the big loss of tomorrow.

So what does an investors do when conducting a due diligence on the skill of the manager to determine overconfidence? On the one hand you want to have a expert running your money, but the true expert is more likely to tell you or admit what he does not know, or at the minimum, create a sense of uncertainty and doubt about some of the actions that he will be taking. On the other hand, the manager who is very confidence, in command of many facts and figure, provides strong details on how much he knows, may be a hazard with a blow-up just waiting to happen when there is a market surprise. Where is the middle ground between confidence and doubt?

The extremes are obvious. The manager who does not express any doubt should be avoided, a clear hazard. The manager who raises self-doubt on key questions and questions the meaning of figures could likely still be a beginner. The question is how do you get information on the manager who can combine true knowledge with the awareness of the unknown. This cannot come from just looking at the numbers. A good track record is combination of skill and luck. There has to be a qualitative judgment.

This is where scenario discussion works well. The classic question combination, "Tell me about your best trade and worst trade in the last year" is a good start. If there is not humility and admittance that luck played a role in the best trade, avoid the manager. If the manager does not admit that he missed something in his work on the worst trade, avoid the manager. 

However, the follow-up questions are critical to determine skill and luck. In the case of the bad trade, the self-awareness of what went wrong is critical. What specifically did the manager miss? Have the manager give the details and why it was missed. How is this missed information now being incorporated in the investment process? In the case of the good trade, what made the manager lucky? The good trade could have right for the wrong reasons. The direction could have been right but the magnitude of the impact could have been wrong.

The critical job of the due diligence analyst is to overcome the uncertainty from the Dunning-Kruger effect. This process is especially difficult in global macro trading and model building, so extra care should be taken with these manager reviews.