Psychometric tests and due diligence

The due diligence of hedge funds is often first rate, but the due diligence on manager skill is still mainly focused on only three factors, past performance, pedigree, and references. Given the tight performance across managers, the wide range of manager experiences, and the limited information from references, investors are still looking for better ways of measuring the skills of portfolio managers. The question is determining how to get inside the head of managers and find out if they have what it takes - skill. 

Skill versus luck is not easy to measure when you have limited data. In sports we can follow a lot of statistics with big samples over years. Even in poker, we can measure skill over a large number of hands with a controlled environment.  Trading in a uncertain environment with highly variable rules makes for more difficult testing.

There has been a growing movement for using different forms of testing or psychometric measures for job interviews. Unfortunately, these methods have been wide-ranging and take many different forms that often seem unrelated to the skills necessary to be successful in a particular job. Can you imagine subjecting a hedge fund manager to job skills test? 

Past research suggests that the quality of undergraduate college and/or SAT scores have an important impact on performance. See the old Journal of Finance (1999) study, "Are some mutual fund managers better than others? Cross-Sectional patterns in behavior and performance". While asking for education makes sense, the relative importance of this education declines as the manager gets older.

What has become very popular has been the quirky math question quiz such as, "How many cigar smokers are there in Denmark?". These are interesting for creative thinking, but it is not clear if they correlate with performance. Unfortunately, these questions have been become so popular that there are actually books that describe these interview questions with "solutions". There are skill tests for programmers, but these do not look at the issue of temperament for risk taking.  There are tests which may show you have an aptitude for risk-taking but that does not mean that you know what are the right risks to take.

Something like a Big Five personality trait tests, which are well-defined, can be used to get some insights on the manager, but again it is not clear how they are related to hedge fund success. A good test should be specific to the skill that is to be measured.

Is there a solution? I will go back to what I think is foundational and information, story-telling. If you can explain what you do, how you do it, and what you learned after the fact, you are more likely to have skill. Hitting a baseball requires muscle memory and reflexes. Finding a good trade and managing risks is a slower process that lends itself to careful considered thought that should be measurable.