Wednesday, March 13, 2024

The information coefficient - Tell me model prediction skill


Measuring trader skill is critical to picking the right manager and just looking at the Sharpe ratio does not clearly answer the question. Of course, skill analysis requires a deep dive into decision making, but a first step is understanding the hit rate. 

The information coefficient (IC) does that:

IC = (2 x proportion correct) -1 where the proportion correct is the number of correct predictions versus the total predictions.

If the proportion correct is 50%, the IC is zero. if the proportion correct is 100%, then the IC is 1. You want to have a high IC, but it is important that a trader has a large sample of predictions from which to calculate the IC.  I want a high IC because it tells me, I am getting the direction right.  A good trader may have an IC above .05. 

The information ratio (IR) is the excess return relative to the amount of risk taken. Note that I can have a high information ratio on few predictions or few correct predictions. Good to see both. 

The active law of fundamental investing ties both of these concepts together. The expected return is equal to the IC times the square root of breadth (number of bets) times the standard deviation or risk taken. This means that the IR is just the IC times sqrt(breadth). There is added a transfer coefficient associated with the structure of the portfolio. It is defined as the constrained versus unconstrained active portfolio. That number in the simple case is equal to 1. 

For a quant model. tell the number of bets you take and tell me the accuracy of predictions from your model. The IC is very informative on model quality.  


No comments: