What separates the good managers from the average managers in the systematic space. Of course, it is the quality of models, but there are a number of ways of determining whether a model is good or bad. Most important, is being able to understand when a model will do well and why. If you cannot say why a model works, it is likely to break. There are a simple set of questions that should be asked:
The historical statistics usually associated with model performance may not provide the answer. Assessing the stats is just looking at what is above the surface. You really want to get below the surface and understand the "why" behind a model. Do you need volatility? Do you need long trends? Does it make money on market reversals? What is a reversal? A trend model tested during trending periods will do well, so what happens if there are no trends for an extended period.
You do not have to predict the future, but you do have to know the comfort zone of a model.
- What is the model trying to do? What type of risk is it trying to capture?
- When should the model do well? Is there a good time to use this model?
- What was the environment when it was tested? Given to the two answers above, what was the testing environment? Did you have the perfect conditions for this model?
- What is the chance of success? How often does this good environment arise?
The historical statistics usually associated with model performance may not provide the answer. Assessing the stats is just looking at what is above the surface. You really want to get below the surface and understand the "why" behind a model. Do you need volatility? Do you need long trends? Does it make money on market reversals? What is a reversal? A trend model tested during trending periods will do well, so what happens if there are no trends for an extended period.
You do not have to predict the future, but you do have to know the comfort zone of a model.
No comments:
Post a Comment