Choices. Choices. What machine learning tool should I use? A recent paper, "Design choices, machine learning, and the cross-section of stock returns", looked at over a thousand machine learning models applied to a single problem and found wide variation in results. You may think you are engaged with systematic investing, but there is a significant amount of discretion when making model design choices. Nevertheless, the idea that you should try and account for nonlinearities is critical in the choice. The key advancement with machine learning is accepting the idea that we live in a non-linear world.
The authors of this study compared algorithms, target variable, target transformation, post-publication treatment, feature selection, training window, and training sample which leads to over a thousand combinations that are tested on a common set of features to analyze out of sample results. The range in performance between the top and bottom monthly returns is greater than 15x. The difference in Sharpe ratios is more than 20x. Design choices matter.
This is a critical piece of research. New techniques have to be benchmarked against the techniques that they are expected to replace. Why choice a new ML model unless you can say something about how it fits within the other choices available.
No comments:
Post a Comment