Performance curves - Still says that simple works relative to complex

 

Machine learning is taking over the modeling process in many fields including asset management. Out with the old modeling and in with the new modeling. With cheap computing, there has been a bias to adding more complexity to models, but over-parameterization does not mean that performance will improve. More is not better. Using extensive testing of different model types (random forests, XGBoost, and deep neural nets) and a wide set of parameters, a recent paper, "The Shape of Performance Curve in Financial Time Series", looks closely at financial time series tests.

The paper's overall conclusion is that there is a flat lining of estimators for mean squared error and accuracy. However, it is always the case that MSE and accuracy are lower during training periods over test periods. Adding more is not always better but rest assured models will do worse outside the training period.

It is always the case that modelers should follow the KISS method, Keep It Sophisticated Simple. Adding complexity is not a substitute for a good simple model.