Data usage and systematic trading

Data! Data! Data! I can't make bricks without clay.
- Sherlock Holmes

If you torture the data long enough, it will confess.
- Ronald Coarse

Systematic trading has significant value, but there are two problems that have to be thoughtfully addressed, data availability and back-testing. All fit within foundational issue of data-mining. Our first quote from Sherlock focuses on the key requirement of any systematic trader, you need lots of data. In fact, there may never be enough. If you pass through using a few years of data for analysis and model identification, you still need a data set for forward  out-of-sample analysis.  The data problems become even bigger when you employ fundamental information. There are small sample size problems, data revisions, data delays, and model uncertainty given changing relationships. The focus on price data is caused by the need to have more "bricks".

There is also the problem that you cannot run through multiple models over the same data set without data snooping. Searching for the best model all causes in the degrees of freedom available for he researcher. Empirical analysis without theory is torture of data without know the crime. Finally, if the world changes, old data has to be thrown out or at least be discounted, yet cutting sample is the equivalent of losing your memory.

The systematic manager has advantages of the discretionary trader through the use of disciplined decision-making, but training to be disciplined with wrong conclusions for a special case plucked from the data can be more dangerous.