Every model will have signal noise or uncertainty, and every model will have imprecise impact uncertainty from trading. This is a certainty because any training set will have to be imperfect. Hence, we should expect performance problems due to parameter uncertainty. Modelers should take this into account in their work.
First, the model’s Sharpe ratio will be lower than expected from what is generated from the training set. There will be estimation error and model misspecification. Second, the model will affect transaction costs, with the impact being greater for smaller and less liquid stocks. Notably, weak signals will have less impact on trading costs, while strong signals will have a greater impact on trading costs. See “Trading with Uncertainty about signals and price impact”.
How do you solve these problems? One way to solve the problem is to define a reference model and then assess the costs associated with variations from it. The uncertainty can be managed to lock in a reasonable Sharpe around the reference level. The math in this paper is not easy, but the key is to be aware of the problem and to place bounds on what is possible.






