Confidence and the model builder

There are models that are effective and can explain a meaningful portion of market behavior and there are models that are very suspect and have low explanatory power. Most model have low explanatory power. This is just a the world we live in. High explanatory power for many models in macro finance will only less than twenty percent. Still, a modeler or user of the model can have a varying level of confidence in a model's quality. 

We can think about a trade-off between confidence and quality with model building that defines how portfolio are made. This trade-off in important with setting position sizes and forming a risk management strategy. Low confidence will be matched with smaller risk taking. Low quality should also have smaller positions. It is dangerous if a investors has high confidence in low quality models. Any investor should appreciate model limitations and understand their level of confidence in the models employed.

In the above graph, we can show the trade-off between quality of modeling and confidence in two dimensions, concern for model uncertainty and the legitimacy or quality of the model. High model quality and confidence is the optimal situation for any investors. However, if there is low model quality and high model concern, there is a need for more robustness or checks to avoid uncertainty.  You want to have go out and build more models or search for improvements. High quality models with high concern for uncertainty makes for a conscientious modeler. Low confidence will lead to constant model checking. It is likely that a model in this situation will be build for a focused purpose and limited usage. There will be a constant concern that the model is wrong even if it is of high quality. The combination of low quality but high confidence is associated with investors who feel that a model gives good intuition even if the actual quality is mixed. These investors are more likely to be discretionary decision-makers who are confident in broad principles.  

What the trade-off grid provides is a framework for a model/portfolio quality discussion. In simple terms, is the quality of the model high and do you have confidence in its ability? Understanding this trade-off provides for better portfolio structure. Less confidence creates a perceived need for more diversification. High confidence and quality will lead to more concentrated portfolios. There is no right answer for the confidence quality trade-off other than it defines the type of portfolio that will be structured.