Friday, March 24, 2023

An ensemble of models is not the same as an ensemble of experts

 


There is the idea that there is a power with group decision-making - an ensemble of thinking will improve predictions. In reality, group decisions and dynamics are difficult and not always productive. An ensemble of experts in a committee is not the same as an ensemble of models. When people refer to the wisdom of crowds they are often thinking of the collective wisdom from averaging opinions and not their collective deliberations. 

An ensemble of models will usually be uncorrelated by construction. These correlations may change with market conditions, but they will not change because a set of models win an argument on which is better. These predictions can be averaged but we will know their level of independence. 

An ensemble of models can be tested backed through time. An ensemble of experts cannot usually be tested or stressed to find how it will react to new situations.

Models are independent of each other. The crowd is not independent. The crowd can be subject to herding or information cascades.  Group dynamics can move to extremes in ways not possible with a collection of models. 

Collective behavior in models will be good especially in periods of uncertainty. The collective behavior of a group may be good, but it is not a given. 



No comments: