The illusion of explanatory depth - Confidence is relative

 

There is a cognitive bias that is often undiscussed yet highly relevant to investment management — the illusion of explanatory depth. It is the bias that we think we know more about how things work than we actually do. When asked to explain some process, we are often far from being able to adequately explain a concept. Our confidence drops when we are asked to explain a core concept that requires a degree of explanatory knowledge. Extreme views can be moderated by asking how a system works.

The phrase was coined earlier, but the bias was aptly shown in the experiments by Rebecca Lawson in "The science of cycology: Failures to understand how everyday objects work". When asked to explain how a bicycle works, many were unable to make a simple drawing of its mechanics. 

If you ask about how monetary policy works, how the political system works, or how a firm makes money, many may be unable to give good explanations. This will impact confidence, but will ultimately make better decision-makers.

We bring this issue up again because many news stories make assertions about how markets work, yet when you ask how the system works, many explanations become less certain.

The illusion of explanatory depth - an investment problem

Curiosity solves the information gap between what we know and what we need to know

For information edge you need expertise

 

An investor can generate returns based on their knowledge. The knowledge used creates an information edge that may allow you to earn a higher return than the market portfolio. To determine an information edge, ask what your level of knowledge is about a topic or firm. We focus on tacit knowledge, the expertise from practice and experience. There is ubiquitous tacit knowledge available to most, and specialist tacit knowledge focused on a much smaller set of individuals. 

Ubiquitous tacit knowledge exists in a number of degrees. We can start with beer-mat knowledge, which is a simple description. For example, the Fed controls monetary policy. There is popular knowledge, which you may read in a newspaper, and primary-source knowledge that may come from research analysts and academic articles. These are all fine, but they will not create an edge.

Specialist tacit knowledge can come in two forms: interactional expertise, which shows you can critically evaluate information and use it, and contributory knowledge, which shows mastery and the ability to use that knowledge to develop unique thinking.

If you want to know whether you have an edge, then ask what your level of knowledge is. If you don’t have a specialist’s level of knowledge, you are unlikely to have a chance of beating the market.

There is more than one type of surprise

 Surprise, there is more than one type:

• A bolt from the blue
• An issue is recognized, but the direction of the expectation is wrong 
• You know what will happen, approximately when it will happen, and in what order, but the timing is off
• The expected duration of the event proves wrong 

Need to think about the quality or characteristics of a surprise to better deal with it as an organization. Not all surprises are the same, so the response will be different.

Fimrs need to be HRO - highly resilient organizations

Better risk management requires mindful organizing that creates high-reliability organizations. Mindful organizing develops decision-making processes to protect against high volatility, chaotic markets, and surprises. 

This post synthesizes the key points of Managing the Unexpected: Sustained Performance in a Complex World by Karl Weick and Kathleen Sutcliffe. 

The mindful organization focuses on the firm’s coordination problem. The core problem of any firm, and its reason for existence, is to coordinate activities to increase efficiency and lower costs. Yet coordination is not easy. There is good and bad coordination. According to Barry Turner, “organizations achieve a minimal level of coordination by persuading their decision-makers to agree that they will all neglect the same kinds of consideration when they make decisions.” 

A good organization can adapt to surprises and be resilient. A highly resilient organization (HRO) should act through what the authors classify by the simple acronym FSORE: failure, simplification, operations, resilience, and expertise.

Mindful organizations look for continuity with a specific focus on some key principles:

• A preoccupation with failure - Work hard to detect failures and pay attention to anomalies, information that does not fit the normal; do not normalize; anticipate mistakes or show institutional wariness of mistakes and anomalies; realize that knowledge is incomplete and organize doubt of what may be occurring. There is a tendency to exaggerate the best case and most optimistic view, so be a skeptic. 
• A reluctance to simplify - Clarification does not mean simplifying in a complex world. Accept that complexity is a process and part of the environment. Simplifying tasks or explanations will cause you to miss important details. 
• A sensitivity to operations - Focus on the work itself, what is actually being done, to protect the system and keep the work flowing. Operational breakdowns will be costly. There should be an integration map, so everyone knows how processes are conducted in a failure or crisis. This requires strong situation awareness. 
• A commitment to resilience - Maintain key functions during a crisis or failure. Manage and anticipate the unexpected, so you create resilience, not just expect it. A response to surprises doesn’t happen without training to account for them.
• A deference to expertise - CHAOS - “the chief has arrived on scene”. To avoid chaos, organizations need to defer to experts or those dealing with an issue in real time. 

Adaptive managing from Gary Klein should follow the simple format - STICC:

• The situation - What problem or situation are we facing? 
• The task - What task has to be undertaken?
• The intent - What is the intent of our actions for this task?
• The concerns - What concerns are associated with this problem and solution?
• The calibration - What are we missing or need to adjust given this situation, task, and intent? 

Organizations need to think beyond measuring risk and valuation models and focus on building better organizations.

Stories are important models yet flawed

“Partial knowledge is more often victorious than full knowledge: it conceives things that are simpler than they are and therefore makes its opinion easier to grasp and more persuasive."    - Nietzche 

"History will justify anything. It teaches precisely nothing, for it contains everything and furnishes examples of everything."

- Paul Valery 

Storytellers are model builders. There is a need to describe the causal basis of experiences. A stream of events needs some causal explanation or sense-making that is realistic and comprehensible. A story will use stylized facts to support an argument, but stories are not the same as models. It is a process of causal inference that is not always based on empirical testing from large samples of data. 

Stories are especially important when data are limited or uncertainty is high. In environments of radical uncertainty or complexity, a story or narrative is a useful tool for describing the environment and making sense of causal relationships. 

Organizations use stories to help explain their actions. The story will serve as a model for how an organization behaves. Nevertheless, these stories focus on certain mythical themes: rationality, hierarchy, individual leader significance, and historical efficiency.

The story can be flawed, so investors must question the underlying assumptions behind any story, no different from how model assumptions are reviewed and tested. Stories should be reviewed through a rigorous process to avoid flawed logic based on partial knowledge.  

The two components of investor intelligence

We can think of intelligence in two components. Both are needed to be an effective trader or investor. First, an intelligent person needs to effectively adapt to an environment. Managing the environment requires resources, so an individual must know how to use them. It requires knowledge about the world and the ability to make decisions. Second, people need to interpret experiences within an environment. The skill to provide meaning and learn. Experiential learning links these two forms of intelligence. 

Intelligence is about adaptation, which may involve low intellect, where success is obtained through limited effort and causal understanding, or high intellect, which is associated with a need to understand the causal structure.

This leads to three forms of intellectual adaptation: rules and heuristics. The second is harder because rules may not work, especially in dynamic environments. The second is imitation, where one actor follows another’s success. The third mechanism is selection, which reproduces attributes associated with prior success and eliminates those associated with failure. Yet, controlling or managing the investment environment is difficult.

Success in controlling an environment is limited because history is complex and stochastically uncertain, and outcomes depend on the sequence of choices that lead to a specific action. Success depends on the sample of experiences and the sample of outcomes. 

Intelligent investing is more than applying models; it is an attempt to control a complex environment that requires understanding, action, and learning.

The ambiguities of experience - impacts decision-making

 

James March is one of the great professors of organizational management, yet finance and investment professionals rarely discuss or even consider much of his work. This is odd because organizations, not individuals, make so many investment decisions. Organizations, through investment committees, are a key driver of all institutional action. Organizations manage risk and oversee individual managers.

In his older book, The Ambiguities of Experience, March focuses on the paradox that experience "may be the best teacher" but "is not a particularly good teacher". Organizations are driven by experience, yet the links between action, decision, outcome, and learning are often unclear. The problem is that there are many ways to interpret the link between action and outcome. We have discussed this before in our work on “wicked” environments. 

Experience can be noisy because many links can attach to any action. We cannot always say that our action caused an outcome. Our experiences are also selective. We will often focus on the decisions we made that turned out to be correct and dismiss our incorrect actions. Similarly, success and failure can be misleading. If something worked well, we will likely repeat it. If some action fails, we are likely to stop, yet this stopping rule may not be correct when there are probabilistic outcomes. Additionally, the world changes, so the lessons in the past may not apply, and our aspirations of what is successful also change. There is ambiguity because the environment is complex, noisy, endogenous, constructed, and miserly from small samples.

Experience may not contain the correct lessons for investors or traders. Ambiguity remains about what happened, why it happened, whether it was good or bad, and whether our actions affected the results. Superstitious learning assumes that because some action of ours is followed by a result, the takeaway is that we did that. Is experience unreliable? No, but it should be tempered. 

The key issue for investors and any asset management organization is that causal inference is difficult. The problem permeates all organizations, and inference is not an issue that is left to science. Organizations, like individuals, have behavioral biases tied to group action and decisions. Organizations must explore to learn, then convert that learning into exploitation or action. 

Kahneman's solution to system1 thinking

 

Daniel Kahnemena came up with the idea that people either use system 1 or system 2 decision-making. In system 1, the brain makes impulsive decisions with little analytical thought, leading to behavioral mistakes. In the case of system 2 thinking, the brain slows down and uses analytics and logic to mkae decisions. There is nothing wrong with system 1 thinking when appropriate, and using system 2 for all thinking may be exhausting. Nevertheless, to combat the problems of system 1 thinking, Kahmean uses the SODAS problem-solving framework.

The SODAS acronym stands for situation, options, disadvantages, advantages, and solution. To bypass the automatic System 1 mind, the decision-maker should first define the exact problem faced, then review or brainstorm possible options to handle the situation. Third, the decision-maker should list the disadvantages and then the advantages for each available option. Finally, the decision-maker should select the best solution after the review. The SODAS method. It’s not the first, nor the last, method for making better decisions. It is simple and direct, and anyone can try it. 

Forming better decisions through known, unclear, and presumed framework

 

This book, Thinking in Time: The Uses of History for Decision Makers is now 40 years old, yet it provides some good, simple insights on how to make better decisions. You may not be interested in the political stories provided, but they provide context on how best to use the Neustadt and May framework.

The method is defined as the K-U-P/L-D, or the known, unclear, and presumed, which is then used to compare with likeness and differences. 

Determine what is known, define what is unclear, and then assess what is presumed by the decision maker. The decision-maker should then look for likenesses and differences in history, which can be used to define what is now of concern and what will be new objectives. 

The authors suggest the Gldberg rule: don't ask "what is the problem?" Rather, ask, "What is the story?" by using a timeline from now to the start of the story.

Be a journalist and ask when, what, where, who, how, and why. 

While this is not a quantitative process, it can be useful for framing discussions. 

The Investor, The Chef, and The Recipe Book

 

I have once again discussed with friends what is proprietary regarding strategy details. Should a manager give up details on how they model and generate returns? I understand that in a competitive market, no one wants to give up their edge or reveal how they generate returns. However, investors still need to understand how returns are derived because backtests and past track records offer only limited insight into the return-generating process. There needs to be a framework for understanding how returns are generated. 

I often use the analogy of a cookbook. There are many great cookbooks available in a bookstore. Great chefs write cookbooks and share their “secrets” in their recipes, often with detailed instructions for preparing the meal. Measurements are given. Ingredients are detailed, and step-by-step instructions are often provided for preparing the meal at home. You may follow all the instructions, yet you will often not get anything close to what you experience in the restaurant. You are frustrated and feel the effort was not worth it. So, what do you do? You go to the fine restaurant and pay the price. You are happy and appreciate all of the work that the chef has done to prepare the meal. All are pleased. Information has been provided with high transparency, resulting in a willingness to pay the price for a great meal. There is transparency, which makes paying the meal’s price more worthwhile.

The core of this story is that there is knowledge from books and theory, and then there is practical knowledge from doing. When you invest with a special strategy, the manager is providing their specialized practical knowledge that cannot be easily learned. The devil is in the details, and that is often the secret sauce. I appreciate the practical wisdom on how to gather the right data, manipulate it, and use specialized techniques to extract signals. That is proprietary, but the overall concept should be discussed. Practical knowledge is contextual and situation-specific, and not easily learned just by being told how something is done.

Reducing anxiety choice overload

When there is high uncertainty, there is anxiety in making a choice. Who wants to mkae a bad choice? The issue is then trying to reduce the level of anxiety to allow for better decision-making. Some simple rules can help reduce anxiety. 

Minimize the options - If there are fewer choices, it is easier to make a decision. One way to reduce the options is to categorize the choices. Allow for one option within a grouping. For example, if you are worried about an energy shock, break down the choice between risk-on and risk-off, and within each category, allow for two options.  

Stick to what you know - If there is an option that is not well known to you, then drop it from the choice list. Active decision-making may not be the time to learn about new options. It can be an expensive education. 

Do not regret wrong decisions - The anxiety of choice is based on the regret from making the wrong decision, so another simple way to reduce anxiety is to think about minimizing regret. What is the downside of making a wrong choice? It is important to remember that there are acts of commission, the actions we take, and acts of omission, the actions we don’t take, and regret is greatest for commission. Take the time to ensure that your actions are well-grounded to reduce regret.

Debiasing decisions - Some simple rules to follow

We all face biases or bring them to decision-making, so it is critical to develop strategies to reduce the risk of bias. The bias problem applies to both quantitative and discretionary decisions. Just because you are using a model does ot mean that you are immune to biases. There should be a checklist to review the potential impact of biased thinking. The paper “A user’s guide to debiasing” does a good job of exploring the basics of reducing biases and improving the quality of judgment in decision-making. Debiasing reduces logical inconsistencies and misperceptions or misjudgments of reality. Debiasing is not the same as gathering more factual informaiton. The objective is to improve decisions given a specific set of information.

Categorizing debiasing methods distinguishes between the person and the task. Improving the person requires training to help the decision-maker overcome their limitations. The second approach is to modify the environment to match the thinking required.

Many of the sources of our biases come from the confusion between system 1, fast and automatic responses, and system 2, which requires slower and more deliberate thinking. It is important to distinguish between narrow thinking and shallow thinking. Narrow thinking focuses attention on a single category of objectives and crowds out the ability to identify other alternative objectives. Shallow thinking devotes too little effort to the required task. For any decision, there has to be a level of readiness to perform better decision-making. 

The person can be modified through better education that generates more alternatives, tempers optimism, focuses on improving judgmental accuracy, and assesses uncertainty. Decision-makers can use defaults to get closer to making better decisions through regular processing, nudges to induce reflection through prompts or planned interruptions, and the formation of a set of active choices. If this can be done for the individual, a similar process can be applied to the organization. 

Words have uncertainty - The enemy of precision for investors

 

With the evolution of LLM and natural language processing, there is a closer connection between the discretionary and quantitative world, yet the two are not perfectly linked. There is uncertainty in both worlds. For the quant, there is model and parameter uncertainty. For the discretionary trader or non-quant, the problem is the precision in words. What you say may not be precise by you as the sender and by the receiver. 

We have discussed this issue of precision in language in the past, yet most investors still seem to be at risk from word uncertainty. Just think of all the Twitter words and Substack posts driven by language. Are these words given any quality control? If you look at the research, the answer is no. If you look at the range of meaning for these words of estimative probability, you will have to agree that there is ambiguity concerning the words often used by any decision-maker. Ask for specifics. Ask for the actual probabilities. Close the range of uncertainty.

See "Variability in the interpretation of probability phrases used in Dutch news articles — a risk for miscommunication."

Words of Estimative Probability (WEP) - Words matter and poor precision leads to decision failures

Imprecise language - The big problem with investment analysis

Language, perception, and numbers - The translation problem

Sherman Kent - the godfather of precision in forecasting language

Sherman Kent - My guy for precision in forecasting language

Sherman Kent - The analyst's analyst - Be precise with your forecasts

Sherman Kent and the problem between warner and warnee

Facts, information, and knowledge - Not all the same

 

There are no facts, only interpretations - Friedrich Nietzsche. 

What is a fact? A snippet of information. Something you did not know. A fact is unprocessed information. It does not have any context. Information is processed facts. There is some meaning or context given to the fact. Knowledge will be the applied understanding of that information or fact. Anyone can spout facts with no meaning but facts are usually used in an argument. Facts are used to persuade. Facts, when used as a tool of persuasion, need to be turned into information through knowledge.

Are facts reality? Yes. Facts do not have feelings. Yet knowing facts is not useful without processing and some knowledge; that is where most get into trouble. The disagreement is not with the facts but with the interpretation.

The foundation for decision-making - measurement

“Measure what is measurable, and make measurable what is not so.” - Galileo Galilei

"If you cannot measure it, you cannot manage it"  - Peter Drucker 

The one-two combination for quant decision-making is: first, measure everything because when something is placed on a scale, it can be compared, and second, remember that if it cannot be measured, it cannot be managed. There are no feelings in investing. There is no place for comments like, "I feel rates are going higher..." There is only a place for precision that can be compared with past predictions and future views. Yes, this is hard, but it is the only way to judge the quality of your decisions. 

The many faces of uncertainty

 

From the PowerPoint notes for the book Warren B. Powell, Reinforcement Learning and Stochastic Optimization: A unified framework for sequential decisions, John Wiley and Sons, Hoboken, 2022 (1100 pages).

This is a fascinating book that dives into the complex issues of stochastic optimization. At a high level, I found the description of many types of uncertainty beneficial. Only through dividing or classifying the many kinds of uncertainty can a researcher understand where the key risks are in a problem. 

Is there data uncertainty or model uncertainty? Is the inference uncertainty or transitional uncertainty? If researchers can explore these differences, they can better define the risks faced and the sensitivity to a given answer.

It would be interesting to take a given strategy and decompose all the uncertainties faced. 

Narrative sentiment and attention matters - Read the news!

 

In the paper "The Power of Narrative Attention: Linking Geopolitical and Economic Storylines to Currency Risk and Return Predictability", the authors find that narrative shocks are not fully priced into FX markets and will impact returns over a period of weeks. Even after accounting for major factors, fluctuations in currency markets show time-varying exposure to economic narratives such as recessions, trade wars, and inflation. Narrative sentiment can vary depending on the tone and volume of coverage.

Even if you are a quant, you should read the news and stay aware of current market narratives. However, these narratives can be systematically measured and incorporated into a model to increase explanatory power. We always knew this was the case, but the key finding is that narrative sentiment persists and is independent of other factors.

Be situationally aware of what is being reported in the news, which can often be systematized. 

Jerome Brunner - Logic and narrative go together

 

Jerome Brunner, one of the towering figures in psychology and cognitive learning, developed a simple cognitive theory through framing experience into two models, propositional and narrative. Propositional thinking focuses on logic and formality, while narrative is based on storytelling. Narrative is emotional and needs to be personally convincing. It is the narrative that holds the propositional logic together as a useful tool. Simply put, we cannot develop theory and logic in a vacuum; instead, we use narrative as a tool to support our thinking or convey it through stories. The narrative provides an emotional connection.

We can use that framework to think about how stories are conveyed on Wall Street. Nothing is done by reviewing model results. No talking head refers to a model. The models are condensed into a concise story or narrative. The narrative can provide a connection that is absent from a model. 

Perhaps an extreme, but Fed independence is not presented as a formal problem in time inconsistency, but as a fight for control over a policy lever. The acquisition of a firm is not described in numerical terms, although a price is associated with the purchase. There is a discussion of strategic advantage and how the whole will be greater than the sum of the parts. 

See also:

Narrative and investing - Follow the facts

What makes a good forecaster from Dan Gardner

Dan Gardner,  an essential writer on forecasting, states that there are three key components to being a good forecaster. 

Aggregation - Good forecasters are great aggregators of information and other opinions. They will utilize all available information, even if it contradicts their current views. They will seek out alternatives. They are open to small ideas and not a single unifying thesis. 

Meta-cognition - They are good probabilistic thinkers who also take into account their own biases. They are fully aware of decision bias and reflect on whether they are engaging in these biases. Hence, they never rush to judgment.

Humility -  Good forecasters will admit when they are wrong. Hence, they show a significant amount of humility in their work. They do not have to prove they are right. They can accept error and then try to adjust. 

Use these three components as a checklist to ask whether you are making good forecasts.

The power and gravitation pull of doing nothing in asset management

There is value in doing nothing in asset management. One, doing nothing reduces transaction costs. Two, doing nothing allows for the power of compounding. Three, doing nothing reduces the emotional biases associated with trying to take action. Four, doing nothing helps to clarify the distinction between effort and work. Showing that you are doing something is not the same as doing work. 

By having a long-term view, there should be less trading. Long-term investing is a do-nothing management approach because the long-term decisions should not be swayed by short-term changes in markets. There is more value in doing less. 

These are all good reasons, but there is also a pull to doing nothing that has a negative effect. Nevertheless, no action at the wrong time will be costly. Fear of making mistakes and emotional regret may stop an investor from taking needed action. Aversion to regret will reduce action. You cannot regret the action not taken. You could, but generally, regret is about what was done, not what was missed. Lack of knowledge or ignorance.

So, there needs to be a checklist for action. Do I have a valid reason for action? Do I have the right time perspective? Have I accounted for the cost of trading? Will I regret this decision if the market direction changes?  Be careful with action. From a model perspective, what is the action beyond a prediction of noise.