Building a toolkit for not-knowing

15/1/2024 ☼ not-knowing ☼ risk ☼ uncertainty ☼ strategy

This is the pre-reading for the 13th episode in a series of conversations exploring how to relate better to not-knowing.

A toolkit for thinking and acting on different types of not-knowing should have compartments for diagnostic tools, action tools, capacity-building tools, and update tools. “Tool” can mean anything from a physical object (like a template) to a way of thinking (like a framework) to a way of working (like a process).

The compartments of (and the logic underlying) a toolkit for not-knowing.

Why these compartments?

Because,

Correct diagnosis of the situation is crucial for choosing the right actions for the situation.
Evolving and emergent situations often require updating both diagnoses and the action choices.
And it is hard to do good diagnoses, change actions, and cope with the discomfort of not-knowing, so ongoing capacity-building is essential.

How the different compartments work with each other

How do we choose the tools to put in these compartments?

The choice of tools in each compartment should be guided by a mindset that distinguishes between different types of not-knowing, and the broad approaches appropriate for each type.

Diagnostic tools help their users
1. Avoid accidentally conflating risk with uncertainty,
2. Distinguish between different types of non-risk not-knowing,
3. Interrogate and reveal hidden/implicit assumptions.
Action tools for each type of not-knowing are those which help their users
1. (For not-knowing about actions) construct diverse portfolios of small experiments about possible actions.
2. (For not-knowing about outcomes) construct diverse portfolios of small experiments about possible outcomes.
3. (For causal not-knowing) build experiments for measuring causation or implement ways of “stacking the deck.”
4. (For not-knowing about values) interrogate/negotiate the relative value of outcomes and articulate how goals sit relative to each other in a hierarchy of value (superordination).
5. (For formal risk) rigorously measure and compare well-understood risks and their methods for mitigation.
Capacity-building tools help their users
1. Increase individual/team/whole-organisation comfort with doing uncertainty work.
2. Change team/organisation structures and processes to be better suited to uncertainty work.
Update tools help their users
1. Sense change that affects their assumptions/work.
2. Reduce individual/team/organisational barriers to changing actions and strategies.

Fortunately, we aren’t starting from a blank slate when putting the toolkit together.

Though a toolkit for thinking and acting in not-knowing doesn’t exist yet, many tools for thought and action are available. They tend to get used in a wide range of planning and execution settings — including many settings in which their users don’t (yet) distinguish clearly between risk and non-risk forms of not-knowing.

This is fine! The same tool can be used with different intent to achieve a different outcome. Being clear — as laid out above — about what a tool is needed for and what its use is meant to achieve is the only way to choose and use tools well. (This, more broadly, is the point of the present project on learning how to relate better to not-knowing).

A preliminary set of tools for the kit (a work in progress)

Diagnostic tools that help their users
1. Avoid conflating risk with uncertainty: Implication analysis, forced choice exercises, protocols to make quantification justifications explicit.
2. Distinguish between different types of non-risk not-knowing: Causal loading analysis, structured situation assessments.
3. Interrogate and reveal hidden/implicit assumptions: Question lists, justification protocols.
Action tools for each type of not-knowing that help their users
1. (For not-knowing about actions) construct diverse portfolios of small experiments about possible actions: Portfolio theory, microfunded experiments, small-prize incentives, declarative (not imperative) instructions, explicit processes for superordinating goals.
2. (For not-knowing about outcomes) construct diverse portfolios of small experiments about possible outcomes: Portfolio theory, field studies of nearby and adjacent fields, broad market research, specifying open-ended goals.
3. (For causal not-knowing) build experiments for measuring causation or implement ways of “stacking the deck”: Conventional hypothesis and experiment design, post-market surveillance, citizen science.
4. (For not-knowing about values) interrogate/negotiate the relative value of outcomes and articulate how goals sit relative to each other in a hierarchy of value (superordination): Forcing tradeoffs to be made explicit, Eliciting non-goals as part of goal-setting process, imaginary ethnography, explicit negotiation of value.
5. (For formal risk) rigorously measure and compare well-understood risks and their methods for mitigation.
Capacity-building tools that help their users
1. Increase individual/team/whole-organisation comfort with doing uncertainty work: Desperation by design, use of stretch goals, discomfort training.
2. Change team/organisation structures and processes to be better suited to uncertainty work: Open-ended roles and negotiated joining, systematic large-group normed feedback.
Update tools that help their users
1. Sense change that affects their assumptions/work: Specifying routine review intervals, explicit design of review trigger events.
2. Reduce individual/team/organisational barriers to changing actions and strategies: Resource commitment staging, apoptosis.

Possible contents of a toolkit for not-knowing.

This is not a complete list! Right now, it is an outline of what tools in each compartment would look like (patterns to match to) and a few exemplars of tools which may match the pattern. A better way to think of this is the beginnings of a toolkit, which I’ll gradually add to with links and explanations of how each tool can be used.