speaking writing consulting book tool not-knowing

Not-knowing discussion #8: What results are worth (summary)

7/9/2023 ☼ not-knowing ☼ ii ☼ iisummary

This is a summary of the eighth session in the InterIntellect series on not-knowing, which happened on 17 August 2023, 2000-2200 CET.

Episode 8: What results are worthEpisode 8: What results are worth

Upcoming: “The fog of time,” 21 Sep 2023, 2000-2200 CET. The ninth episode in my Interintellect series about not-knowing is about how the length of the time horizon affects not-knowing by allowing us to learn about new actions and new outcomes, change norms about value over time, and respond to changes in the scarcity of resources. More information and tickets here. As usual, get in touch if you want to come but the $15 ticket price isn’t doable — I can sort you out. And here are some backgrounders on not-knowing from previous episodes.

What results are worth

Reading: Not-knowing about actions and outcomes.

Participants: Carmen N., Paul M., Ines A., Alyson S., Indy N., Chris B., Jim O.

We usually assume that the values (= worthiness) of outcomes are stable, known in advance, and agreed among stakeholders. Turns out, this isn’t always true. We talked about five ways we could not-know about the value of outcomes — and found at least two more. Not-knowing about value makes it hard to compare desired outcomes and to choose actions. But not-knowing about values also creates space for new ideas of what should be valuable and new stakeholders whose ideas of value should be taken into account.

Discussion highlights

  1. Seven distinct ways of not-knowing about the value of X. Five ways were previously described, and two new ways came up during the discussion.
    1. Different estimations of X’s objective value.
    2. Different opinions of X’s subjective value. 
    3. Different opinions about un/acceptable tradeoffs in obtaining X. 
    4. New: Different publicly vs privately expressed valuations for X.
    5. Changes to relative scarcity of X.
    6. Changes to norms about X.
    7. New: Changes in individual preferences for X (distinct from norms about X).
  2. Intentional actions necessarily embed valuation of particular outcomes. All actions that are taken with the intention of achieving specific outcomes — intentional actions — contain explicit (or sometimes implicit) valuations for those outcomes. Examples include saving money with the intent of using it to buy a nice car or majoring in finance with the intent of becoming an investment banker. Intentional actions reveal valuations of outcomes. (This is also the basis of revealed preference theory.)
  3. Value can also be embedded in things that are not human: Reactions, systems, processes, objects that don’t have conventional intentionality. Technology therefore embeds value, though this may not be immediately evident — this may be connected to ideas of panpsychism and animism.
    1. Instinctive reactions can capture evolved or previously learned value systems. Examples include fight/flight responses to threat or anger responses to being told you’re not doing a good job at a performance review. Evolved/learned responses that embed value can be adaptive or maladaptive. Value systems embedded in evolved/learned responses may not be accessible in the moment of the instinctive response, but may be changeable or retrainable over time.
    2. Systems that dominantly lead to particular outcomes embed valorisation of those outcomes. Examples include HR systems that valorise extraversion by focusing performance reviews on evidence of group participation and urban design that valorises car use by emphasising very low-density housing.
    3. Algorithms inherently embed valorisation of particular processing methods and data being processed. Examples include facial recognition algorithms that emphasise high-contrast-feature extraction and recommendation algorithms that emphasise engagement frequency.
    4. Tools can embed valorisation of particular deployments by being designed to provide particular affordances that encourage (or discourage) particular modes of use. Examples include carbon steel knives which are fragile in ways that promote more careful use or phones that are designed to prevent doomscrolling.
  4. Patterns of acceptable and unacceptable tradeoffs reveal differences between individuals in their subjective valuations of the same outcome X.
    1. Even-over statements for identifying relative priorities via tradeoffs.
    2. Boris process for articulating and converging on shared tradeoff configurations.
    3. Tradeoffs may be (?) connected to Cynefin constraints and anti-goals.
  5. Not-knowing about value also emerges from the interaction between individual valuation and social valuation for a particular outcome.
    1. Interaction with others changes how individuals value X (via persuasion, emulation, imitation, new information etc).
    2. Individuals predict what other people value and how much they value it. They also differentially weight the valuations of others, and present their own valuations differently for strategic reasons.
    3. Interaction between individual and social valuation is affected by the context of the interaction (whether adversarial or cooperative), the power dynamics of the interaction, and the institutional frameworks within which the interaction happens.
    4. An example: A proponent of fracking changes their mind after moving to a town where there is a fracking operation and meeting people who have had their groundwater contaminated as a result of fracking [1]. The arguments against fracking provided by one family are especially compelling because their son has developed a medical disorder that seems closely tied to the contamination [2]. The town’s residents bring a lawsuit against the fracking company; the court interprets the law on duty of care in favor of the town’s residents and shuts down the fracking operation — thus valuing the health of the residents over the expected value of the oil that would have been extracted by fracking [3].
  6. Knowledge about the valuation of X is almost always imperfect, but for different reasons. If X has an objective, “true” value (e.g., X is the number of marbles in a large jar), imperfect knowledge is the result of estimation error. If X has only a subjective value (e.g., the value in dollars of the psychic harm suffered by a fast food restaurant customer due to a scalding injury caused by an incorrectly fastened lid on a hot beverage container), imperfect knowledge is the result of inconclusive determination of the relevant system of valuation.
  7. Different types of value are susceptible to not-knowing at different times and in different ways. This is analogous to pace layering for not-knowing about value.
  8. Preferences obviously affect valuation, and preferences change over time (both individually and collectively).
    1. Some factors which modify preferences over time include detecting value decay, path dependency, and evolution.
    2. Third-party value discovery. Cool-hunters identify and endorse things which may then increase in value through social mechanisms. Uncool-hunters do the inverse, and harbingers of failure may reveal underlying uncoolness/lack of social valuation by what they choose to individually value.
  9. Social mechanisms of valuation depend on power dynamics between constituencies (i.e., those with power get to influence what is valued). The emergence/formation of new constituencies creates new systems of valuation. Examples include unborn children and minors being deemed to have standing in lawsuits about the consequences of current actions with long term environmental impact, natural features or ecosystems being deemed to have rights and attendant protections, and minority recognition and attendant rights.

Fragmentary ideas/questions that came up which seem valuable

  1. If social valuation depends on power dynamics, where should we attack existing structures of power to create changes in how/what gets valued? Related to the question of the Overton window of acceptable policies at any given time, and how to shift it/change its size using some or all of the sources of not-knowing about value.
  2. What forms of expression create conditions for changing ideas of value? Examples include the emergence of the epistolary form of the novel (which extended the idea of what kinds of individual get to have feelings, and thus who should be taken into account) or laws that grant entity status to new types of entities (such as corporate personhood).
  3. If there are non-commensurable systems of value, how can they be reconciled with each other? Examples include businesses that are trying to optimise for multiple bottom lines at the same time (e.g. B Corporations like Patagonia or purpose-driven cooperative corporations like Mondragon) or governments that have a duty to multiple constituencies with different value systems.
  1. Pace layering: An approach to thinking about action in complex, loosely interdependent systems.
  2. Boris: A process for converging on relative value by talking about tradeoffs.
  3. Espoused theory vs theory-in-use. 
  4. “Even Over” statements: An approach for articulating relative value of outcomes.
  5. Moral Machine: A platform for gathering human perspectives on decisions about value that are made by machines.
  6. Hoping for A to Z but rewarding only A: Issues that arise when organizations have multiple valued outcomes but incentivise only some of them.
  7. B Corporations: Organisations that pursue multiple valued outcomes simultaneously.
  8. On justification: Theorising about multiple systems for understanding value.
  9. Commensuration as a social process: On the social processes involved in comparing different things using the same metric.