Welcome to the third recurring decision-making workshop. Making good decisions is synonymous with being effective in the world. The art of accurate decision-making is at the intersection of epistemic rationality (understanding the world) and instrumental rationality (taking action).
This week you will begin with some review and exercises, and then move on to a Bayes problem in your own life.
Bayes Exercises [55 minutes]
Bayes Theorem Introduction [15 minutes]
We have, until now, treated probabilities as subjective assessments made by the decision-maker and used to make better decisions in uncertain situations. We have not addressed how to treat your probabilities when new information is discovered. What do you do when you gather new evidence that has some bearing on your decision process? You should update your prior probability to better reflect the reality that you observe. The rule by which you should nudge your previously held probability after a relevant observation is described by Bayes' Theorem.
You will often see Bayes' Theorem expressed this way:
P(X) means "the probability of X", and | means "given that." You will also need to know ¬ means "not".
Bayes' Theorem can be used for situations involving many possible outcomes and even continuous variables, but for this exercise we will focus on simple binary problems. The most succinct derivation of Bayes' Theorem I have ever come across, applying to a binary situation, is contained in this image:
(https://creativecommons.org/licenses/by-sa/4.0/)
Take at least a minute to examine the image quietly before moving on.
The resulting equation can be read as: "The probability that the subject is an assassin given that the subject is being suspicious, equals the probability of a crew member being suspicious given that they are an assassin, times the probability of any crew member being an assassin, divided by the probability of being suspicious."
For reference, refer to this version of Bayes' theorem, which is more directly helpful in situations where B is a binary variable:
The only difference between this form and the previous one is that P(B) has been expanded and made explicit. This simply makes it easier to calculate. We will now work through an exercise that will help you internalize how all this works.
The Mystery Coin [40 minutes]
Solution Approach [5 minutes]
Before moving to the spreadsheet, spend 5 minutes by the clock discussing a solution approach. Move on when 5 minutes are up.
Definitions
Start with the tab Definitions. Your cohort will collaboratively fill in all the definitions that are grayed out, given the provided starting definitions as examples. Split up the task however you like.
Check your answers with the answer key when you're finished with this step, and if there are any serious disagreements, discuss them until satisfied that you understand. Minor differences in wording are fine.
Starting Probabilities
Move to the tab Starting Probabilities and collaboratively fill in the grayed out cells. Again, one answer is provided to get you started.
Check the answer key after you have come to an agreement, or if agreement cannot be reached.
Problem Setup (1)
Proceed to the sheet Problem Setup (1). Fill in the proper equation for the two blue cells (F29 and G29). F29 reflects the odds that it's a trick coin given that heads has been observed. G29 reflects the odds that it's a trick coin given that tails has been observed. Try to come to a consensus.
If you reach consensus, or after ten minutes, move on to Problem Setup (2) and check the answer.
Problem Setup (2)
Check the reference of cell D30 in Problem Setup (2). The value for P(A) at the start of the second flip equals P(A|B) after the first flip. Why? Because heads was observed, as indicated in the orange cell. If tails had been observed, then P(A) after the first flip would be 0.25.
How would the rest of the sheet be extended? You may attempt to do this on your own, or discuss the approach as a group.
Dynamic Sheet
For the remainder of the 40 minutes allotted to this section, examine the final sheet, Dynamic Sheet, an interactive worksheet that simulates the problem dynamically. You can change the red cell from Trick to Fair to change the behavior of the sheet. The sheet uses the RAND() function, meaning every time you change something in the sheet, the values will update. Try to come up with a collaborative answer for how many coin flips it would take to feel relatively certain that the coin is either fair, or a trick coin. There is no firm answer to this, but helpful questions to ask might be:
- How many flips does it usually take to get to 90% certainty, one way or the other?
- How often does the result look like it's going one way, and then change directions?
Break [5 minutes]
Take a five-minute break. Stand up from your computer, stretch, get some water.
Bayes Problem Workshop [20-30 minutes]
Note. The tasks below should be performed sequentially for each member of the cohort. For example, your first cohort member should complete Personal Situation and then Shared Situation. Then your second cohort member completes both, and so on.
Personal Situation [15 minutes]
This week, we will not be working on a personal decision tree. Instead, each cohort member will think of a recent situation to which Bayes Theorem could potentially be applied. Write down Bayes Theorem for the situation, defining all the variables appropriately. Write down your initial probability estimates and calculate how the update should be performed.
Shared Situation [15 minutes]
Return to the cohort session and share the Bayes problem from the previous task. Describe your thought process in choosing your variable definitions and your probabilities, and show your result. The group should provide feedback, especially if something seems to be a mistake.
Wrap Up
If there's time remaining, return to the Mystery Coin sheet and experiment with the values in the green cells, exploring how the dynamics of the problem change as the assumptions are altered.
If you want to become truly comfortable manipulating Bayes' Theorem, try rebuilding the spreadsheet from scratch.