We cannot hope to navigate complex decisions unless we first understand how to correctly analyze an event, and to have reasonably well-calibrated expectations about the future. This week our goal is to learn the tools and methods for improving our individual prediction accuracy, and to practice those skills.
Instructional Material
Note. The YouTube videos in this list are slightly out of date, but the transcripts have been edited to include feedback from the first version of the Decision-Making course. You don't need to do any of the exercises or assignments in the YouTube videos.
- Thinking in Buckets explains the heuristic of breaking any prediction down into its possible outcomes.
- Probabilities and Distinctions discusses the rules of constructing probability trees and using them to solve confusing problems.
- Prediction and Calibration will convey the best practices for making complex and confusing predictions under uncertainty
Pre-Meeting Assignment
Metaculus
If you don't already have one, make an account on Metaculus, and make one prediction every day this week.
Whether you end up with a high score is not important for now, the habit of making the predictions is meant to get you thinking in terms of probabilities and distinct outcomes. You are encouraged to collaborate with your cohort on these predictions. Pay attention to the ways in which the Metaculus questions have articulated the possible outcomes.
Fermi Estimate
Using Getguesstimate.com, make a guess at the weight of the heaviest recorded crocodile. Follow my workflow in the Prediction and Calibration lecture.
Optional Extra Assignments
- At least once this week, try to resolve or clarify an argument by using a prediction or bet.
- Play the Metaculus Calibration Game.
- This week, get into the habit of making friendly bets with other Guild members.
Cohort Activity
Wisdom of the Crowd [15 minutes]
Collect everyone’s estimate for the weight of the largest crocodile ever recorded, and discuss Fermi estimates in general. Encourage everyone to describe their approach to the crocodile question. Take the average of all your estimates, and bring your estimate back to the main session; the true answer will be revealed, and the cohort with the closest mean estimate is victorious. "Winning" strategies or mental models should be shared by the winning cohort in the general session.
Metaculus [15 minutes]
Lead a discussion on Metaculus. Ask the students which questions they made predictions on recently, and how they approached the process. Any particularly good or particularly flawed/bad Metaculus questions should be shared, and brought to the general session for discussion. (What is "good" or "bad" about them?)