Causality—Bayes' Theorem
Understand Bayes' theorem through the engaging example, and learn how to update the beliefs of a model with new evidence.
In this lesson, our objective is to get familiar with one of the most crucial formulas in the realm of probability: Bayes' theorem. This powerful theorem is fundamental to scientific advancements, serves as a key instrument in the field of artificial intelligence, and has even been employed in treasure hunting.
For instance, in the 1980s, Tommy Thompson used Bayesian search techniques to locate the shipwreck of the SS Central America from a century and a half prior, laden with gold now valued at approximately $700,000,000. You can read the full history in Ship of Gold in the Deep Blue Sea by Gary Kinder, which provides a detailed account of the treasure-hunting adventure.
Bayes' theorem is an essential concept that is well worth mastering.
Judgment under uncertainty
We explore the fascinating world of Bayes' theorem, starting with an intriguing example drawn from the study conducted by psychologists
The story goes like this: Steve is a man who is very shy and withdrawn, with very little interest in people or the world of reality. He requires order and structure and a passion for detail. Now, consider this question: Which of the following descriptions do you find more likely, "Steve is a librarian" or "Steve is a farmer"?
Kahneman and Tversky's study found that most people, after hearing the description of Steve, were more likely to say he is a librarian than a farmer. This was because Steve's traits seemed to align better with the stereotypical view of a librarian than that of a farmer. However, the researchers pointed out that this judgment might be considered irrational, as people generally failed to incorporate the ratio of farmers to librarians into their judgments.
We can start by imagining a representative sample of farmers and librarians in a small city, for instance, 200 farmers and ten librarians. To make it easier to understand. Let's use the image below, where each square symbolizes a person. The ones with gray backgrounds are librarians and the ones with green backgrounds are farmers.
Upon hearing Steve's description, let's assume our gut instinct tells us that 40% of librarians would fit that description, and only 10% of farmers would.
This would mean that around four librarians (40% of ten) and 20 farmers (10% of 200) fit the description.
In the image below. Librarians following the description have a dark gray background, and farmers following the description have a dark green background:
Consequently, the probability that a random person fitting this description is a librarian would be 4/24. Let's see why:
Event H (Hypothesis): The person is a librarian.
Event E (Evidence): The person fits the description ("a man who is very shy and withdrawn...")
In general, that means:
Corresponds to the number of persons that are librarians given that they fit the description.
That is four people (librarians who fit the description) divided by the total number of people who fit the description (four librarians plus 20 farmers).
It's crucial to recognize that even if we believe a librarian is four times as likely as a farmer to fit Steve's description, it doesn't negate the fact that there are significantly more farmers. The essential principle underlying Bayes' theorem is that new evidence should not solely dictate our beliefs; instead, it should update our prior beliefs.
Bayes' theorem becomes relevant in situations where we have a hypothesis, such as Steve being a librarian, and we encounter some evidence, like the verbal description of Steve. Our objective is to determine the probability that the hypothesis holds true, given the validity of the evidence presented. ...
Get hands-on with 1400+ tech skills courses.