Terminology in Probability
Five words carry the whole subject. A random experiment produces an outcome; collect all outcomes and you have the sample space; pick some of them and you have an event.
A process that produces an outcome.
A result of a single trial.
A single performance of an experiment.
A subset of the sample space (specific outcomes).
Getting heads: {H}
Drawing a heart: {13 heart cards}
The set of all possible outcomes of an experiment.
Coin: S = {H, T}
Cards: S = {52 cards}
Key connections between terms: a trial produces an outcome; outcomes live in the sample space; an event groups outcomes we care about.
Types of Events
Events with probability 0 (impossible) and 1 (sure).
Sure: getting a number between 1–6 on a die (P = 1)
Events consisting of a single outcome from the sample space.
Getting heads on a coin flip: {H}
Drawing the ace of spades
Events consisting of more than one outcome from the sample space.
Events that do not affect (independent) or do affect (dependent) each other.
Events that cannot occur simultaneously (no common outcomes).
Rolling a 2 and a 5 on a single die
Drawing a heart and a spade in one draw
A set of events that together cover the entire sample space.
Coin: {H} and {T}
Cards: {red cards} and {black cards}
Events that are complements of each other (E and E′). P(E) + P(E′) = 1.
OR: event A ∪ B (A or B or both). AND: event A ∩ B (A and B together).
AND: rolling even and greater than 3: {4, 6}
What Is Probability
Probability is the measure of the likelihood that an event will occur.
P(E) = Number of Favorable OutcomesTotal Number of OutcomesProperties
0 ≤ P(E) ≤ 1 · P(S) = 1 (the sample space is certain) · P(∅) = 0 (the empty set is impossible).
P(rolling a 4) = 1/6
P(getting heads) = 1/2
P(drawing an ace) = 4/52 = 1/13
P(rolling an even number) = 3/6 = 1/2
Empirical vs Theoretical Probability
Based on observations during an experiment.
Based on reasoning or formulas, without conducting an experiment.
Comparison example — rolling a die 50 times, counting sixes
| Type | Calculation | Result |
|---|---|---|
| Empirical | 8 sixes / 50 rolls | 0.16 (16%) |
| Theoretical | 1 face / 6 faces | 0.167 (16.7%) |
As the number of trials increases, empirical probability approaches theoretical probability. With few trials they may differ significantly; with many trials the empirical value converges.
Random Variables
A random variable is a variable whose possible values are numerical outcomes of a random phenomenon.
Countable outcomes that can be listed or enumerated.
Number on a die: {1, 2, 3, 4, 5, 6}
Customers in a store: {0, 1, 2, 3, …}
Defective items in a batch: {0, 1, 2, …, n}
Uncountable outcomes that can take any value within an interval.
Time until next bus: any non-negative value
Temperature: any value within a range
Weight of a product: within a tolerance range
Discrete: probability of specific values > 0, described by a Probability Mass Function (PMF).
Continuous: probability of any exact value = 0, described by a Probability Density Function (PDF).
Probability Distributions — PMF, PDF & CDF
A probability distribution describes the likelihood of different possible values of a random variable. All probabilities lie between 0 and 1, and they sum to 1. Common shapes: Binomial, Normal, Uniform, Poisson.
Think of the three functions as three ways of looking at the same distribution: lumps of probability at points (PMF), a density spread across a continuum (PDF), and probability accumulating as you move along the number line (CDF).
P(X = x) = f(x). Used for discrete variables; probability of exact values > 0; visualized as bars.
P(a ≤ X ≤ b) = ∫ab f(x) dx. Used for continuous variables; probability of exact values = 0; visualized as a smooth curve — probability is area.
F(x) = P(X ≤ x). Values range from 0 to 1; non-decreasing; visualized as an S-shaped (step) curve.
Mean & Variance of a Random Variable
The weighted average of the possible values of a random variable.
x = each value · P(x) = its probability · Σ = sum over all values
How spread out the values are around the mean. Standard deviation σ is the square root of the variance.
σ² = E[X²] − (E[X])²
Worked example — rolling a fair die
Interpretation: over many rolls, the average converges to 3.5, and outcomes typically sit about 1.71 away from that mean.
Venn Diagrams
A Venn diagram is a visual tool using circles to represent sets, showing relationships between events and probabilities.
Universal set: contains all possible outcomes.
Event sets: circles representing specific events.
Intersection: overlapping area for simultaneous events.
Compare the area of regions to the total area: P(A) = Area of A / Total Area.
Venn diagrams visualize event relationships, simplify complex probability calculations, and make joint and conditional probabilities intuitive — as you will see below.
The addition rule hides inside the diagram: P(A ∪ B) = P(A) + P(B) − P(A ∩ B) — add the circles, then subtract the double-counted overlap. Drag P(A ∩ B) to zero and the events become mutually exclusive.
Contingency Tables
A contingency table shows relationships between two or more categorical variables — also called a cross-tabulation. Rows are categories of one variable, columns of another, and cells hold frequency counts for specific combinations.
Joint probability: cell value ÷ grand total.
Marginal probability: row/column total ÷ grand total.
Conditional probability: cell value ÷ row/column total → P(A|B) = P(A∩B) / P(B).
The table below is a real survey: 223 customers classified by gender and computer purchase. It is fully editable — change any count and every total, marginal, joint and conditional probability updates live. The next three sections each use one mode of this same lab.
| counts ↓ | PC | Mac | Row Totals |
|---|---|---|---|
| Male | |||
| Female | |||
| Column Totals |
Applications: market research, medical diagnosis, risk assessment, social surveys, data analysis.
Joint Probability
The probability of two or more events occurring simultaneously — the overlap of the Venn diagram, a single cell of the table.
P(A ∩ B) = P(A) × P(B) independent events P(A ∩ B) = P(A | B) × P(B) general case — joint = conditional × marginalCalculation methods
Venn diagram: overlapping area ÷ total area. Contingency table: cell value ÷ total — in our survey, P(Male ∩ PC) = 66/223 ≈ 0.296.
Rolling two dice, probability of both being 6: P(6 ∩ 6) = 1/6 × 1/6 = 1/36 (independent rolls).
Marginal Probability
The probability of a single event occurring, without considering other events. It lives in the margins of the table — the row and column totals.
P(A) = Σ P(A ∩ Bi) P(B) = Σ P(Ai ∩ B) sum the joint probabilities over every level of the other variableCalculation methods
Venn diagram: single circle area ÷ total area. Contingency table: row/column total ÷ total.
P(Female) = Female row total / Total = 117/223 ≈ 0.525. Likewise P(PC) = 96/223 ≈ 0.430 and P(Mac) = 127/223 ≈ 0.570.
Conditional Probability
The probability of one event occurring given that another event has already occurred. The given information shrinks the universe: only the outcomes inside the conditioning event still count.
P(A | B) = P(A ∩ B)P(B) P(B | A) = P(A ∩ B)P(A) conditional = joint ÷ marginalCalculation methods
Venn diagram: overlapping area ÷ area of the given circle. Contingency table: cell value ÷ row/column total.
P(Mac | Female) = 87/117 ≈ 0.744 — the probability of buying a Mac, given the customer is female.
How the Three Connect
P(A|B) = P(A ∩ B) / P(B)
Conditional = Joint ÷ Marginal
P(A ∩ B) = P(A|B) × P(B) = P(B|A) × P(A)
Joint = Conditional × Marginal
P(A) = Σ P(A|Bi) × P(Bi)
Marginal = weighted sum of conditionals
Medical diagnosis: predicting disease probability from symptoms.
Market analysis: customer purchase behavior prediction.
Risk assessment: financial and insurance risk modeling.
Bayesian inference: updating beliefs with new evidence.
Machine learning: foundation for classification algorithms.
Data mining: discovering hidden relationships in data.
Bayes' Theorem
P(A|B) = P(B|A) × P(A)P(B) Alternative form: P(A|B) = P(B|A)×P(A) / [P(B|A)×P(A) + P(B|¬A)×P(¬A)]Core principle: it inverts conditional probability. Key insight: it updates beliefs with new evidence.
Initial belief before evidence.
Probability of the evidence if the hypothesis is true.
Updated belief after seeing the evidence.
Normalizes the calculation (via the law of total probability).
Joint probability: P(A∩B) = P(B|A)×P(A). Conditional probability: inverts P(A|B) to find P(B|A). Marginal probability: P(B) comes from the law of total probability.
Medical diagnosis — the classic scenario
A test for a rare disease: 1% prevalence, 99% true-positive rate, 2% false-positive rate. If your test comes back positive, what is the probability you actually have the disease? The answer surprises almost everyone — drag the sliders to see why.
DEFAULTS MATCH THE SLIDE: 1% prevalence · 99% sensitivity · 2% false-positive rate
Out of 1,000 people: about 10 have the disease (9.9 test positive), but about 20 healthy people also test positive — so a positive result means only ≈ 1/3 chance of disease. The rareness of the disease (the prior) dominates.
Bayes' Theorem Example — Titanic Survival
Question: what is the probability that a passenger survived, given they were in first class?
| Passenger Class | Survived | Not Survived | Total |
|---|---|---|---|
| 1st Class | 203 | 122 | 325 |
| 2nd Class | 118 | 167 | 285 |
| 3rd Class | 178 | 528 | 706 |
| Total | 499 | 817 | 1316 |
First-class passengers had a 62.4% chance of survival — significantly higher than the overall survival rate of 37.9%. (Check it directly from the table: 203/325 ≈ 0.625 — Bayes gets the same answer by inverting the conditional.)
Key Concepts & Formula Sheet
✓ Venn diagrams: visualize event relationships and probabilities.
✓ Contingency tables: organize categorical data systematically.
✓ Joint probability: likelihood of multiple events occurring together.
✓ Marginal probability: likelihood of a single event occurring.
✓ Conditional probability: likelihood of one event given another.
▸ Decision support: probability-based informed decision making.
▸ Risk assessment: quantify uncertainty, manage risks.
▸ Predictive modeling: build data-driven prediction systems.
▸ Scientific reasoning: foundation for Bayesian inference.