Probability

Definitions

Random Process / Probability Experiment

• Must be repeatable
• Allows for exact listing of all possible outcomes
• e.g. rolling a die and observing the top face

Sample Space

• The collection of all possible outcomes of a random process
• Denoted $S$
• e.g. for rolling a die, $S=\{1,2,3,4,5,6\}$

Event

• A subset of the sample space
• e.g. for rolling a die, the event “die shows an even number” is $\{2,4,6\}$

Mutually Exclusive

Events $A$ and $B$ are mutually exclusive when they cannot occur at the same time $A\cap B=\varnothing$ $P(A\text{ and }B)=0$

Independent

Events $A$ and $B$ are independent when the occurrence of one event does not influence the likelihood of the other event happening $P(A|B)=P(A)$ $P(B|A)=P(B)$ Conditional independence, where $A$ and $B$ are conditionally independent given an event $C$ with $P(C)>0$: $P(A\cap B|C)=P(A|C)\times P(B|C)$

Types of Random Variable

• Discrete Random Variable
  • Where the sample space forms a discrete numerical variable
  • e.g. Rolling a die
• Continuous Random Variable
  • Where the sample space forms a continuous numerical variable
  • e.g. Normally distributed IQ scores with mean 100 and variance 25

Uniform probability

• Where every outcome in the sample space has the same probability

Rules of Probabilities

1. The probability of each event $E$, denoted $P(E)$ is between 0 and 1 (inclusive)
2. For a sample space $S$, $P(S)=1$
3. If $E$ and $F$ are Mutually Exclusive events, $P(E\cup F)=P(E)+P(F)$

Conditional Probability

• Probability that the outcome $\in E$, if we already know that the outcome $\in F$
• Denoted $P(E|F)$ “Probability of $E$, given $F$”
• $P(E|F)=\frac{P(E\cap F)}{P(F)}$

Properties of a test

• Sensitivity
  • True positive rate
  • P(Test +ve | Actually +ve)
• Specificity
  • True negative rate
  • P(Test -ve | Actually -ve)
• These are not very useful, we would want to know P(Actually +ve | Test +ve)
  • To find this, we need the base rate, or P(Actually +ve)
  • To find this, draw the table with columns Test +ve, Test -ve, Row total and rows Actually +ve, Actually -ve, Column total. Then start with any number for the total population, and fill in the other blanks to determine the rate (and associated probability)

Comparison with Rates

Probability	Rates
Probability experiment	Random sampling
Sample space	Sampling frame
An event $A$ of the sample space	A subgroup of $A$ of the sampling frame
$P(A)$	rate($A$)
Independent events	Not associated
This comparison can be made using the random process of selecting one unit from the population with [[#uniform-probability	Uniform probability]]. Then, all the laws for probabilities and rates can be applied to each other.
e.g. rate(Heart Disease	Smoker) = P(Heart Disease

Probability Laws

Note: $P(A\cup B)=P(A\text{ or }B)$, $P(A\cap B)=P(A\text{ and }B)$

Complement Rule

$P(A)=1-P(A^c)$

Addition Rule

$P(A\cup B)=P(A)+P(B)-P(A\cap B)$ If $A$ and $B$ are mutually exclusive $P(A\cup B)=P(A)+P(B)$

Multiplication Rule

$P(A\cap B)=P(A)\cdot P(B|A)=P(B)\cdot P(A|B)$ If $A$ and $B$ are independent $P(A\cap B)=P(A)\cdot P(B)$

Law of Total Probability

If $E$, $F$ and $G$ are events from the sample space $S$ such that

• $E$ and $F$ are mutually exclusive
• $E\cup F=S$ Then $P(G)=P(G|E)\cdot P(E)+P(G|F)\cdot P(F)$ $=P(G\cap E)+P(G\cap F)$

Fallacies

Prosecutor’s Fallacy

When thinking that $P(A|B)=P(B|A)$

Base Rate Fallacy

When ignoring the rate of occurrence of some trait (base rate) when answering the question e.g. in the case of a covid test P(infected | +ve test), base rate is P(infected). For countries with different P(infected), P(infected | +ve) will be different

Conjunction Fallacy

Thinking that the chance of two things happening together $>$ the chance of one of these things happening alone e.g. Someone is loitering with a knife. Thinking that P(He is a thief and is jobless) > P(He is a thief)