Random Variable

Definition

• Random Variable: a function $X$ that assigns a real number to every $s\in S$
  • $X$ is a function from $S$ to $\mathbb{R}$ ($X:S\mapsto \mathbb{R}$)
  • E.g. Experiment is rolling a die, $X$ is the number on the top face
• Range space of $X$ is the set of real numbers $R_X=\{x|x=X(s),s\in S\}$ (Set of all possible values that $X$ can take, or set of all outputs of the function)
• Notation
  • Capital letters denote random variables
  • Lower case letters denote observed values in the experiment
  • $\{X=x\}$ means $\{s\in S:X(s)=x\}$, and is a subset of $S$
  • If $A\subset \mathbb{R}$, then $\{X\in A\}$ means $\{s\in S:X(s)\in A\}$, and is a subset of $S$
  • $P(X=x)=P(\{s\in S:X(s)=x\})$
  • $P(X\in A)=P(\{s\in S:X(s)\in A\})$

Probability Distribution

• Discrete random variable: The number of values in $R_X$ is finite or countable: $R_X$ can be written as $\{x_1,x_2,\cdots \}$
• Probability (mass) function (pf/pmf): $f(x)=P(X=x)$ for $x\in R_X$, $0$ otherwise
  • $f(x_i)\ge 0$ for all $x_i\in R_x$ ; $f(x)=0$ for all $x\notin R_X$
  • Sum of all $f(x_i)=1$
  • $P(X\in B)=$ sum of $f(x)$ for all $x\in B\cap R_X$
  • Range: $0\le f(x)\le 1$
• Probability distribution: Collection of pairs $(x_i,f(x_i))$
• Continuous random variable: $R_X$ is an interval or a collection of intervals
• Probability (density) function (pf/pdf): $f(x)$ satisfies:
  • $f(x)\ge 0$ for all $x\in R_X$ ; $f(x)=0$ for all $x\notin R_X$
  • ${\int }_{R_X}f(x)dx=1$ , or ${\int }_{-\infty }^{\infty }f(x)dx=1$
  • If a function satisfies these two conditions^, it is a pdf
  • For any $a\le b$ , $P(a\le X\le b)={\int }_a^{b}f(x)dx$
  • For any specific value $x_0$, $P(X=x_0)={\int }_{x_0}^{x_0}f(x)dx=0$ “$P(A)=0$ ,but $A$ might not be $\varnothing$”
  • In $P(a\le X\le b)$, any $\le$ can be swapped with $<$ , and the value will be the same
  • E.g. for a pdf $f(x)=cx$ for $0<x<1$, $1={\int }_{-\infty }^{\infty }f(x)dx={\int }_0^{1}cxdx=c\cdot \frac{x^2}{2}{|}_0^1=c/2$
  • Range: $f(x)\ge 0$, but not necessary that $f(x)\le 1$ (pdf does not represent probability)
• Cumulative distribution function cdf: $F(x)=P(X\le x)$
  • Applies whether $X$ is discrete or continuous
  • Always non-decreasing
  • For any proabability function, the cdf is uniquely determined, and vice versa
  • Range: $0\le F(x)\le 1$
  • For discrete random variable:
    • $F(x)=$ sum of $f(t)$ or $P(X=t)$ for all $t\in R_X:t\le x$
    • For any $a<b$, $P(a\le X\le b)=P(X\le b)-P(X<a)F(b)-F(a^{-})$ where $a^{-}$ is “largest value in $R_X$ that is smaller than a”, or ${\lim }_{x\uparrow a}F(x)$
  • For continuous random variable:
    • $F(x)={\int }_{-\infty }^{x}f(t)dt$ ; $f(x)=dF(x)/dx$
    • $P(a\le X\le b)=P(a<X<b)=F(b)-F(a)$

Expectation

• For discrete rv: expectation or mean of $X={\mu }_X=E(X)={\sum }_{x_i\in R_X}{x}_{i}f(x_i)$
• For continuous rv: ${\mu }_X=E(X)={\int }_{-\infty }^{\infty }xf(x)dx={\int }_{x\in R_X}xf(x)dx$
• Properties
  • For $a,b\in \mathbb{R},E(aX+b)=aE(X)+b$
  • $E(X+Y)=E(X)+E(Y)$
  • For any function $g$, rv $X$ with probability function $f(x)$ and range $R_X$
  • If $X$ is discrete, $E[g(X)]={\sum }_{x\in R_X}g(x)f(x)$
  • If $X$ is continuous, $E[g(X)]={\int }_{R_X}g(x)f(x)dx$
  • Iff $X$ and $Y$ are independent, $E(XY)=E(X)E(Y)$

Variance

• Variance ${\sigma }^2$ of $X$: ${\sigma }_X^2=V(X)=E(X-{\mu }_X{)}^2$ ← this means $E((X-{\mu }_X{)}^2)$
  • If $X$ is discrete rv, $V(X)={\sum }_{x\in R_X}(x-{\mu }_X{)}^{2}f(x)$
  • If $X$ is continuous rv, $V(X)={\int }_{-\infty }^{\infty }(x-{\mu }_x{)}^{2}f(x)dx$
• $V(X)\ge 0$ for any $X$. $=0$ iff $P(X=E(X))=1$ i.e. $X$ is a constant
• For real numbers $a$ and $b$, $V(aX+b)=a^{2}V(X)$
• $V(X)=E(X^2)-(E(X){)}^2$
• Standard deviation ${\sigma }_x=\sqrt{V(X)}$