Joint Distributions

Multiple random variables

• 2-dimensional random vector: For experiment $E$ with sample space $S$, and $X$ and $Y$ are two functions each assigning a real number to each $s\in S$, $(X,Y)$ is a 2d r vector, or 2d r variable
  • can be extended to n-dimensional random vector with $(X_1,\cdots ,X_n)$
• Range space $R_{X,Y}=\{(x,y)|x=X(s),y=Y(s),s\in S\}$
• Discrete 2d rv: no. of possible vals of $(X(s),Y(s))$ is finite/countable, OR $X$ and $Y$ are discrete rv
• Continuous 2d rv: possible values of $(X(s),Y(s))$ can assume any value in some region of ${\mathbb{R}}^2$, OR both X and Y are continuous rv
• Discrete joint probability function $f_{X,Y}(x,y)=P(X=x,Y=y)$
  • $f_{X,Y}(x,y)\ge 0$ for any $(x,y)\in R_{X,Y}$
  • $f_{X,Y}(x,y)=0$ for any $(x,y)\notin R_{X,Y}$
  • ${\sum }_{i=1}^{\infty }{\sum }_{j=1}^{\infty }{f}_{X,Y}(x_i,y_j)={\sum }_{i=1}^{\infty }{\sum }_{j=1}^{\infty }P(X=x_i,Y=y_j)=1$
  • $\sum {\sum }_{(x,y)\in R_{X,Y}}f(x_i,y_j)=1$
  • For any subset $A$ of $R_{X,Y}$ : $P((X,Y)\in A)={\sum }_{(x,y)\in A}\sum f_{X,Y}(x,y)$
• Continuous joint probability function $f_{X,Y}(x,y)$
  • $P((X,Y)\in D)={\iint }_{(x,y)\in D}{f}_{X,Y}(x,y)dydx$ , for any $D\subset {\mathbb{R}}^2$
  • Specifically, $P(a\le X\le b,c\le Y\le d)={\int }_a^b{\int }_c^d{f}_{X,Y}(x,y)dydx$
  • $f_{X,Y}(x,y)\ge 0,\text{ for any }(x,y)\in R_{X,Y}$
  • $f_{X,Y}(x,y)=0,\text{ for any }(x,y)\notin R_{X,Y}$
  • ${\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }{f}_{X,Y}(x,y)dxdy=\int {\int }_{(x,y)\in R_{X,Y}}f(x,y)=1$

Marginal Distributions

• If $Y$ is discrete rv, for any $x$, marginal distribution of $X$, $f_X(x)={\sum }_y{f}_{X,Y}(x,y)$
• If $Y$ is continuous rv, for any $x$, $f_X(x)={\int }_{-\infty }^{\infty }{f}_{X,Y}(x,y)dy$
• Marginal distribution of $X$ is distribution of $X$ ignoring the presence of $Y$
• $f_X(x)$ should not involve the variable $y$: it is either summed out or integrated over
• $f_X(x)$ is a probability function and satisfies all properties of pfs

Conditional Distributions

• For any $x$ such that $f_X(x)>0$, conditional probability of $Y$ given $X=x$ is: $f_{Y|X}(y|x)=\frac{f_{X,Y}(x,y)}{f_X(x)}$
• $f_{Y|X}(y|x)$ is defined only for $x$ such that $f_X(x)>0$
• $f_{Y|X}(y|x)$ means the distribution of $Y$ given that $X$ is observed to be $x$
• With $y$ as the variable ($x$ is fixed value), $f_{Y|X}(y|x)$ is a pf, and satisfies all properties
  • it is NOT a pf for $x$
• If $f_X(x)>0,f_{X,Y}(x,y)=f_X(x)f_{Y|X}(y|x)$
• If $f_Y(y)>0,f_{X,Y}(x,y)=f_Y(y)f_{X|Y}(x|y)$
• Probability $Y\le y$, given $X=x$: $P(Y\le y|X=x)={\int }_{-\infty }^{\infty }{f}_{Y|X}(y|x)dy$
• Average value of $Y$ given $X=x$: $E(Y|X=x)={\int }_{-\infty }^{\infty }y{f}_{X|Y}(y|x)dy$