Learning Outcomes What you will know

Competent with basics of univariate and multivariate Gaussians

• You understand how to use and operate univariate Gaussians, including certain definite integrals involved in moment calculations
• You understand what jointly Gaussian random variables are, and can give an example of two Gaussians that are \emph{not} multivariate Gaussian
• You understand how to find the conditional expectations with multivariate Gaussian random variables

Referencing modules: Gaussians

Sets, Logic and Functions

• You are comfortable with sets, and can use the set comprehension notation naturally
• You understand unions, intersections, complements, and can prove set identities involving these operations
• You understand the power set of a set as well as the Cartesian product between sets
• You understand functions, arithmetic of functions, and indicator functions
• You understand how logical statements can translate to sets and vice versa
• You understand how set operations translate to arithmetic of indicator functions.

Referencing modules: Sets, Logic and Functions

Setup of a probability space

• You understand what sample space, an event and a probability assignment are
• You can set up the sample space for problems
• You know how to make a probability assignment for a countable sample space
• You understand that intervals get probabilities in an uncountable sample space via integration
• You recognize there is a distinction between the power set and the set of all events in the uncountable case

Referencing modules: Probability Space

Conditional Probabilities and Independence

• You understand how to translate partial information to conditional probabilities
• You understand what independence of (two or more) events means
• You can distinguish independence from pairwise independence
• You understand the multiplication rule with conditional probabilities
• You understand the Prosecutor’s Fallacy, the Simpson’s paradox and the Monty Hall problem

Referencing modules: Conditional Probabilities and Independence

Bayes Theorem

• You use the law of total probability comfortably
• You can invert the order of conditioning using Bayes’ Theorem
• You understand how to formulate problems in the Bayesian way

Referencing modules: Bayes Theorem

Random Variables

• You understand what functions are, and what a random variable is
• You understand composition of functions, and functions of random variables
• You understand the pmf on (individual and multple) discrete random variables
• You can compute the expectation of random variables (and their functions)
• If \(X_1\) and \(X_2\) are independent random variables \({\mathbb E}[X_1X_2] = {\mathbb E}[X_1] {\mathbb E}[X_2]\)
• You understand why the expectation is a linear function of the random variables
• You can compute the higher moments of a random variable, in particular the variance
• You can define the moment generating function
• You understand why the moment generating function is called as such

Referencing modules: Random Variables

Assessed by: Problems in class

Bernoulli + friends

• You understand Bernoulli random variables
• You understand repeated independent Bernoulli trials when the number of trials is fixed in advance
• You understand how Binomial and Geometric random variables come from repeated independent Bernoulli trials
• You understand the probability distributions induced on Binomial and Geometric random variables via independent Bernoulli trials
• You can compute the mean and variance of Bernoulli and Geometric random variables

Referencing modules: Bernoulli and Friends

Geometric and Poisson

• You understand the probability space of independent Bernoulli trials when the number of trials is not fixed in advance
• Geometric random variables model the time to first success from repeated independent Bernoulli trials
• Poisson \(\lambda\) r.v. is the limit as \(n\to\infty\) of Binomial(\(n\), \(\frac{\lambda}n\)), ie large number of rare events
• You can compute the mean and variance of Geometric and Poisson random variables

Referencing modules: Geometric and Poisson

Continuous RV

• Review the notion of pdfs
• Assign probabilities to events formed by continuous random variables
• Cummulative Distribution functions
• Derive pdf transforms for functions of random variables
• Expectation and expectation of functions of random variables, Variance *

Referencing modules: Continuous Random Variables: Exponential

Conditional Expectation

1. There are three concepts in this module:
   • Conditional pmf/pdfs

   • Taking expectations with respect to (conditional) pmf/pdf on \(X\) conditioned on an event \(A\), $${\mathcal E} [ X

     A]$$

   • The special random variable: $${\mathcal E} [X

     Y]\(, a function of\)Y\(, and not a nubmer like\){\mathcal E} [ X

     A]$$

2. If we want to estimate a rv \(X\) (that we do not get to see), but make an observation \(Y\), then \({\mathcal E}[X|Y]\) is the best estimate of \(X\) (in the mean square sense)

3. Covariance, correlation, conditional variance

4. Law of iterated expectations \({\mathcal E}\bigl[{\mathcal E}[X|Y]\bigr] = {\mathcal E} X\)

5. $${\mathcal E}[X	Y]\(is uncorrelated with the error\)X- {\mathcal
   E}[X	Y]\(made in estimating\)X\(from\)Y$$

Referencing modules: Conditional Expectation