EE 445 | Learning Outcomes

You understand the basic building blocks of machine learning
You understand every module at level covered in class
You can handle datasets, implement the material learned in class using libraries or your own code
You understand the connections between concepts here and in other courses you have learned
You have the ability to follow the advanced material in modules with some help from me
You have a good picture of the diversity of applications from the modules

You understand how to use and operate univariate Gaussians, including certain definite integrals involved in moment calculations
You understand what jointly Gaussian random variables are, and can give an example of two Gaussians that are \emph{not} multivariate Gaussian
You understand how to find the conditional expectations with multivariate Gaussian random variables

Referencing modules: Gaussians

Understand the purpose behind this formulation.
You should be able to set up a learning problem in the standard probabilistic framework (which we will call Probably Approximately Correct or PAC learning), specifically the notions of \emph{confidence} and \emph{accuracy}.
For the problem of learning conjunctions, you should understand how to derive the PAC bound on the training sample, and be able to work out similar bounds for simple problems.

Referencing modules: Linear Classifiers

Formulate linear least squares from Maximum Likelihood and Bayesian perspectives
Understand Bayesian linear least squares and ridge regression
Connect Ridge Regression with conditional means of multivariate Gaussians
Appreciate the distinction between geometry of regression and generalization
Some idea of significance of features ((t-)statistics)
Feature selection using LASSO

Referencing modules: Linear Regression

You are able to build simple neural networks (feedforward/autoencoders)
You are familiar with multivariate Taylor series, the gradient and the Hessian (see background if not)
You understand (at a high level) stochastic gradient descent, and its accelerations to replicate some second order behavior using past values of gradients
You can train networks using stochastic gradient descent/variants
You understand autoencoder architectures (linear activations) in relation to the singular value decomposition
You can train autoencoders to project training datasets to low dimensional (potentially non-linear) manifolds

Referencing modules: Neural networks

Build and train a neuron on keras/tensorflow using stochastic gradient descent
Form an initial idea of Stochastic Gradient Descent using the example of Perceptron Learning Algorithm
Understand how the choice of activation/loss and regularization can replicate well known machine learning algorithsm

Referencing modules: Single neuron networks

Understand how to set up the maximum margin classifier in the linearly separable case
Extend the linear separable case to non-linearly separable data
Understand the convex optimization setup, Lagrangians and the dual formulation
Appreciate the nature of solutions of the SVM:
- only certain examples (support vectors) determine the solution, identify these from Lagrange multipliers
- Appreciate that the SVM formulation only deals with pairwise dot products among examples
Kernel trick that allows us to move from linear to non-linear classification boundaries in the example space

Learning Outcomes What you will know