Learning Outcomes What you will know

Take aways from EE 645

• You understand how things tick in some current state of the art AI and ML techniques
• You can formulate problems and take a principled way to solve it
• You get a good idea of the connections between machine learning/AI concepts and applications in other courses
• You can communicate to a broader audience regarding the utility, risks and guarantees of AI/ML solutions

Referencing modules: Introduction

Linear Models

• You understand the basic principles and visualizations behind linear regression and some classification approaches
• You can multiply matrices, and interpret matrix multiplication as linear combinations (both row and column views)
• You follow the idea of projections into linear spaces
• You understand logistic regression through the probabilistic and maximum entropy lens
• You have set up a working python environment for machine learning and deep learning

Submissions required for this module

You have to submit at least one of the two assessments (the theory or simulations). Whatever you submit, make sure you follow the other assignment as well. All submissions will be on Laulima.

• A detailed analysis of logistic regression as a maximum entropy approach is here. The theory homework is based off this writeup (most of the solutions are already in the writeup, a few details need to be filled up). If you submit this part, make sure you have everything set up to do the simulations in the second assignment (you will need it later).

• We did a demo with logistic regression and the weather data on Jan 22. You will have to implement the same in your simulation homework. In addition, you have to build a single neuron network that simulates linear regression (ordinary least squares as well as regularized linear regression). If you submit this part, make sure you have the basic understanding of probability needed for the theory part.

Referencing modules: Soft start: Linear methods

Assessed by: Linear methods: theory, Logistic Regression

Kernel Methods

This module was also a warp-speed introduction to optimization, convex functions, and some linear algebra in addition to kernel methods. You are done with this module when:

• You can write out Lagrangians for constrained optimization
• You understand primal and dual formulations (min-max and max-min) with Lagrangians
• You can write out the Support vector classification formulations, both in the linearly separable and non-linearly separable case
• You understand SVC formulations as the hinge loss with \(\ell_2\) penalty view
• You understand quadratic forms and positive definite matrices
• You understand positive definite kernels and the kernel lifting
• You understand introducing non-linearities in algorithms that only depend on dot products using kernel lifting

Referencing modules: Kernel methods