Module: Soft start: Linear methods

Prerequisite Modules

Linear Regression

Linear Regression

EE 345

Linear Classifiers

Linear Classifiers

EE 445

Single Neuron Networks

Basic computational unit of a neural network

EE 445

Feedforward Neuron Networks

Basic feedforward neural network (Icon: Wikipedia)

EE 445

Learning Outcomes

Linear Models

• You understand the basic principles and visualizations behind linear regression and some classification approaches
• You can multiply matrices, and interpret matrix multiplication as linear combinations (both row and column views)
• You follow the idea of projections into linear spaces
• You understand logistic regression through the probabilistic and maximum entropy lens
• You have set up a working python environment for machine learning and deep learning

Submissions required for this module

You have to submit at least one of the two assessments (the theory or simulations). Whatever you submit, make sure you follow the other assignment as well. All submissions will be on Laulima.

• A detailed analysis of logistic regression as a maximum entropy approach is here. The theory homework is based off this writeup (most of the solutions are already in the writeup, a few details need to be filled up). If you submit this part, make sure you have everything set up to do the simulations in the second assignment (you will need it later).

• We did a demo with logistic regression and the weather data on Jan 22. You will have to implement the same in your simulation homework. In addition, you have to build a single neuron network that simulates linear regression (ordinary least squares as well as regularized linear regression). If you submit this part, make sure you have the basic understanding of probability needed for the theory part.

Experiential Learning

Assessments