The following python notebook helps you

• implement and train basic linear algorithms using single neuron networks,
• teaches the SVD using a linear autoencoder architecture

The implementation is in pytorch, you can reuse the notebook when you write your own code.