Deep-Learning-Labs

Various ML implementations and mathematical proofs for the UofT Neural Networks and Deep Learning course

Mathematical proofs

HW1 - Neural Networks & Backpropagation

This folder contains the solutions to four problems relating to neural network design, computation graphs and backpropagation. Read the instructions here and the solutions here.

HW2 - Model-Agnostic Meta-Learning (MAML) and Optimisers (Momentum SGD, RMSProp, Adam)

This folder contains the implementation of Model-Agnostic Meta-Learning (MAML) using autodiff as well as mathematical proofs behind momentum SGD, RMSProp and Adam. Read the instructions here and the solutions here.

HW3 - Dropout and Recurrent Neural Networks (RNNs)

This folder contains mathematical proofs behind dropout and a hand derived RNN implementation for binary addition. Read the instructions here and the solutions here.

HW4 - LSTM Backpropagation Through Time, MDRNNs & Reversible Architectures

This folder contains mathematical proofs behind LSTM backpropagation through time (BPTT), multidimensional RNNs and reversible architectures. Read the instructions here and the solutions here.

Programming

PA1 - Learning Distributed Word Representations (Neural Language Model)

This folder contains the implementation of a Neural Language Model and the analysis of its output characteristics. Read the instructions here and the solutions here.

PA2 - Convolutional Neural Networks

This folder contains the implementation of a Convolutional Neural Network (CNN) for colourisation of horses in the CIFAR-10 dataset. Read the instructions here and the solutions here.

PA3 - Attention-Based Neural Machine Translation

This folder contains the implementation of an Attention-Based Neural Machine Translation to translate words from English to Pig-Latin. Read the instructions here and the solutions here.

PA4 - Deep Convolutional GAN (DCGAN) and CycleGAN for emoji generative models

This folder contains the implementation of a Deep Convolutional Generative Adversarial Network (DCGAN) and a CycleGAN for emoji generation. Read the instructions here and the solutions here.