Quick Deep Learning Notes January, 2022

When I started exploring deep learning, I had to look up different sources for each topic; there are so many of them. Now that I have come to a conclusion about sources that work for me, I share some of these (hand-written) notes in case somebody is looking for fast-paced yet A-to-Z tutorials on Deep Learning and PyTorch. Although typing all the notes is a thing now, I still prefer hand-written notes. So, please excuse my scribbles and doodles!

• Basic Concepts
  
  • Defining Some Terms
  • Numpy Introduction
  • Perceptron
  • Perceptron Algorithm
  • Errors
  • Gradient Descent
  • Neural Network Architecture
  • FeedForward
  • Back Propagation
  • Gradient Descent Algorithm
  • Multi-layer Perceptron
  • Multiclass Classification
  • Maximum Likelihood
  • Random Restart
  • Dropout
  • Regularization
  • Sentiment Analysis
• PyTorch Basics
  
  • Tensors
  • Neural Networks
  • Nueral Networks Module (nn)
  • Training Neural Networks in PyTorch
  • Avoiding Overfitting
  • Save and Load Models
  • Handling Image Data
  • Transfer Learning
  • Full Nueral Network Example
• Convolutional Neural Networks (CNN)
  
  • Interpreting Images
  • Some Terms
  • Convolutional Layers
  • MNIST in PyTorch
  • CNN Components
  • CNN versus MLP
  • Convolutional Layers in PyTorch
  • Bird's Eye View of Convolutional Layers
  • Bird's Eye View of CNN
  • CIFAR in PyTorch
  • Invariant Representation
  • ResNet Architecture
  • Transfer Learning Cheatsheet
  • Transfer Learning in PyTorch
  • Initial Weights
  • Auto-encoders
  • Style Transfer in PyTorch
• Sequence-to-sequence Models
  
  • Some Terms
  • Recurrent Neural Networks (RNN)
  • Training RNNs
  • RNN in PyTorch
  • Long Short-term Memory Cell (LSTM)
  • More Advanced Cells
  • LSTM in PyTorch
  • Tuning Hyperparameters
  • LSTM Sentiment Analysis in PyTorch
  • Embedding
  • Word2vec in PyTorch
  • Attention Mechanism
  • Transformer
• Graph Neural Networks
  
  • GNN Overview
  • Toy PyTorch Geometric Example
  • Graph Attention Networks in PyG
• Generative Adversarial Networks
  
  • GAN Overview
  • GAN in PyTorch
  • Deep Convolutional GAN
  • Batch Normalization
  • DCGAN in PyTorch
• Deep Graph Library (DGL)
  
  • Node Classification Implementation
  • Working with Graph Data
  • Link Prediction Implementation
  • Graph Classification Implementation (soon)
  • Implement a GNN (soon)

References:

• Udacity Deep Learning Nano-degree
• https://github.com/udacity/deep-learning-v2-pytorch
• Hands-on Machine Learning with Scikit-Learn, Keras, and Tensorflow by Aurelien Geron
• Deep Learning by Ian Goodfellow, Yoshua Benigo, and Aaron Courville
• https://www.youtube.com/watch?v=ABCGCf8cJOE
• https://www.youtube.com/watch?v=-UjytpbqX4A
• Colab Notebooks and Video Tutorials
• A Blitz Introduction to DGL

---