Deep Learning
Overview
Deep learning uses artificial neural networks with multiple layers to learn hierarchical representations of data. Unlike classical ML where features are hand-engineered, deep learning automatically discovers the representations needed for detection or classification.
Why Deep Learning?
graph TD
A[Classical ML] --> B[Manual Feature Engineering]
B --> C[Domain Expertise Required]
C --> D[Limited to Known Features]
E[Deep Learning] --> F[Automatic Feature Learning]
F --> G[Learns from Raw Data]
G --> H[Discovers Unknown Patterns]
| Aspect | Classical ML | Deep Learning |
|---|---|---|
| Features | Manual engineering | Automatic learning |
| Data requirements | Small to medium | Large |
| Compute | CPU | GPU/TPU |
| Interpretability | High | Low |
| Performance on unstructured data | Limited | State-of-the-art |
Topics in This Section
| Topic | Key Concepts | Interview Frequency |
|---|---|---|
| Neural Network Basics | Perceptron, MLP, universal approximation | ⭐⭐⭐⭐⭐ |
| Backpropagation | Chain rule, computational graphs | ⭐⭐⭐⭐⭐ |
| Activation Functions | ReLU, sigmoid, GELU, Swish | ⭐⭐⭐⭐ |
| CNNs | Convolution, pooling, ResNet | ⭐⭐⭐⭐⭐ |
| RNNs & LSTMs | Vanilla RNN, LSTM, GRU | ⭐⭐⭐⭐ |
| Batch Normalization | Layer norm, group norm | ⭐⭐⭐⭐ |
| Dropout | Training vs inference | ⭐⭐⭐⭐ |
| Optimizers | Adam, AdamW, learning rate schedules | ⭐⭐⭐⭐⭐ |
| Transfer Learning | Fine-tuning, feature extraction | ⭐⭐⭐⭐ |
| Attention Mechanism | Self-attention, multi-head attention | ⭐⭐⭐⭐⭐ |
The Deep Learning Revolution
graph LR
A[1950s: Perceptron] --> B[1980s: Backpropagation]
B --> C[1990s: SVMs dominate]
C --> D[2012: AlexNet - Deep Learning revolution]
D --> E[2014: GANs, VAEs]
E --> F[2017: Transformers]
F --> G[2020s: LLMs, Foundation Models]
Key breakthroughs:
- 2012: AlexNet wins ImageNet (GPU training, ReLU, dropout)
- 2014: GANs (generative models), VAEs
- 2015: ResNet (skip connections), Batch Normalization
- 2017: Transformers (“Attention is All You Need”)
- 2018-2020: BERT, GPT-2/3 (pre-training revolution)
- 2022+: ChatGPT, GPT-4 (LLMs as general-purpose AI)