References
Build Your Own Deep Learning Framework
Twitter
LinkedIn
Preface
Part I: Forward Only
1
What is a Deep Learning Framework?
2
Tensor Basics
3
Operators
Part II: Learning to Learn
4
Loss Functions
5
The Computational Graph
6
Backpropagation
7
Your First Training Loop
Part III: Smarter Training
8
Optimizer Design
9
SGD with Momentum
10
Adam Optimizer
11
Learning Rate Schedules
Part IV: Going Deeper
12
Activation Functions
13
Regularization
14
Normalization
15
MLP for Iris Classification
Part V: Clean Architecture
16
Layer and Module
17
Container Modules
18
State Dict: Save and Load
19
Dataset and DataLoader
Part VI: To Production
20
Introduction to ONNX
21
Building the ONNX Exporter
22
ONNX Runtime Inference
Part VII: Hardware Acceleration
23
Introduction to GPU Computing
24
Backend Abstraction
25
GPU Training
Part VIII: The Grand Finale
26
Embedding Layers
27
The Attention Mechanism
28
Multi-Head Attention
29
The Transformer Block
30
Building GPT
31
Text Generation
Appendix
32
Design Decisions
33
Further Reading
References
References
33
Further Reading