<aside> <img src="/icons/table_gray.svg" alt="/icons/table_gray.svg" width="40px" />

Table of Content

</aside>

Lecture 1: Handling Big Models

Quantization is shrinking models to small size, so that any one can run it on their own computer with no performance degradation.

Model Compression Techniques (Not covered in the course):

1. Pruning:

Remove connections/nodes/weights that are not important for the model.

2. Knowledge Distillation

Train Smaller Model (student) using the original model (teacher)

Quantization

Idea: Store the parameters of the model in lower precision (for example, from fp32 → int8)

Lecture 2: Data Types and Sizes

Integer Data Type:

Unsigned Integer (Always Positive):

• Range: for n-bits → $[ 0 , 2^n-1 ]$
• So, if:
  • INT8, then range [0, 255] → $255 = 2^8-1$
  • INT4, then range [0, 127] → $127 = 2^4-1$

Signed Integer:

• Range: for n-bits → $[-2^{n-1}, 2^{n-1}-1]$