Quantization vs Distillation: Key Differences
The short answer Quantization vs distillation solves the same problem in different ways. Quantization keeps the same model and stores each weight in fewer bits. The parameter count never changes; only the bytes per weight do. Distillation, instead, trains a…