By Web Asset House · data reviewed 2026-10-22
Quantization stores model values with a reduced numerical representation. It usually changes both the weight-file size and model behavior. A label such as Q4_K_M is not exactly four bits for every byte in the file: tensor formats, mixed precision, metadata, and alignment contribute to the recorded size.
What the calculator includes
The dataset records Q4_K_M, Q5_K_M, Q8_0, and FP16 only when verified repository listings contain complete files. Split files are summed. An unavailable format stays unavailable rather than being estimated from a different format. The source link identifies the repository and revision.
Start with the task, then use the memory budget
A smaller file can create headroom for context, but a lower memory requirement is not a quality guarantee. Evaluate representative nonsensitive examples on your own runtime before selecting a model for regular use. We do not label one quantization universally best.
Read the speed proxy correctly
The formula divides file bytes by total parameters to approximate bytes per weight, then multiplies by active parameters for a per-token read estimate. That ratio includes format overhead. It is a transparent planning approximation, not an exact accounting of kernel traffic. Cache behavior, compute, and hardware support can materially change measured performance.
File format and model license still matter
GGUF is a model-file format, not a license or a promise that every runtime supports every model architecture. Verify the original model terms, the conversion source, and the intended runtime’s support.
Primary sources
- Ollama: memory, context, local and cloud behavior
- llama.cpp: supported build backends
- Hugging Face: GGUF documentation
Questions, answered.
Is Q4_K_M exactly four bits per parameter?
Not as a file-size guarantee. Mixed tensor types, metadata, and alignment affect the file. This site uses recorded complete file sizes.
Why is a quantization missing?
The source snapshot did not establish a complete supported file set for that format. The site does not invent missing file sizes.