// tag archive

#GPU inference

2 articles // sorted by date

Diagram showing LLM quantization levels from FP16 to Q4 with corresponding VRAM requirements for a 14B model on consumer hardware

LLM Quantization Explained: What Q4, Q5, and Q8 Actually Mean for Your GPU

You pulled a model and saw Q4_K_M in the name. Here is what that means, what each level costs in VRAM and quality, and which one you should actually be running.

Diagram comparing full GPU inference speed versus partial CPU offload performance in local LLM inference on consumer hardware

AI Productivity & Workflows

LLM Inference Explained: Tokens Per Second, VRAM, and What You Can Actually Control

You ran a local model and it's generating at three tokens per second. Here is why, and what you can…

Read →