// tag archive

#GPU inference

4 articles // sorted by date

Inference speed optimization decision stack for local LLM on consumer GPU and Android hardware

How to Improve LLM Inference Speed Without Buying New Hardware

Your local model is slow. Before you buy anything, work through this sequence. Most inference speed problems are configuration problems, not hardware problems.

Diagram showing LLM quantization levels from FP16 to Q4 with corresponding VRAM requirements for a 14B model on consumer hardware

AI Productivity & Workflows

LLM Quantization Explained: What Q4, Q5, and Q8 Actually Mean for Your GPU

You pulled a model and saw Q4_K_M in the name. Here is what that means, what each level costs in…

May 1, 2026 Read →

Diagram comparing full GPU inference speed versus partial CPU offload performance in local LLM inference on consumer hardware

AI Productivity & Workflows

LLM Inference Explained: Tokens Per Second, VRAM, and What You Can Actually Control

You ran a local model and it's generating at three tokens per second. Here is why, and what you can…

Read →

Local multimodal AI model analyzing an image on a GPU workstation

AI Fundamentals & Careers

Multimodal LLMs Locally: What Vision Models Can Actually Do on Your GPU

Local multimodal models can handle screenshot analysis, document extraction, and image description without sending data to an API. Here's an…

Feb 26, 2025 Read →