quantization Archives - Engineered AI

Inference speed optimization decision stack for local LLM on consumer GPU and Android hardware

How to Improve LLM Inference Speed Without Buying New Hardware

Your local model is slow. Before you buy anything, work through this sequence. Most inference speed problems are configuration problems, not hardware problems.

Diagram showing the three-layer local LLM stack with llama.cpp as the runtime foundation

AI Tools & Comparisons

What Is llama.cpp: The Engine Running Local AI on Your PC

If you’ve run a local LLM even once, llama.cpp was almost certainly involved. It doesn’t have a flashy interface. It…

May 5, 2026 Read →

Diagram showing LLM quantization levels from FP16 to Q4 with corresponding VRAM requirements for a 14B model on consumer hardware

AI Productivity & Workflows

LLM Quantization Explained: What Q4, Q5, and Q8 Actually Mean for Your GPU

You pulled a model and saw Q4_K_M in the name. Here is what that means, what each level costs in…

May 1, 2026 Read →

Android phone running a local LLM outdoors with no internet connection

AI Productivity & Workflows

Can You Run a Local LLM on Your Android Phone? I Tested It So I Didn’t Have to Go to My Desk

Cloud session limits are tightening. Your phone is already in your hand. Here's whether it's actually capable enough to run…

Apr 25, 2026 Read →

Four GPU cards arranged by VRAM tier showing local AI model compatibility from GTX 1660 to RTX 3090

AI Productivity & Workflows

Best Local AI Models for Your GPU: What Actually Runs at Every VRAM Tier

VRAM sets the ceiling. Architecture determines the speed. This guide maps specific Ollama model recommendations to every GPU tier from…

Apr 12, 2026 Read →

GPU inside workstation with power meter showing wattage during AI inference

AI Fundamentals & Careers

Running AI Inference on a Power Budget: GPU Wattage vs Performance Tradeoffs

Raw GPU performance benchmarks ignore power consumption. Here's how to evaluate wattage versus inference throughput for local AI workloads and…

Mar 3, 2025 Read →

Small single-board computer running local AI inference in a dark technical workspace

AI Fundamentals & Careers

Edge AI on Low-Power Devices: What Actually Runs Without the Cloud

Edge AI on low-power devices works for narrow tasks with the right model size and quantization. Here's an honest breakdown…

Feb 24, 2025 Read →

#quantization

How to Improve LLM Inference Speed Without Buying New Hardware