#tokens per second

1 article // sorted by date

Diagram comparing full GPU inference speed versus partial CPU offload performance in local LLM inference on consumer hardware

LLM Inference Explained: Tokens Per Second, VRAM, and What You Can Actually Control

You ran a local model and it's generating at three tokens per second. Here is why, and what you can actually do about it.

May 1, 2026 · 18 views Read →

EngineeredAI

Real tools. No hype. AI is just another tool — this is where I engineer with it.

Navigate

Home About Blog Contact Links Vault Substack

Topics

AI Applications AI Development & Coding AI Fundamentals & Careers AI Productivity & Workflows AI Tools & Comparisons Content Creation & Writing Critical Analysis Prompt Engineering

Legal

Advertise Media Kit Disclosure DMCA Privacy Terms

Sitemap

Manage Consent

To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.

Functional Functional Always active

The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.

Preferences Preferences

The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.

Statistics Statistics

The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.

Marketing Marketing

The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.

Manage options
Manage services
Manage {vendor_count} vendors
Read more about these purposes

View preferences

{title}
{title}
{title}