gpu vram

Will this model run on my GPU?

Estimate the GPU memory needed to self-host an open model. Weights are exact; KV cache and overhead are estimated, since they depend on the model's architecture and serving framework.

Model size (billion parameters)

e.g. Llama 70B = 70, Qwen 32B = 32, 8B = 8

Quantization

lower = smaller, slightly lower quality

Context length (tokens)

Concurrent requests

how many users served at once

The math: weights = parameters × bytes-per-parameter (exact). KV cache ≈ context × concurrency × size, scaled for the model (estimate). Overhead ≈ 1.5 GB for the runtime. Real KV cache depends on layers, attention type (GQA/MQA) and framework (vLLM PagedAttention can cut waste a lot), so treat this as a planning estimate and keep 10-15% headroom.