self-host vs api

Self-host or pay per token?

Running an open model on your own GPU has a fixed cost; an API charges per token. There is a volume where one overtakes the other. Enter your numbers and find the break-even, with no marketing spin.

API side

Reference model (pulls live API price)

Input price ($ / 1M tokens)

Output price ($ / 1M tokens)

Output share of your tokens (%)

the rest are input tokens

Self-host side

GPU cost ($ / hour)

e.g. ~$1-3/hr for a rented A100 / H100

Throughput (output tokens / second)

a 70B on one A100 with vLLM is roughly 1000-3000

Engineering overhead ($ / month)

setup, monitoring, on-call (often $750-3000 in reality)

Your expected volume (tokens / day)

Live API price by OpenRouter. The math is yours to check: self-host cost/day = GPU $/hr × 24 + engineering/30; break-even volume = self-host cost/day ÷ blended API price per token. This ignores model-quality differences, network egress, and idle time, so treat it as a starting point, not gospel.