What is a token in AI?
If you've looked at any AI API pricing, you've seen costs quoted "per token" or "per million tokens". Here's what a token actually is, how it maps to words, and why understanding it is the key to predicting your bill.
The short definition
A token is a chunk of text a language model reads and writes, usually a short word, part of a longer word, or a piece of punctuation. Models don't process raw letters or whole sentences; they break text into these tokens first. As a rough rule of thumb in English, 1 token ≈ ¾ of a word, so 1,000 tokens is around 750 words.
Why text is split this way
Tokenization lets a model handle any text, including names, code, and made-up words, with a fixed vocabulary. Common words are usually a single token; uncommon ones get split into pieces. Code tends to use more tokens per line than prose because of brackets, symbols and indentation, so a 1,000-line file can be 10,000+ tokens.
Input tokens vs output tokens
Providers bill two kinds of tokens separately:
- Input tokens, everything you send: your prompt, system instructions, conversation history, and any documents or context.
- Output tokens, everything the model generates back.
Output almost always costs more per token than input, often several times more, because the model produces output one token at a time, while it can read your input in parallel. That's why the length of the answer usually affects your bill more than the length of your prompt.
You don't have to do the math by hand. The TokenSwarm calculator turns tokens-per-request and requests-per-day into a daily and monthly cost for any model.
How to estimate token cost
The basic formula for a single call is: (input tokens ÷ 1,000,000 × input price) + (output tokens ÷ 1,000,000 × output price). Multiply by your number of calls for a daily figure, then by thirty for a monthly estimate. A sensible habit is to add a 1.7–2× buffer (based on typical overhead observed across providers) for retries and overhead. For a quick word-based estimate, divide your word count by 0.75 to approximate tokens.
Estimate your cost now → Pick a model and a workload preset, and see the live per-call, daily and monthly cost.Key takeaways
- A token is a small piece of text; 1 token ≈ ¾ word in English.
- You pay for input (what you send) and output (what you get) separately.
- Output is pricier per token, so answer length drives cost.
- Code uses more tokens than prose for the same length.
- Use a calculator to turn token counts into a real monthly bill, and see how to cut costs once you understand the math.
Token-to-word ratios are approximate and vary by model and language; each provider uses its own tokenizer. For exact counts, use a provider's tokenizer; for cost, use the live TokenSwarm calculator. TokenSwarm is independent and not affiliated with any provider.