What is a token in AI?
If you've looked at any AI API pricing, you've seen costs quoted "per token" or "per million tokens". Here's what a token actually is, how it maps to words, and why understanding it is the key to predicting your bill.
The short definition
A token is a chunk of text a language model reads and writes — usually a short word, part of a longer word, or a piece of punctuation. Models don't process raw letters or whole sentences; they break text into these tokens first. As a rough rule of thumb in English, 1 token ≈ ¾ of a word, so 1,000 tokens is around 750 words.
Why text is split this way
Tokenization lets a model handle any text — including names, code, and made-up words — with a fixed vocabulary. Common words are usually a single token; uncommon ones get split into pieces. Code tends to use more tokens per line than prose because of brackets, symbols and indentation, so a 1,000-line file can be 10,000+ tokens.
Input tokens vs output tokens
Providers bill two kinds of tokens separately:
- Input tokens — everything you send: your prompt, system instructions, conversation history, and any documents or context.
- Output tokens — everything the model generates back.
Output almost always costs more per token than input — often several times more — because the model produces output one token at a time, while it can read your input in parallel. That's why the length of the answer usually affects your bill more than the length of your prompt.
💡 You don't have to do the math by hand. The TokenSwarm calculator turns tokens-per-request and requests-per-day into a daily and monthly cost for any model.
How to estimate token cost
The basic formula for a single call is: (input tokens ÷ 1,000,000 × input price) + (output tokens ÷ 1,000,000 × output price). Multiply by your number of calls for a daily figure, then by thirty for a monthly estimate. A sensible habit is to add a 1.7–2× buffer for retries and overhead. For a quick word-based estimate, divide your word count by 0.75 to approximate tokens.
Estimate your cost now → Pick a model and a workload preset, and see the live per-call, daily and monthly cost.Key takeaways
- A token is a small piece of text; 1 token ≈ ¾ word in English.
- You pay for input (what you send) and output (what you get) separately.
- Output is pricier per token, so answer length drives cost.
- Code uses more tokens than prose for the same length.
- Use a calculator to turn token counts into a real monthly bill, and see how to cut costs once you understand the math.
Token-to-word ratios are approximate and vary by model and language; each provider uses its own tokenizer. For exact counts, use a provider's tokenizer; for cost, use the live TokenSwarm calculator. TokenSwarm is independent and not affiliated with any provider.