Token

What a token is

A token is the smallest unit a language model splits text into. It is not a letter and not a word but a frequent piece of text, usually three to four characters long. A common word is one token; a rare one is split into several. Special characters, numbers and accented letters cost more than letters because they occur less often.

The splitting explains an observation from practice: a number such as 1234 consumes more than one token in many models, while a common word takes one. Invoicing is therefore more expensive for text with many numbers and special characters than for running prose.

How cost arises from it

Cost is measured in tokens, separated into input and output. Input is more expensive than output with some providers and cheaper with others. What matters is that input does not only cover the question but everything sent along with it. A long system text, an example and retrieved passages all belong to it and are billed again with every request.

The most effective saving follows from that: cut the recurring parts. Anyone sending the same long instruction with every request pays for it with every request. Anyone limiting retrieved passages to the relevant sentences lowers cost considerably without losing quality.

What the context window limits

• Input and output together: the window limits both at once. A long contract leaves correspondingly less room for the answer.
• No memory: a model does not remember earlier calls. Every call starts fresh, which is why a conversation history is sent along and grows with it.
• Length costs quality: very long input worsens the answer, because what matters disappears among what does not. Cutting therefore helps twice.
• Limit per model: the size differs per model and changes with model versions, so it is not a fixed planning figure.

Rules of thumb for practice

Three values help with estimates. One page of text corresponds roughly to 500 to 800 tokens, depending on language. German texts need more tokens than English ones, because words are longer and compounds rarer. A customer record query with twenty fields sits at about 200 tokens, a longer product record with a description at several hundred.

The calculation for bulk work matters as well. Anyone intending to produce a thousand product texts should know the token need per text before starting. An estimate that is off by a factor of two doubles the cost. A test on a small volume therefore belongs before every bulk run.

Frequent questions

• Can I count tokens myself? Roughly yes, by characters divided by four. Exact values come only from the providers' tools.
• Why is my German text more expensive? Because German words take more tokens than English ones.
• Does a shorter answer save money? Yes, when output is billed higher than input.