What is a context window in a large language model?

What is a token?

A token is a piece of text the model works with, often a word fragment. Common English words are often one token, while rarer words, code and many non-English languages are split into more tokens. Limits and prices are usually counted in tokens, not words.

What happens when the window fills up?

Either the request is rejected, or older content is dropped or summarised by the application. In a long chat, early instructions can be pushed out, which is why a model may seem to forget rules you gave at the start.

Does a bigger window mean better answers?

Not automatically. Models can use information in the middle of a very long input less reliably than at the start or end, and more tokens mean higher cost and slower responses. Sending only the relevant material usually works better than sending everything.

How do you work within the limit?

Retrieve only the passages that matter, summarise earlier turns, split large documents into parts and combine the results, and keep standing instructions short and near the start. Restate key rules when a conversation runs long.

How does it affect cost and speed?

Providers generally charge per token for both input and output, and long inputs take longer to process. Reusing a stable prefix across requests, where the provider supports caching, can reduce both.

IICTP is an independent technical credentialing body.