In both AI training and inference, the size of the context window determines how many tokens can be processed in a single sequence. The context window is measured in gigabytes of working memory or in number of tokens. For example, a 32GB context window specifies 32 gigabytes of available memory, whereas a 32K token window measures the amount of text it can hold. Approximately 10 thousand words or 13 thousand tokens would fit in 32GB of memory.
Using a book as an example, a chapter that takes 12,000 tokens can easily fit into a context window that holds 16,000 tokens. With a 4,000-token window, the chapter has to be processed in stages. See
long-horizon context.