Back to Glossary
Infrastructure

Tokens

The small chunks of text that AI models process — roughly 3/4 of a word each.

Share

Definition

Tokens are the basic units that AI language models use to process text. A token is roughly 3/4 of a word in English — so "hello" is one token, "chatbot" might be split into "chat" and "bot" (two tokens), and a 1,000-word document is roughly 1,300 tokens. Tokens matter because AI models have limits on how many tokens they can process at once (the context window), and most AI APIs charge based on the number of tokens used.

Understanding tokens helps you estimate costs, stay within context limits, and optimize your prompts. Shorter, more efficient prompts save money and leave more room for the AI's response.

Examples

1

The sentence "I love artificial intelligence" is approximately 5 tokens

2

A typical blog post of 2,000 words is roughly 2,600 tokens — important for cost estimation when using AI APIs

Related Terms

Frequently Asked Questions

How do I count tokens?
Most AI platforms show token counts in their interface or API responses. OpenAI has a free tokenizer tool at platform.openai.com/tokenizer. As a rough estimate, multiply your word count by 1.3.
Why do tokens matter for prompts?
Every token in your prompt takes up space in the context window. Efficient prompts leave more room for the AI's response and cost less when using paid APIs.

Build prompts using this concept

Explore our prompt library and put tokens into practice with ready-to-use templates.

Build prompts using this concept