Core AI Concepts
Token
A token is the unit of text an AI model processes — roughly a short word or word-fragment. Pricing, context limits, and retention all count in tokens.
LLMs read and generate text as tokens — subword chunks averaging about four characters in English. "Confidentiality" is several tokens; "the" is one. Model limits and API pricing are stated in tokens, which is why long documents hit caps that short questions don't.
For governance, tokens are a useful reminder of granularity: providers process everything submitted, token by token — there is no notion of "the model only looked at the relevant part." What's in the prompt is in the request.
Where this shows up
Related terms
See it in your own organization.
Sanitized AI inventories the AI tools in use and redacts sensitive data from prompts before it leaves.