AI glossaryDefinition

Token

A small unit of text that AI language models use to process and generate language. A token can be a word, part of a word, or a punctuation mark.

Related terms

Large Language Model Natural Language Processing Transformer

Go beyond the jargon

Follow a structured learning path

Instead of learning terms one by one, work through a guided path that builds your AI knowledge step by step. Every path is free and includes a certificate.

Browse learning paths

Back to glossary