Natural Language Processing

Natural language processing (NLP) is the branch of artificial intelligence dedicated to helping computers make sense of human language, whether written or spoken. Language is one of the hardest things for machines to handle: the same word can mean different things depending on context, sentences can be ambiguous, and meaning is often implied rather than stated directly. NLP provides the tools and techniques that let software navigate this complexity.

The field has existed for decades, but it was transformed by the arrival of transformer models after 2017. Modern NLP systems can understand nuanced questions, generate fluent paragraphs of text, translate between dozens of languages, and even detect the emotional tone of a message. These capabilities sit inside the large language models that power today's AI assistants.

How it works#

NLP systems typically break language down into smaller units before doing anything with it. The first step is often tokenisation: splitting text into tokens, the individual pieces (words, sub-words, or punctuation marks) that the model processes. From there, the system analyses the relationships between tokens to build up a picture of meaning.

Modern NLP relies on techniques including:

• Parsing, which works out the grammatical structure of a sentence.
• Named entity recognition, which identifies people, places, dates, and other specific categories of word.
• Sentiment analysis, which classifies text as positive, negative, or neutral.
• Semantic similarity, which measures how close two pieces of text are in meaning, even when they use different words.
• Text generation, which produces new text that is grammatically correct and contextually appropriate.

In practice, all of these tasks are now handled by large neural networks trained on vast text datasets, rather than by hand-coded rules.

Examples#

Voice assistant commands. When you tell a voice assistant to "set a timer for 10 minutes", NLP converts your spoken words to text, identifies the intent (set a timer) and the key parameter (10 minutes), and passes a structured instruction to the device. It understands the meaning behind the words rather than looking for an exact match to a stored phrase.

Email spam filtering. A spam filter reads the text of incoming emails and uses NLP to spot patterns associated with unwanted messages: unusual sender phrasing, urgency cues, suspicious links, and topic clusters that have historically been spam. The filter learns from millions of examples and improves over time without anyone updating a list of blocked keywords by hand.

Why it matters for business#

NLP-powered tools are now embedded in many common business applications. Customer service teams use chatbots that can handle routine queries without human involvement. Marketing teams use sentiment analysis to monitor what people are saying about their products on social media. HR teams use text analysis to screen large volumes of job applications quickly.

The arrival of powerful LLMs has lowered the barrier significantly. Tools that once required specialist engineers to build can now be set up by non-technical staff using written instructions in everyday English. Learning to give those instructions well, the skill known as prompt engineering, is increasingly valuable across almost every function.

To understand how NLP and language models fit into the wider AI landscape, start with What is AI? or explore the full range of topics in the tutorials hub.