LLM Glossary

The process of breaking down text into smaller units called tokens that Large Language Models can understand and process. LLMs use algorithms like Byte-Pair Encoding (BPE) to split text into subword units, allowing them to handle any word, including rare or misspelled ones.

A subword tokenization algorithm that starts with individual characters and iteratively merges the most frequently occurring pairs. Used by GPT, Claude, and most modern LLMs. BPE creates a vocabulary of common subwords, balancing vocabulary size with the ability to represent any text.

Vector representations of words, sentences, or documents in a high-dimensional space. Words with similar meanings are positioned close to each other. For example, "king" and "queen" have similar embeddings. Typical dimensions range from 768 (BERT) to 12,288 (GPT-4).

A mechanism that allows Transformers to weigh the importance of different parts of the input when processing each token. It computes similarity scores between Query, Key, and Value vectors to determine which tokens to focus on. Multi-head attention enables the model to attend to different aspects simultaneously.

A variant of attention where each token in a sequence attends to all other tokens in the same sequence. This allows the model to capture dependencies between any two positions regardless of distance. Self-attention is the core building block of the Transformer architecture.

A neural network architecture introduced in the "Attention is All You Need" paper (2017). Transformers use self-attention mechanisms instead of recurrence or convolution, enabling parallel processing and capturing long-range dependencies. All modern LLMs (GPT, Claude, Llama) are based on Transformer architecture.

The process of using a trained model to generate predictions or outputs for new inputs. During inference, the model processes the input through its layers and produces tokens one at a time (autoregressive generation). Inference speed depends on model size, hardware, and optimization techniques like KV-cache.

An optimization technique that caches the Key and Value vectors from previous tokens during autoregressive generation. Without KV-cache, the model would recompute attention for all previous tokens at each step. This dramatically speeds up inference but requires significant memory.

A parameter that controls the randomness of the model's output distribution. Temperature of 0 makes the model always pick the most likely token (greedy decoding). Higher values (0.7-1.0) increase randomness and creativity. Values above 1.0 make outputs increasingly random.

A sampling method (also called nucleus sampling) that considers only the smallest set of tokens whose cumulative probability exceeds a threshold p. For example, top-p=0.9 considers tokens covering 90% of the probability mass. This dynamically adjusts the candidate pool based on the distribution shape.

A technique to reduce model size and memory usage by representing weights with fewer bits. Common formats: FP16 (16-bit, ~50% size), INT8 (8-bit, ~75% size), INT4 (4-bit, ~87.5% size). Quantization enables running large models on consumer hardware with minimal quality loss.

The process of further training a pre-trained model on a specific dataset to specialize it for a particular task or domain. Full fine-tuning updates all model weights, while LoRA (Low-Rank Adaptation) updates only a small subset, making it more efficient. Use fine-tuning when prompting isn't enough.

The ability of LLMs to perform tasks without any examples in the prompt. You simply describe what you want in natural language. Modern LLMs like GPT-4 and Claude excel at zero-shot tasks due to extensive pre-training on diverse data.

A prompting technique where you provide 2-8 examples of the desired input-output format before the actual task. The model learns the pattern from examples and applies it to new inputs. More examples typically improve performance but consume more tokens.

A prompting technique where you ask the LLM to show its reasoning step-by-step before giving the final answer. Research shows CoT can improve accuracy by 30-50% on complex reasoning tasks. Two variants: Zero-Shot CoT ("Let's think step by step") and Few-Shot CoT (providing examples with reasoning).

An enhancement to Chain-of-Thought where the model generates multiple reasoning paths for the same problem and selects the most common answer via majority voting. This reduces the impact of individual reasoning errors and improves reliability on complex tasks.

A framework that extends Chain-of-Thought by exploring multiple reasoning paths organized as a tree. At each step, the model generates several possible thoughts, evaluates them, and selects the most promising branches to continue. Enables backtracking and strategic planning.

A technique where an LLM is used to generate, refine, or optimize prompts for itself or another model. The model acts as a prompt engineer, producing better instructions than a human might write. This can be iterative, with the model improving prompts based on output quality.

A technique where the model reflects on its own output, identifies mistakes or areas for improvement, and generates a corrected response. This self-reflection loop can be repeated multiple times, progressively improving the quality of the answer.

A prompting strategy that breaks down a complex problem into a series of simpler subproblems. Each subproblem is solved in order, with solutions to earlier subproblems feeding into later ones. Particularly effective for tasks requiring compositional generalization.

A technique where the model generates executable code (e.g., Python) to solve a reasoning problem instead of performing text-based chain-of-thought. The code is then executed to get the precise answer. Especially effective for mathematical and logical reasoning tasks.

A technique where the model first generates a draft response, then creates verification questions about its claims, answers those questions independently, and revises the original response based on the verification. Reduces hallucinations and factual errors.

Retrieval-Augmented Generation — a technique that combines information retrieval with text generation. Instead of relying solely on the model's training data, RAG retrieves relevant documents from a knowledge base and includes them in the prompt. This reduces hallucinations and allows LLMs to access up-to-date information.

A technique where a complex task is broken into a series of simpler LLM calls, with the output of one call feeding into the next. Each step can use a different prompt, model, or even include validation logic. Enables building reliable pipelines for complex workflows.

Techniques for constraining LLM output to a specific format (JSON, XML, YAML, etc.). Can be achieved through prompt instructions, few-shot examples, or API features like JSON mode. Essential for integrating LLM outputs into software systems that expect structured data.

Automatic Prompt Engineering — a technique where an LLM automatically generates, evaluates, and selects optimal prompts for a given task. The model proposes multiple prompt candidates, tests them against examples, and selects the best-performing one. Removes the need for manual prompt tuning.

The practice of designing and optimizing text prompts to elicit desired behaviors and outputs from LLMs. Encompasses techniques like role assignment, few-shot examples, Chain-of-Thought, output formatting, and iterative refinement. A core skill for working effectively with AI models.

A special prompt that sets the model's behavior, role, and constraints before the user interaction begins. System prompts are typically hidden from the user and can define persona, output format, safety guidelines, and domain expertise. They persist across the entire conversation.

The maximum amount of text (measured in tokens) that a model can process in a single request, including both input and output. Modern models range from 8K to 200K+ tokens. Larger context windows allow processing longer documents but increase cost and may reduce accuracy on distant information.

The organized arrangement of elements within a prompt for optimal model performance. Key components include: role/persona, context/background, instruction/task, input data, examples, output format, and constraints. Well-structured prompts consistently outperform unstructured ones.

The practice of anchoring model responses to specific, verifiable information provided in the context. Grounded generation reduces hallucinations by requiring the model to base its answers on given documents, data, or facts rather than its parametric knowledge alone.

The practice of strategically allocating the limited context window space among different prompt components (system prompt, examples, context, user query, output). Effective token budgeting ensures the most important information fits within the context window while leaving enough room for the model to generate a complete response.

A prompting framework combining Reasoning and Acting in an interleaved manner. The model follows a loop: Thought (reasoning about the situation) → Action (calling a tool or taking a step) → Observation (processing the result) → repeat. This is the foundation of most LLM agent architectures.

A capability where the LLM can generate structured calls to predefined functions or APIs. The model receives function schemas (name, parameters, descriptions), decides when to call them, and formats the arguments correctly. This enables LLMs to interact with external systems, databases, and tools.

The fundamental execution pattern of an AI agent where it repeatedly processes input, reasons about the next step, takes an action, observes the result, and decides whether to continue or stop. The loop runs until the task is completed or a stop condition is met.

Systems that allow AI agents to store and retrieve information beyond the current context window. Short-term memory uses the conversation context, while long-term memory relies on external storage (vector databases, files). Memory enables agents to maintain state, learn from past interactions, and handle multi-session tasks.

Architectures where multiple specialized AI agents work together to solve complex problems. Agents can have different roles (researcher, coder, reviewer), communicate through messages, and coordinate through an orchestrator. Enables tackling problems too complex for a single agent.

The ability of an AI agent to break down complex tasks into manageable subtasks and create an execution plan. Planning can be done upfront (plan-then-execute) or dynamically (replan as you go). Advanced planning includes self-evaluation and plan revision based on intermediate results.

Design patterns and frameworks for building LLM-based agents. Common architectures include ReAct (reasoning + acting), MRKL (modular reasoning with tool access), LATS (Language Agent Tree Search), and plan-and-solve approaches. Each architecture optimizes for different trade-offs between reliability, flexibility, and cost.

A popular open-source framework for building applications powered by LLMs. Provides abstractions for chains (sequential LLM calls), agents (autonomous tool-using systems), memory, and retrieval. Available in Python and JavaScript. Alternatives include LlamaIndex, Haystack, and Semantic Kernel.

The ability of an LLM to interact with external tools and services to accomplish tasks beyond pure text generation. Tools can include web search, code execution, file operations, API calls, database queries, and more. Tool use is what transforms a language model into an agent.

The process of coordinating multiple AI agents, tools, and workflows to accomplish complex tasks. An orchestrator manages task distribution, agent communication, error handling, and result aggregation. Can be implemented as a dedicated agent or a programmatic workflow engine.

A security vulnerability where an attacker inserts malicious instructions into a prompt to override the model's intended behavior or system prompt. Can be direct (user input) or indirect (hidden in external data like web pages). One of the most significant security challenges for LLM applications.

Techniques used to bypass an LLM's built-in safety restrictions and content filters. Methods include role-playing scenarios, hypothetical framing, token manipulation, and multi-step escalation. Understanding jailbreaking is essential for building robust AI safety measures.

When an LLM generates text that sounds plausible and confident but is factually incorrect, fabricated, or inconsistent with the provided context. Hallucinations are an inherent limitation of current LLMs. Mitigation strategies include RAG, grounding, chain-of-verification, and temperature reduction.

Systematic prejudices in model outputs that reflect imbalances, stereotypes, or biases present in training data. Can manifest as gender bias, racial bias, cultural bias, or political bias. Addressing bias requires diverse training data, evaluation benchmarks, and ongoing monitoring of model outputs.

The practice of adversarial testing where a team attempts to find vulnerabilities, failure modes, and unsafe behaviors in an AI system. Red teaming involves crafting edge cases, adversarial inputs, and creative attack scenarios to stress-test the model's safety guardrails before deployment.

Safety mechanisms implemented around LLM applications to constrain model behavior and prevent harmful outputs. Can include input validation, output filtering, content classification, rate limiting, and human-in-the-loop review. Essential for production LLM deployments.

The use of LLMs to automatically write, complete, refactor, or translate programming code. Modern models can generate code from natural language descriptions, fix bugs, write tests, and explain existing code. Key applications include GitHub Copilot, Claude Code, and Cursor.

The task of assigning predefined categories or labels to text. LLMs can perform classification zero-shot (without examples) or few-shot (with examples). Common applications include sentiment analysis, topic categorization, spam detection, and content moderation.

The task of condensing a longer text into a shorter version while retaining the most important information. LLMs excel at both extractive summarization (selecting key sentences) and abstractive summarization (generating new text that captures the essence). Can be customized for length, style, and focus.

The process of extracting structured information from unstructured text. Includes named entity recognition (NER), relation extraction, event extraction, and key-value pair extraction. LLMs can perform these tasks zero-shot with structured output formats like JSON.

A subtask of information extraction that identifies and classifies named entities in text into predefined categories such as person names, organizations, locations, dates, monetary values, etc. LLMs can perform NER zero-shot and handle complex, nested, or ambiguous entities.

The task of generating accurate answers to natural language questions. Can be open-domain (using general knowledge) or closed-domain (based on provided context/documents). RAG-enhanced QA systems combine retrieval with generation for more accurate, grounded answers.

The task of determining the emotional tone or opinion expressed in text. Can range from simple polarity (positive/negative/neutral) to fine-grained analysis of specific emotions, aspects, or intensity. LLMs provide nuanced sentiment analysis without task-specific training.

Using LLMs to create synthetic data for various purposes: training data augmentation, test case generation, content creation, and simulation. LLMs can generate diverse, realistic data samples that follow specified patterns, constraints, and distributions.

A model that can jointly process and reason about both visual (images) and textual data. VLMs can describe images, answer questions about visual content, and perform tasks that require understanding both modalities. Examples include GPT-4V, Claude Vision, and Gemini.

The application of vision-language models to interpret, describe, and extract information from images. Capabilities include object detection, scene description, text extraction (OCR), chart/graph analysis, and visual question answering. Critical for document processing, accessibility, and content moderation.

Optical Character Recognition — the technology for converting text in images, scanned documents, or photographs into machine-readable text. Modern LLMs with vision capabilities perform OCR as part of image analysis, handling complex layouts, handwriting, and multiple languages.

An AI agent that communicates through spoken language, combining speech-to-text (STT), language model processing, and text-to-speech (TTS). Modern voice agents can handle real-time conversations with natural-sounding speech, emotion detection, and turn-taking.

The ability of AI models to analyze and reason about video content by processing frames, understanding temporal relationships, and describing actions and events. Current approaches typically sample key frames and analyze them with vision-language models.

AI systems capable of processing and generating multiple types of data (modalities) including text, images, audio, and video. Multimodal models can understand cross-modal relationships and perform tasks that require reasoning across different data types simultaneously.

The process of choosing the most appropriate LLM for a specific use case based on factors like task complexity, latency requirements, cost constraints, quality needs, and data privacy. Often involves benchmarking multiple models and considering trade-offs between capability and efficiency.

Standardized evaluation datasets and metrics used to measure and compare LLM performance. Common benchmarks include MMLU (general knowledge), HumanEval (code generation), GSM8K (math reasoning), and HellaSwag (common sense). Benchmarks help in model selection but don't capture all aspects of real-world performance.

A specialized database designed to store, index, and query high-dimensional vector embeddings efficiently. Used in RAG systems to find semantically similar documents. Popular options include Pinecone, Weaviate, Qdrant, ChromaDB, and pgvector (PostgreSQL extension).

The practice of monitoring and understanding the behavior of LLM applications in production. Includes tracking prompts, responses, latency, token usage, costs, error rates, and output quality. Tools like LangSmith, Helicone, and Weights & Biases provide LLM-specific observability features.

Strategies and techniques to reduce the cost of running LLM applications. Includes prompt caching, model routing (using cheaper models for simple tasks), prompt compression, batch processing, response caching, and optimizing token usage. Critical for scaling LLM applications.

Common architectural patterns for integrating LLM APIs into applications. Includes streaming responses, request batching, retry logic with exponential backoff, rate limiting, request queuing, and fallback to alternative models. These patterns ensure reliable and efficient LLM API usage.

Strategies and infrastructure for deploying LLM applications to production. Options include cloud API providers (OpenAI, Anthropic), self-hosted models (vLLM, TGI), edge deployment (ONNX, TensorRT), and hybrid approaches. Key considerations: latency, cost, data privacy, and availability.

The time delay in LLM applications between sending a request and receiving a response. Key metrics include Time to First Token (TTFT) and total generation time. Latency depends on model size, input/output length, server load, and network. Optimization techniques include streaming, caching, and smaller models.

An optimization where the model provider caches the computed internal representations of frequently used prompt prefixes. When the same prefix is sent again, the cached computation is reused, reducing both latency and cost. Supported by Anthropic, OpenAI, and others for system prompts and common prefixes.

A technique where the model's output is sent to the client token by token as it's generated, rather than waiting for the complete response. This dramatically improves perceived latency and user experience. Implemented via Server-Sent Events (SSE) or WebSockets in most LLM APIs.

Model Context Protocol — an open standard developed by Anthropic for connecting LLMs to external data sources and tools. MCP provides a unified protocol for tool discovery, invocation, and data access, enabling any LLM application to connect to any MCP-compatible server.

A service that implements the Model Context Protocol to expose tools, resources, and prompts to LLM applications. MCP servers can provide access to databases, APIs, file systems, or any other external capability. They can be local processes or remote services.

A child agent spawned by a main (parent) agent to handle a specific subtask independently. Sub-agents can work in parallel, have their own tool access, and return results to the parent. Used in Claude Code for parallelizing research, code review, and complex multi-step tasks.

A user-defined agent in Claude Code with customized system prompts, tool access, and behavioral instructions. Custom agents are defined in .claude/agents/ directory and can be specialized for specific tasks like code review, testing, or documentation. They extend the base agent capabilities with domain-specific knowledge.

Shell commands configured to execute automatically in response to specific Claude Code events (e.g., before/after tool calls, on session start). Hooks enable custom workflows like auto-formatting, linting, notification, and safety checks without manual intervention.

Reusable prompt templates in Claude Code that can be invoked via slash commands (e.g., /commit, /review). Skills encapsulate common workflows into a single command, making repetitive tasks faster and more consistent. Users can create custom skills for their specific needs.

A markdown file placed in the project root that provides persistent instructions, conventions, and context to Claude Code across all sessions. Acts as the project's "memory" — containing architecture decisions, coding standards, important paths, and workflow preferences.

The core execution loop of Claude Code where it reads the context (files, conversation), reasons about the next action, uses tools (file read/write, bash, search), observes results, and repeats until the task is complete. Each iteration is called a "turn" and consumes API tokens.

A programmatic interface that allows running Claude Code as a subprocess from scripts and applications. The SDK enables automation of code generation, review, and refactoring tasks, integration into CI/CD pipelines, and building custom tools on top of Claude Code's capabilities.

The set of tools available to Claude Code by default for interacting with the local environment. Core tools include: Read (file reading), Write (file creation), Edit (file modification), Bash (command execution), Glob (file search), Grep (content search), and Task (sub-agent spawning).