The vocabulary
What the words on the rest of this site mean — what you are billed for, how the things work, and how they are sold. Each entry answers in its first sentence.
A
- AgentA program that runs a model in a loop with tools, deciding what to do next until the job is done.
- Agent memoryWhat an agent carries between runs, as opposed to what fits in one context window.
- Agent2AgentAn open protocol for agents built by different vendors to discover one another and hand over work.
- AlignmentThe work of making a model's behaviour match what its makers intend.
- API keyThe string that identifies you to a seller and against which you are billed.
- ARC-AGIVisual puzzles where each task follows a rule the solver has never seen, designed so memorisation cannot help.
- AttentionThe mechanism by which a model decides, for each word it is producing, which earlier words to weigh.
- AutoregressiveProducing an answer one token at a time, each conditioned on all the ones before.
B
- BatchA queue: you hand over the work, the seller runs it when there is room, and you pay about half.
- BenchmarkA fixed set of questions every model is asked, so their answers can be compared.
- Benchmark contaminationA benchmark's questions having leaked into a model's training data, so its score measures memory rather than ability.
- BM25The classic keyword search ranking function: rare words that match count for more than common ones.
C
- C2PAA standard for attaching tamper-evident provenance to a file: what made it, when, and what was done to it since.
- ChunkingCutting documents into pieces small enough to embed and retrieve usefully.
- Cold startThe delay when a model has to be loaded onto a GPU before it can answer, because nobody has asked for it recently.
- Committed spendA discount in exchange for promising a minimum over a year.
- Computer useA model driving a screen: it is shown a screenshot, and it replies with clicks, keystrokes and scrolls.
- Context windowHow much a model can hold in mind at once — everything you send plus everything it writes back, counted in tokens.
- Continuous batchingSlotting new requests into a running batch as older ones finish, instead of waiting for the whole batch to complete.
- Copyright indemnityA promise by the seller to defend you if what their model produces is claimed to infringe someone's copyright.
- CreditA seller's own unit of account, bought in advance and spent at rates the seller sets.
D
- Data processing agreementThe contract governing what a seller may do with the data you send it.
- Data residencyA requirement that data be stored and processed inside a named country or region.
- DeprecationA seller withdrawing a model, usually with a few months' notice.
- DiarisationWorking out who spoke when, so a transcript is split by speaker.
- Diffusion modelA model that starts from noise and removes it step by step until an image, a video or a sound emerges.
- DistillationTraining a small model to imitate a large one, so it answers nearly as well at a fraction of the cost.
- DPOA cheaper way to train on preferences that skips the reward model and optimises the model directly against pairs of better and worse answers.
E
- EgressWhat a cloud charges to move data out of it.
- EloA rating derived from head-to-head wins, borrowed from chess and used where quality cannot be scored directly.
- EmbeddingA list of numbers standing for a piece of text, arranged so that similar meanings sit close together.
- EU AI ActThe European Union's law on artificial intelligence, which sorts systems by risk and attaches duties to each level.
- Eval setYour own collection of cases with known good answers, used to decide whether a change helped.
F
- Fine-tuningContinuing a model's training on your own examples so it adopts a format, a tone or a domain.
- Free tierAn allowance a company gives away, capped by requests, tokens or time.
- Frontier modelThe most capable model a maker sells, priced accordingly.
- Function callingThe model replies with a structured request to run one of your functions, instead of with prose.
G
- GatewayA company that resells many providers' models behind one API and one bill.
- GDPRThe European data protection regulation, which applies whenever you process personal data about people in the EU.
- GGUFThe file format for a quantised model meant to be run locally, holding the weights and everything needed to load them in one file.
- GPQAGraduate-level science questions written to be hard for anyone who is not a specialist, and unsearchable on the web.
- GPU hourOne hour of one accelerator, the unit underneath every per-token price.
- GuardrailA separate, usually small model that inspects what goes in or comes out and blocks what should not pass.
- Guidance scaleA dial on an image model controlling how literally it follows the prompt.
H
- HallucinationA confident, fluent, wrong answer.
- HIPAAThe United States law governing health information.
- HTTP 402The HTTP status code reserved in 1997 for "Payment Required" and left unused for twenty-five years.
- Human in the loopRequiring a person to approve before an agent does something it cannot undo.
- HumanEval164 small Python functions the model must write from a docstring, graded by running the tests.
- Hybrid searchRunning keyword search and vector search together and merging the two result lists.
- HyDEAsking a model to write a hypothetical answer, then searching with that instead of with the question.
I
- Image tokensThe unit an image becomes when a model reads it: a picture is cut into patches, and each patch costs like a token.
- InferenceRunning a trained model to get an answer — as opposed to training, which produced it.
- InpaintingRegenerating part of an image while leaving the rest untouched.
- Instruction tuningTeaching a base model to follow instructions rather than merely continue text.
K
L
- Late interactionKeeping one embedding per token instead of one per document, and comparing them individually at query time.
- Lip syncReshaping a face in video so its mouth matches supplied audio.
- llama.cppA small, dependency-free engine for running models on ordinary computers, including on a laptop's CPU.
- LLM as judgeUsing a strong model to grade another model's answers against a rubric.
- LMArenaA site where people compare two anonymous answers and vote, producing an Elo rating for each model.
- LoRAFine-tuning by training a small pair of matrices alongside the frozen model instead of changing the model itself.
M
- MicropaymentA payment small enough that conventional card fees would exceed it.
- Mixture of expertsA model split into many specialised parts, of which only a few run for any given token.
- MMLUA multiple-choice exam of about 16,000 questions across 57 subjects, from law to microbiology.
- Model Context ProtocolAn open standard for connecting a model to tools and data, so a client and a tool server that have never met can work together.
- MultimodalA model that takes or returns more than text — images, audio, video, files.
O
- OCRReading text out of an image or a scan.
- Open weightsThe trained parameters are published, so you can download the model and run it on your own machines.
- OpenAI-compatible APISpeaking the shape of OpenAI's chat endpoint, so that code written for one seller runs against another by changing a base URL.
- Output tokenA token the model writes, priced several times higher than one you send.
P
- ParametersThe trained numbers that constitute a model, counted in billions.
- pass@kThe chance that at least one of k attempts is correct.
- Per million tokensThe standard way an API model is priced: dollars per million tokens, quoted apart for input and output.
- PretrainingThe first and by far the most expensive stage: reading a very large body of text to learn language and world knowledge.
- Prompt cachingPaying once to have the model remember the beginning of your prompt, then paying a fraction of the rate each time you reuse it.
- Prompt injectionText inside a document, a web page or a tool result that the model reads as an instruction rather than as content.
- Provisioned throughputRenting a fixed slice of a seller's hardware by the hour or the month instead of paying per token.
Q
- QuantisationStoring a model's numbers with less precision so it fits in less memory and runs faster.
R
- Rate limitA ceiling on how much you may send per minute or per day, independent of what you are willing to pay.
- ReActThe basic agent loop: think, act with a tool, read the result, repeat.
- Realtime APIA streaming, two-way connection carrying audio in and audio out, so a model can hold a spoken conversation.
- Reasoning modelA model trained to work through a problem in steps before answering, spending tokens on thinking that you pay for and usually do not see.
- Recall@kThe share of the right answers that appear in the first k results.
- Red teamingDeliberately attacking your own system to find what makes it misbehave before somebody else does.
- RefusalA model declining to answer.
- RerankingA second pass that re-orders search results by reading the query and each candidate together.
- Retrieval-augmented generationFinding the relevant documents first and putting them into the prompt, so the model answers from your material rather than from memory.
- RLHFTraining a model on human preferences: people rank two answers, a reward model learns the pattern, and the model is tuned to score well against it.
- RoPEThe usual way a model is told where each token sits in the sequence, by rotating its representation by an angle proportional to its position.
S
- SafetensorsThe standard format for distributing model weights, designed so that loading a file cannot execute code.
- SeatA licence for one named person, billed monthly.
- SeedThe number that fixes the randomness, so the same prompt and the same seed give the same result.
- Self-hostingRunning an open-weight model on hardware you control.
- Service level agreementA contractual promise about uptime and support, with money attached when it is missed.
- Silent updateThe model behind a name changing without the name changing.
- Sliding-window attentionLetting each token attend only to a few thousand neighbours instead of the whole sequence, so cost grows with length rather than with its square.
- Small language modelA model small enough to run on ordinary hardware, typically under about 15 billion parameters.
- SOC 2An audit report on how a company handles security, availability and confidentiality.
- Speculative decodingA small model guesses the next several tokens and the large one checks them all at once, which is faster than generating them one at a time.
- Speech generationTurning text into speech, sold by the character, the second or the token depending on the seller.
- Speech recognitionTurning speech into text, usually sold by the minute or the second of audio.
- State space modelAn alternative to attention that carries a fixed-size summary forward, so cost grows in a straight line with length.
- Structured outputForcing the model's answer to match a schema you supply, so it parses every time.
- SubagentAn agent spawned by another to do one bounded piece of work and report back.
- SWE-benchReal GitHub issues from real repositories: the model must produce a patch that makes the project's own tests pass.
- Synthetic dataTraining data written by another model rather than by people.
T
- Time to first tokenHow long you wait before the answer starts appearing.
- TokenThe unit a language model reads and writes, and the unit almost all of them are billed by: roughly three-quarters of an English word.
- Token budgetA ceiling you set on how much a single task may spend.
- Tokens per secondHow fast the answer streams once it has started.
- TransformerThe architecture nearly every modern model uses: layers that let each token look at every other token and decide which ones matter.
V
- Vector databaseA store that finds the nearest embeddings to a query rather than exact matches.
- Vision-language modelA model that reads images alongside text — screenshots, documents, charts, photographs.
- vLLMThe most widely used open server for running language models, notable for handling many concurrent users on one GPU efficiently.
- Voice activity detectionDeciding, moment to moment, whether anyone is speaking.
- Voice cloningBuilding a synthetic voice from a sample of a real one, sometimes from seconds of audio.