Context Engineering vs Prompt Engineering: What the Model Sees Is the Design

Prompt engineering asks: how should I word this? Context engineering asks a bigger question: of everything I could show the model right now, what should it actually see?

The second question matters more as systems grow. A single chatbot prompt can be tuned by hand. An agent that runs thirty steps, calls tools and pulls from a knowledge base is assembling a new context on every turn. If nobody designs that assembly, it fills up with stale history, duplicated documents and tool output nobody needed.

What goes into a context window

• System instructions. The job and the rules.
• Tool definitions. Names, descriptions and schemas for each tool.
• Retrieved knowledge. Documents or entries fetched for this request.
• Memory. Facts carried from earlier sessions.
• History. Previous turns in this conversation or run.
• Tool results. Output from calls made during this run.
• The current request.

Every one of these competes for the same window and the same budget. Anthropic's engineering team calls context a finite resource with diminishing returns: past a point, adding tokens makes the model worse at finding what matters.

The rules I use

Load by reference, not by copy

Give the model an index and a search tool instead of the whole knowledge base. In the page builder I run, the knowledge base has around 850 entries, but the model sees a single index of short "use when" lines and fetches the two or three entries it needs. The rest never enter the window.

Summarise history, do not replay it

After a handful of turns, replace older history with a short summary of decisions made and facts learned. The model needs the conclusions, not the path to them.

Trim tool output

A tool that returns a full web page or a 500-row query result floods the window. Make tools return only the fields the agent asked for, or a short summary with a reference to fetch more.

Put stable things first

Instructions and tool definitions rarely change, so place them at the start where prompt caching can reuse them. Variable content goes last.

Give each agent its own context

When work is split across agents, each one gets a fresh window with only its brief. See how to split work across agents.

Context engineering vs prompt engineering

They are not rivals. Clear instructions still matter. But in a system with tools and retrieval, the context assembly code decides more about cost and quality than the instruction wording does. It sits at the centre of AI system design.

A worked example: shrinking one call

Here is a context audit I ran on one agent call, before and after. The numbers are rounded but real in proportion.

Output quality went up, not down. With less noise, the model followed the remaining instructions more consistently.

How to run your own context audit

1. Log the full assembled prompt for twenty real calls.
2. Label every block by type: instructions, tools, knowledge, memory, history, tool results, request.
3. Count the tokens per type.
4. For each block ask: did the output use this? If not on most calls, remove it or make it fetchable.
5. Re-run your test set and compare quality and cost.

Where I use it

Context engineering is the main reason the systems on my systems page stay affordable at volume. It is also the idea behind the line on my homepage: the best AI systems are designed, not prompted.

Context engineering skills worth learning

• Retrieval design: chunking, labelling and indexing knowledge so the right piece can be found cheaply.
• Tool design: tools that return small, specific results instead of raw dumps.
• Memory design: what to store as durable facts and how to load it as a summary.
• Compaction: summarising long histories into decisions and facts.
• Prompt layout: ordering blocks so stable parts can be cached.
• Measurement: logging the assembled context per call and counting tokens per block.

Context engineering for Claude and other models

The principles are model-independent, but details differ by provider: how caching is triggered, how tool definitions are counted, how long the window is and how well the model uses information deep in it. Test your layout on the model that will run it. A long window is not a reason to fill it. Even models with very large windows do better with a short, relevant context.

Frequently asked questions

What is context engineering?

Context engineering is designing what a language model sees on each call: instructions, tool definitions, retrieved knowledge, memory, history and tool results. The aim is the smallest set of relevant tokens that lets the model do the current step well.

What is the difference between context engineering and prompt engineering?

Prompt engineering improves the wording of instructions. Context engineering decides which information enters the context window, in what order and at what size, and is usually done by code on every call. Prompt engineering is one part of context engineering.

Why does context engineering reduce cost?

Models charge for every input token on every call. Loading only relevant knowledge, summarising history, trimming tool output and caching stable instructions all cut the tokens sent per call, often by more than half.

What is an example of context engineering?

Replacing a full knowledge base in the prompt with a short index and a fetch tool, so the model loads only the two or three entries it needs. Summarising old conversation turns and trimming tool output are other examples.

Is context engineering only for AI agents?

No. Any application that sends context to a model benefits, including single-call features like summarisation or classification. It matters most for agents because they assemble new context on every turn.

Do large context windows make context engineering unnecessary?

No. A larger window lets you send more, but every token still costs money and adds noise. Models generally perform better and cheaper with a short, relevant context than with a full window.