Agentic AI System Design: How to Split Work Across Agents

Once people see that one giant prompt does not scale, the next instinct is to create agents. Lots of them. A researcher agent, a writer agent, a critic agent, a manager agent to supervise the others. Six agents later the system costs more than the single prompt did, because every agent is reading the full conversation history of every other agent.

More agents is not the goal. Better boundaries is. This is the same lesson microservices taught backend teams: splitting a system badly is worse than not splitting it at all.

When a job deserves its own agent

I split work into a separate agent only when at least one of these is true:

• It needs different context. The intake step needs the customer conversation. The build step needs the brief and the knowledge base. Neither needs the other's raw material.
• It needs a different model. Classifying an email is a small model job. Planning a fifty-step build is a large model job. Running both on the large model wastes money on every classification.
• It can run in parallel. Researching five competitors is five independent jobs. Running them as parallel workers cuts wall-clock time.
• It needs a different trust level. An agent that can send email or write to a CRM should be separate from one that only reads, so its tools and permissions stay narrow.

If none of these apply, it is the same agent with a longer checklist. Anthropic's guide to building effective agents makes the same point: start with the simplest workflow that works and add agents only when they earn their place.

The handoff is the design

The single most expensive mistake in multi-agent systems is passing the full transcript from one agent to the next. The receiving agent pays to read thousands of tokens of reasoning it does not need, and it inherits every wrong turn the previous agent took.

Instead, each agent ends by producing a small structured object, defined by a schema, and that object is the only thing the next agent receives. In the landing page builder I run, the intake agent talks to the customer for as long as it takes, then writes a brief: business, offer, audience, tone, proof points. The build agent never sees the intake chat. It sees the brief. The chat might be 6,000 tokens. The brief is closer to 400.

A useful test: if you could not write the handoff as a JSON schema, the boundary between the two agents is not clear yet.

Four splits that work in practice

1. Intake, then build

A small, fast model handles the back and forth with a person, asking questions until required fields are filled. A large model receives the finished brief and does the expensive work once. The large model never pays for small talk.

2. Planner, then workers

One agent breaks a task into independent pieces and writes a short spec for each. Workers execute the specs in parallel, each with only its own spec and the tools it needs. The planner gets back short results, not the workers' full traces. Anthropic used this pattern for its own multi-agent research system.

3. Generator, then checker

One step produces, another checks. The checker should be code first: schema validation, length limits, banned words, required fields. A model checker runs only on what code cannot judge, such as tone or factual consistency with a source.

4. Router, then specialists

A cheap classifier reads the request and sends it to the one specialist that handles that type. Each specialist has a short prompt and a narrow toolset. This is how I keep voice and chat agents fast: the router decides in milliseconds, and the specialist never loads instructions for cases it will never see.

A method for drawing the boundaries

1. Write the whole process as a list of verbs, one per line.
2. Next to each verb, list the inputs it needs and the output it produces.
3. Mark the lines that are pure rules. They become code, not agents.
4. Group the remaining lines that share the same inputs and the same model size. Each group is a candidate agent.
5. For each boundary between groups, write the schema of what crosses it.
6. Decide the check that runs at each boundary before anything moves on.

This usually produces two to four agents, not ten. That is normal. The agents are small, the handoffs are cheap, and when the output is wrong you can see exactly which boundary let it through.

What to avoid

• Manager agents that only forward messages. If an agent's whole job is to pass work along, replace it with code.
• Agents that debate each other. Two models arguing in a loop burns tokens fast and rarely converges. Use a code check or a single reviewer with a clear rubric.
• Shared global context. If every agent reads the same giant context, you have one agent with extra steps and a bigger bill.

For the principles behind this, start with the overview of AI system design. The same thinking applies to the AI agents I build for clients.

A worked example: one agent becomes three

A client had a single agent that qualified leads from web forms, researched the company, wrote a personalised first email and booked a call. Its prompt was over 4,000 words and it held tools for web search, CRM read, CRM write, email send and calendar booking. It was slow, expensive, and once in a while it emailed a lead before the research step had finished.

Applying the method above produced three agents and some code:

The writing agent no longer holds the send tool, so it cannot email anyone. The research agent no longer reads the whole form. Each prompt is a few hundred words about one job, and the order of steps is enforced by code instead of by hoping the model follows instructions.

Questions to ask at every boundary

• Could the receiving agent do its job with only this object? If not, the object is missing a field.
• Does this object contain anything the receiver does not need? If so, remove it.
• What check runs on this object before it moves on?
• If this step fails, can it be retried without repeating the steps before it?

How I apply it

The same splitting method is behind the systems I run, from the outreach system that separates audience data, offer testing and sending, to the page builder that separates intake from building. If you want a process split this way, bring it to a call.

Frequently asked questions

What is agentic AI system design?

Agentic AI system design is the work of dividing a task between AI agents: deciding which job each agent owns, what context and tools it gets, which model runs it, and what structured result it hands to the next agent. Its aim is small agents with clean handoffs, rather than many agents sharing one large context.

How many agents should an AI system have?

As few as the work requires. Split only when jobs need different context, different model sizes, parallel execution or different permissions. Most production systems end up with two to four agents plus code steps, not ten or more.

How should AI agents pass information to each other?

Through small structured outputs defined by a schema, such as a JSON brief, rather than full transcripts. This keeps the receiving agent's context short, cuts token cost, and stops one agent's mistakes from spreading into the next.

Is a multi-agent system always better than a single agent?

No. Multi-agent systems cost more to run and are harder to debug. A single agent with good retrieval, structured outputs and code checks is often better. Multiple agents win when tasks are parallel or need very different context.

What is the difference between agentic AI and AI agents?

AI agents are the individual units: a model with instructions, tools and a loop. Agentic AI describes systems where agents make decisions and take actions, usually several agents and code steps working together.

Which tools should each agent get?

Only the tools it needs for its own job. An agent that drafts should not be able to send, and an agent that reads a CRM should not be able to write to it unless writing is its job. Code should check permissions on every tool call.