AI Solution Architecture: From a Business Process to a System of Agents

Clients rarely come to me asking for an AI solution architecture. They come with a process that hurts: leads that sit for two days before anyone calls, invoices typed by hand from PDFs, support tickets answered with the same five replies. The architecture is how that pain becomes a system.

Step 1: Map the process as it really runs

Not the process in the policy document. The one people actually follow, including the spreadsheet someone keeps on the side. Write each step as a verb with an input and an output: receive email, find the customer, check the order, write the reply, send it, log it.

Step 2: Split rules from judgement

Mark each step as rule (can be written as code) or judgement (needs reading, interpreting or writing). Finding a customer by email is a rule. Deciding whether an angry email is a refund request or a complaint is judgement. Rules become code, which is cheap and predictable. Only judgement steps are candidates for a model.

Step 3: Decide the model steps

For each judgement step, ask how hard the judgement is. Classifying into five categories is a small model task. Writing a careful reply using policy and order history is a larger model task. Grouping steps that need the same context and model size gives you your agents, as described in agentic AI system design.

Step 4: Define the data at every boundary

Write the schema of what each step receives and returns. This is where integrations show up: the CRM record, the order from the ERP, the policy from the knowledge base. Each integration is a tool or a code step with its own error handling. In my experience, most of the build time goes here, not into prompts.

Step 5: Place checks and people

After every model step, a code check. Before every irreversible action, decide whether a person approves it, at least for the first weeks. Approval can be relaxed later once the logs show the step is reliable.

Step 6: Plan cost and observability

Estimate calls per day per step, tokens per call and model price. This turns a vague worry about AI cost into a number. Then decide what gets logged per step so you can see cost, latency and failure rate from day one.

What the architecture document should contain

• The process map with each step marked as code, model or person.
• The agent list with each agent's job, model, tools and input and output schemas.
• The integration list with authentication, rate limits and failure handling.
• The check list: what is validated after each step.
• The cost estimate and the spend caps.
• The rollout plan: shadow mode, approval mode, then autonomous.

This is AI system design applied to a real business, and it is the same approach I use on the CRM automation projects I build.

A worked example: invoice intake

A finance team receives supplier invoices as PDFs by email and types them into an accounting system. Applying the six steps:

1. Map: receive email, save PDF, read supplier, invoice number, date, lines, tax, total, match to purchase order, enter into accounting, file the PDF.
2. Rules vs judgement: saving, matching by PO number, entering and filing are rules. Reading an unstructured PDF is judgement.
3. Model steps: one extraction step on a mid-size model with a strict schema.
4. Data: the schema for an invoice; the accounting API's fields and rate limits; the purchase order lookup.
5. Checks and people: line totals must add up, tax must match the rate, the supplier must exist, the PO must match. Any failure goes to a person with the PDF and the extracted fields side by side.
6. Cost: one model call per invoice. At a few hundred invoices a month the model cost is small; the value is in the hours saved and the errors caught.

Only one step uses a model. That is a typical result. Good AI solution architecture often has fewer AI calls than people expect, placed exactly where they are needed.

Red flags in an AI architecture proposal

• Every step is labelled "AI".
• No schemas between steps.
• No mention of what happens when the model is wrong.
• No cost estimate per month.
• No rollout plan with a human review phase.

If you are comparing proposals, my guide on how to evaluate AI agent development companies covers the questions to ask.

How I work

This is how every engagement I take starts: process first, architecture second, prompts last. The systems on my systems page came out of exactly this method, and you can book a call to walk through a process of your own.

Choosing models, tools and platforms

Pick technology last. Once the process map and schemas exist, the choices are usually clear: a workflow tool if integrations dominate, custom code if the product depends on cost and context control, hosted models by default, and the CRM or ERP the business already runs as the system of record. Architectures that start from a platform choice tend to bend the process to fit the tool.

AI solution architecture deliverables

1. A one-page summary of the job, the benefit and the risk.
2. The process map with code, model and person steps.
3. An agent and step specification with schemas.
4. The integration list with access requirements.
5. A monthly cost estimate.
6. A rollout and monitoring plan.

A client should be able to read the first page and decide, and a developer should be able to build from the rest.

Frequently asked questions

What is AI solution architecture?

AI solution architecture is the design of a system that solves a business problem with a mix of code, AI models and people. It covers which steps use a model, how data moves between steps, which integrations are needed, what is checked, and how cost and failures are handled.

How do you design an AI solution?

Map the real process, separate rule-based steps from judgement steps, assign models only to judgement steps, define the data schemas between steps, add code checks and human approval where mistakes are costly, then estimate cost and plan logging.

What should an AI architecture document include?

A process map, the list of agents with models, tools and schemas, the integrations, the validation checks, a cost estimate with spend caps, and a rollout plan from shadow mode to autonomous operation.

What does an AI solutions architect do?

An AI solutions architect maps a business process, decides which steps should use AI models and which should be code or people, designs the data flow and integrations, and plans checks, cost and rollout.

How long does it take to design an AI solution?

For a single business process, mapping and architecture usually take days, not months. Building and rolling out with a review phase typically takes a few weeks, depending mostly on the integrations involved.

Should I choose the AI platform first or design first?

Design first. Map the process, define the steps and schemas, then choose platforms and models that fit. Starting from a platform tends to bend the process around the tool's limits.