AI Agent Architecture

The loop

The classic agent loop is observe, think, act, observe again. The application sends a goal and context to the model. The model chooses an action, usually a tool call. Your code validates the arguments, executes the tool, records the result, and returns the observation. The loop continues until the agent reaches a completion criterion, asks for help, or hits a stop condition.

ReAct is still the clean mental model because it interleaves reasoning and acting rather than treating action as a one-shot result. Modern products should avoid exposing hidden chain-of-thought, but the architectural idea remains useful: separate the model's decision, the tool execution, and the observed result.

Tools are contracts, not decorations

A tool should be a narrow, typed contract. Give it a name, argument schema, description, permission level, timeout, retry policy, and audit behavior. Avoid giant tools like "run_admin_task" or "do_everything." Broad tools make the model harder to guide and your logs harder to interpret.

The Model Context Protocol makes this boundary more standard by separating resources, prompts, and tools. That does not remove the need to design permissions. It simply gives agents a common way to discover and call capabilities exposed by servers.

State and memory are different

State is what the agent needs to finish the current run: user request, messages, current plan, tool outputs, errors, and progress markers. Memory is what the system may use across runs: user preferences, durable facts, previous decisions, account settings, or domain-specific notes. Mixing the two is a common failure mode. The agent remembers too much, forgets the current state, or cites old facts as if they were live truth.

LangGraph's checkpointer/store distinction is a useful way to think about it. Checkpoints support resumable threads and human review. Stores support longer-lived memory. A production design should say which facts go where, when they expire, and how users can inspect or delete them.

Planning should be bounded

Planning is valuable when a task has several possible routes. It is harmful when it becomes free-form busywork. Use small plans with explicit checkpoints: identify missing context, choose a tool, perform one action, inspect result, update status. For longer tasks, store the plan in structured state so the application can enforce maximum steps, retries, and cost budgets.

Guardrails and observability are core components

Guardrails check the system before damage happens. They can inspect input, retrieved context, tool arguments, tool results, final output, or the whole trace. Observability lets you understand what happened after the run. You want model inputs and outputs, tool arguments, tool results, latency, token usage, cost, policy decisions, approvals, and final status in one trace.

The minimum trace answers five questions: what was the goal, what did the agent know, what did it call, what did the world return, and why did the run stop? If you cannot answer those questions, you cannot debug the agent.

Stop conditions matter

Every agent needs a stop policy. Stop after success. Stop after maximum steps. Stop after repeated tool errors. Stop when the next action requires permission. Stop when confidence is low. Stop when cost or latency crosses the budget. A model that keeps trying is not resilient by default; it may simply be spending money while making the state less reliable.