The Anatomy of an Agent
Under the surface, every AI agent — no matter how sophisticated — is built from a handful of core components. Understanding these components gives you the vocabulary to evaluate agent products, ask the right questions, and spot the difference between genuine capability and marketing.
The Brain: Foundation Models
At the center of every agent is a large language model. This is the reasoning engine — the component that interprets goals, decides what to do next, and generates responses. The choice of model shapes everything about an agent's capability.
Different models bring different strengths. Some excel at coding tasks. Others are better at nuanced reasoning or following complex instructions. The model is not the agent, but it sets the ceiling on what the agent can achieve.
The Hands: Tools and Functions
Tools are what give agents the ability to act. Without tools, a language model can only generate text. With tools, it can:
- Search the web for current information
- Query databases for specific data
- Call APIs to interact with services
- Read and write files on a filesystem
- Execute code to perform calculations or transformations
- Send messages via email, Slack, or other channels
Each tool is essentially a function that the agent can invoke. The agent decides when to use a tool and what parameters to pass, based on its understanding of the current goal and context.
The quality of an agent's tools matters as much as the quality of its brain. A brilliant model with poorly designed tools is like a genius locked in a room with a broken phone.
The Memory: Context and State
Agents need to remember things — what the user asked, what steps they have completed, what results they received, and what they have learned from past interactions.
Memory comes in layers:
Short-term memory is the conversation context. It holds the current task, recent messages, and intermediate results. This is limited by the model's context window — typically tens or hundreds of thousands of tokens.
Long-term memory persists across sessions. It might store user preferences, past interactions, or learned facts. This is usually implemented through external databases or vector stores.
Working memory holds the agent's current plan and progress. Which steps are done? What is next? What has failed? This is the scratchpad that keeps multi-step tasks on track.
The Loop: Plan, Act, Observe, Reflect
The defining pattern of agent behavior is a loop:
- Plan — Given the goal and current context, decide what to do next
- Act — Execute the chosen action (call a tool, generate text, ask for clarification)
- Observe — Examine the result of the action
- Reflect — Determine if the goal is met, if the approach is working, or if adjustments are needed
This loop — sometimes called the ReAct pattern (Reasoning + Acting) — runs until the agent determines that the goal is complete, or until it decides it cannot proceed and needs human input.
The sophistication of an agent lives in this loop. Simple agents run it once or twice. Advanced agents run it dozens of times, branching, backtracking, and adapting along the way.
The Guardrails: Safety and Constraints
Production agents include boundaries that limit what they can do:
- Permission systems that control which tools an agent can access
- Approval workflows that require human confirmation for high-stakes actions
- Output filters that catch harmful or inappropriate content
- Budget limits that cap how many tokens or API calls an agent can consume
- Timeouts that prevent agents from running indefinitely
These guardrails are not optional extras — they are essential infrastructure. An agent without guardrails is a liability.
Putting It Together
When you evaluate an agent product, ask about each component:
- What model powers it? Can you swap models?
- What tools does it have access to? Can you add custom tools?
- How does it handle memory? Does it learn from past interactions?
- How does its planning loop work? How many steps can it take?
- What guardrails are in place? Who approves high-stakes actions?
The answers reveal whether you are looking at a genuine agent or a chatbot with a marketing upgrade.