What is an AI Agent? Architecture, Planning, Memory and Multi-Agent Systems

From Chatbot to Agent: A Fundamental Shift

A chatbot responds. An agent acts.

A standard LLM interaction is a single round-trip: user sends a message, model generates a response, done. The model has no persistent memory, no ability to take actions in the world, and no way to break down a complex task into steps.

An AI agent is an LLM-powered system that perceives its environment, reasons about what needs to be done, takes actions (calling tools, browsing the web, writing code, querying databases), observes the results of those actions, updates its understanding, and repeats — until the task is complete or it reaches a stopping condition.

The shift from chatbot to agent is the shift from passive response to autonomous task completion. This is what enables an AI to research a company, draft a personalised outreach email, look up their LinkedIn, find their email via a hunter.io API call, add them to a CRM, and schedule a follow-up — all without human intervention at each step.

The ReAct Loop: Reason → Act → Observe

The most widely used agent architecture is ReAct (Reasoning and Acting), introduced by Yao et al. at Princeton (2022). The agent operates in a loop:

1. Thought: The LLM reasons about the current state of the task. "I need to find the CEO's name. I should search the company website."
2. Action: The LLM outputs a structured action — a tool call with parameters. search_web(query="Acme Corp CEO")
3. Observation: The tool executes and returns a result. The result is added to the context. "Search returned: CEO is John Smith, joined 2019."
4. Repeat: The LLM reads the observation, updates its reasoning, and decides the next action — or outputs a final answer if the task is complete.

Each iteration of this loop appends thought, action, and observation to the growing context window. The agent "remembers" what it has done within a session through this context accumulation. The loop runs until the agent calls a special finish action or the maximum number of iterations is reached.

Agent Architecture Components

1. The Planning Module (Brain)

The LLM is the planning module — the agent's brain. Its quality is the primary determinant of agent performance. GPT-4o and Claude 3.5 Sonnet are the current state-of-the-art for agentic tasks because they reliably follow multi-step instructions, understand tool schemas, and produce well-formed JSON tool calls.

The system prompt defines the agent's persona, available tools (their names, descriptions, and parameter schemas), constraints, and task framing. A well-designed system prompt is the most impactful way to improve agent reliability.

Advanced planning approaches:

• Chain-of-Thought (CoT): Instruct the model to think step-by-step before acting. Dramatically reduces errors on complex tasks.
• Tree of Thought (ToT): Explore multiple reasoning paths in parallel, evaluate them, and select the best branch.
• Plan-and-Execute: First generate a full plan (list of steps), then execute each step. Reduces mid-task drift.

2. Memory Systems

An agent without memory starts fresh on every task. Production agents need several types of memory:

• Short-term memory (Working memory): The context window itself — the running record of thoughts, actions, and observations in the current session. Limited by context window size (128K tokens for GPT-4o). Managed via context summarisation when approaching limits.
• Long-term memory (Episodic): Past conversations and task outcomes stored in a database and retrieved via semantic search. "What did I do last time this user asked about billing?" A vector database (pgvector, Qdrant) enables this.
• Semantic memory (Knowledge): Facts about the world, product specs, user preferences — stored in structured form and retrieved as needed. RAG is the primary mechanism.
• Procedural memory: Learned skills and task templates stored as reusable code or structured prompts. "To qualify a sales lead: step 1, check company size; step 2, verify budget range..."

3. Tools (Actions the Agent Can Take)

Tools are functions the agent can call. They define what the agent can do in the world. A tool has a name, description (used by the LLM to decide when to call it), and a JSON Schema defining its parameters.

Common tool categories:

• Information retrieval: web_search, rag_search, database_query, read_file, get_calendar_events
• Write/create: send_email, create_crm_record, write_file, post_to_slack
• Computation: run_python_code, calculate, parse_json
• External APIs: create_calendar_event, charge_stripe_customer, lookup_linkedin_profile
• Agent control: ask_human_for_clarification, finish (terminate the loop with final answer)

The agent does not directly call tools — the LLM outputs a structured JSON object specifying the tool name and parameters, and the tool executor (application code) actually runs the function and returns the result.

4. The Executor

The executor is the application layer that:

• Runs the agent loop (calling the LLM, parsing tool calls, executing tools, appending observations)
• Enforces safety constraints (prevent the agent from calling destructive tools without confirmation)
• Manages context window limits (summarise old context when approaching the limit)
• Handles errors (tool failures, malformed LLM outputs, timeouts)
• Logs every step for debugging and monitoring

LangGraph, CrewAI, and AutoGen are frameworks that provide executor infrastructure. For production, I build custom executors in Python/Laravel with full control over each step.

Multi-Agent Systems: Agents Working Together

Complex tasks benefit from multi-agent systems where specialised agents collaborate, each responsible for a different aspect of the task.

The Orchestrator-Worker Pattern

An orchestrator agent receives a high-level task and breaks it into subtasks, delegating each to a specialised worker agent. Worker agents complete their subtask and return results to the orchestrator, which synthesises the final output.

Example — automated competitive intelligence report:

• Orchestrator: "Generate a competitive analysis of Acme Corp."
• Research Agent: Searches the web, reads news articles, scrapes the company website.
• Financial Agent: Pulls revenue data from Crunchbase, analyses trends.
• Social Agent: Analyses LinkedIn, Twitter, and review sites.
• Writer Agent: Takes all inputs, generates a structured report.
• Orchestrator: Reviews the report, requests revisions if needed, finalises.

Critic-Refinement Pattern

A generator agent produces an output. A separate critic agent evaluates it against defined criteria. The generator revises based on the critique. This loop runs until the critic approves or a maximum iteration count is reached. This pattern dramatically improves output quality for writing, code generation, and structured data extraction tasks.

CrewAI and LangGraph

CrewAI provides a high-level abstraction for multi-agent systems. You define agents with roles, backstories, and tools; define tasks with descriptions, expected outputs, and assigned agents; and define the crew (group of agents) and process (sequential or hierarchical). CrewAI handles the orchestration and inter-agent communication.

LangGraph models agent workflows as directed acyclic graphs (DAGs) — nodes are LLM calls or tool executions, edges define conditional flow. This gives fine-grained control over branching, looping, and parallel execution. Better for complex, custom agentic workflows where you need to handle specific failure modes.

Agent Reliability: The Core Engineering Challenge

The biggest challenge in production agents is reliability. LLMs are probabilistic — they can output malformed JSON, call the wrong tool, get stuck in loops, or make wrong decisions. Engineering around this requires:

• Structured output parsing: Use OpenAI function calling / JSON mode to force well-formed tool call outputs.
• Retry logic: Retry with error feedback when a tool call is malformed.
• Max iterations: Hard cap on loop iterations to prevent infinite loops.
• Human-in-the-loop: For high-stakes actions (sending emails, charging payments), require human confirmation before execution.
• Comprehensive logging: Log every thought, action, and observation for debugging.
• Confidence thresholds: If the agent has low confidence, ask for clarification rather than acting.

The best production agents I've built run at 95%+ reliability on their defined task scope with structured output parsing, retry logic, and tight tool schemas that constrain the action space.