AI Agents in Production: Tool Calling, Planning, Memory, Agent Loops & Reliability

In previous lessons, we mastered fine-tuning, inference serving with vLLM, and enterprise production architectures with RAG, semantic caching, and AI gateways. In those applications, the execution sequence is largely linear and predefined: a user submits a query, the application retrieves context chunks, passes them to the LLM, validates the output, and returns a response.

An AI Agent operates on an entirely different paradigm. Given a high-level goal, an agent dynamically selects which tools to invoke, observes their outputs, updates its internal state, reasons about the outcome, and decides the subsequent action until the goal is achieved or a stopping condition is met.

Consider this enterprise request: "Find order #ORD-1024, check its delivery status, and email the customer with an updated tracking link if the package is delayed."

A traditional LLM explains how a customer support representative might accomplish this. An AI agent, by contrast, executes the entire operational workflow:

1. Queries the order database for ORD-1024.
2. Inspects the courier delivery API for transit timestamps.
3. Evaluates whether the package is delayed against SLA thresholds.
4. Drafts a personalized apology notification with updated tracking.
5. Checks organizational policy to determine whether human manager approval is required.
6. Triggers the authorized notification service upon receiving approval.
7. Records the completed action in the customer audit ledger.

Formally, an AI agent is a software system that uses a foundation model as a reasoning engine to select and execute discrete actions toward a specified goal, observing environmental feedback to guide subsequent iterations.

Every production agent architecture relies on six distinct architectural responsibilities:

Key Architectural Takeaway: An agent is not simply an LLM with a 4,000-word prompt. Its defining attribute is agency: the delegated capability to select operations dynamically, ingest external tool results, and iteratively adjust its path toward goal completion.

Consider an enterprise user requesting: "Find total revenue across EMEA for Q3, compare against our quarterly target, and send a summary PDF to the CFO."

Traditional LLM pipelines can indeed call tools (such as performing a SQL lookup in a RAG step). The critical distinction lies in how much control over the sequence of actions is delegated to the foundation model.

4.1 Goal Definition

A production agent requires a verifiable, bounded goal. Vague prompts invite hallucinated tool loops and ambiguous termination conditions.

4.2 The Agent Controller

The controller is the supervisory backend orchestrator. It is written in deterministic code (Python, TypeScript, Go) and governs:

• Formatting conversation history and available tool schemas into model prompts.
• Parsing model responses for structured tool calls.
• Verifying that the requested tool is allowed under the current user's session token.
• Executing authorized functions with timeout safeguards.
• Appending tool observations back into the working context.
• Enforcing loop iteration limits and token spend budgets.

4.3 Durable Task State

A production agent must not rely solely on the LLM's conversation history as its state store. If an application worker crashes or restarts mid-task, conversation history may be lost. State must be durably stored:

The Agent Loop is the fundamental operational engine of an AI agent. In each iteration, the agent decides whether it requires more information from tools or whether it has collected sufficient evidence to formulate a final response.

Consider the catalog query: "Find the five cheapest ultrabooks in our inventory and summarize their specs."

Tool calling (also termed Function Calling) allows language models to emit structured JSON arguments conforming to an explicit schema, rather than outputting arbitrary text. The model does not execute the function; it outputs an instruction requesting that the application execute it.

6.1 Tool Definition Schema

6.2 Tool Design Principles

• Single Responsibility: Prefer granular, focused tools (get_order_status, cancel_order) over monolithic "do-everything" tools (manage_system(action, payload)).
• Explicit Parameter Types & Regex: Constrain inputs using strict regex patterns and enums to prevent hallucinated argument variations.
• Structured JSON Return Values: Always return clean JSON key-value pairs (e.g., {"status": "delayed", "delay_days": 2}) rather than natural language paragraphs. Structured data minimizes token consumption and prevents ambiguous model interpretation.
• Timeout and Resilience: Every tool execution must wrap external network calls in hard timeouts (e.g., 3,000ms) with circuit-breaker protection.

Not all tools carry equal blast radius. Conflating read operations with write operations in an agent's security model is one of the most critical vulnerabilities in production AI systems.

Production Defense: Even if a model outputs a valid tool call for issue_customer_refund, the backend controller must independently verify:

1. Does the active user session possess the BILLING_ADMIN role?
2. Is the refund amount $\le \$500$ (automated policy limit)?
3. Has a refund already been processed for this order (idempotency check)?
4. Is the order within the 30-day return policy window?

Complex real-world goals cannot be resolved in a single step. An agent must break down a compound objective into a sequence or Directed Acyclic Graph (DAG) of actionable subtasks.

Pioneered by Yao et al., the ReAct (Reasoning + Acting) framework interleaves verbal reasoning ("Thoughts") with action execution ("Actions") and environmental feedback ("Observations").

Production Insight: In user-facing production systems, never expose raw internal reasoning thoughts directly to users. Chain-of-thought tokens may leak prompt instructions or confuse non-technical users. Store thoughts in an internal telemetry span for debugging, and present only polished final answers.

Production agents require distinct memory subsystems tailored to specific temporal scopes:

For mission-critical enterprise workflows, an agent must be governed by an explicit Finite State Machine (FSM). State machines enforce valid state transitions and prevent illegal operational shortcuts.

Under this FSM:

• A task in PENDING transitions to VALIDATING once user inputs and RBAC permissions are confirmed.
• If a tool requires human oversight, the task moves to AWAITING_APPROVAL and halts execution.
• The backend rejects any direct transition from PENDING directly to COMPLETED, eliminating bypass vulnerabilities.

One of the most consequential decisions in AI engineering is choosing between deterministic software pipelines and autonomous agents:

When tasks encompass disparate domains requiring distinct toolsets and prompt specializations, a single monolithic agent can become overwhelmed. Multi-agent systems decompose tasks across specialized agents:

When to Avoid Multi-Agent Architecture: If a task can be accomplished reliably with a single agent equipped with 3 well-scoped tools, do not introduce multi-agent complexity. Every additional agent multiplies context overhead, token costs, coordination failures, and debugging difficulty.

Because agents can trigger real-world mutations, systems must strictly enforce Least Privilege and Human-in-the-Loop (HITL) governance.

14.1 Role-Based Tool Matrix

14.2 The HITL Asynchronous Approval Pattern

When an agent encounters an action flagged as sensitive, the controller freezes task state and issues a signed approval request:

Parameter Binding Security: The approval token must bind strictly to the exact arguments proposed. If the refund amount changes from \$1,250 to \$1,251, the HMAC signature fails validation, blocking tampering attacks.

Without strict execution boundaries, an agent can easily fall into an infinite loop (e.g. searching with empty results, encountering recurring 400 errors, or flapping between two conflicting tools):

16.1 Idempotency Keys

If an agent calls issue_refund(order_id="ORD-1024") and the HTTP socket times out after the payment provider debits the account, a naive retry could issue a double refund.

Every write tool invocation must carry a deterministic Idempotency Key:

16.2 Structured Tool Errors

Tools must return structured error objects allowing the agent to distinguish recoverable faults from fatal rejections:

Evaluating an agent requires inspecting its trajectory (the entire sequence of actions taken), not just its final text output. An agent that answers a question correctly by brute-forcing 15 unauthorized database queries is an operational failure.

Below is a clean, dependency-free reference implementation of a production-style agent controller featuring explicit state tracking, tool registries, step caps, and structured error propagation:

from dataclasses import dataclass, field
from typing import Callable, Dict, Any, List

# 1. State Definition
@dataclass
class AgentState:
    task_id: str
    goal: str
    max_steps: int = 5
    current_step: int = 0
    status: str = "PENDING"
    history: List[Dict[str, Any]] = field(default_factory=list)
    variables: Dict[str, Any] = field(default_factory=dict)

# 2. Tool Registry & Implementation
TOOLS: Dict[str, Callable] = {}

def register_tool(name: str):
    def decorator(fn: Callable):
        TOOLS[name] = fn
        return fn
    return decorator

@register_tool("get_order")
def get_order(order_id: str) -> Dict[str, Any]:
    db = {"ORD-1024": {"status": "delayed", "eta": "2026-10-09", "carrier": "FedEx"} }
    if order_id not in db:
        return {"success": False, "error": "ORDER_NOT_FOUND"}
    return {"success": True, "order": db[order_id]}

@register_tool("notify_customer")
def notify_customer(order_id: str, message: str) -> Dict[str, Any]:
    # Production write tool: check approval or execute
    return {"success": True, "dispatched": True, "recipient": "customer@example.com"}

# 3. Secure Execution Dispatcher
def execute_tool(name: str, args: Dict[str, Any]) -> Dict[str, Any]:
    if name not in TOOLS:
        return {"success": False, "error": f"UNKNOWN_TOOL: {name}"}
    try:
        return TOOLS[name](**args)
    except Exception as e:
        return {"success": False, "error": f"TOOL_EXCEPTION: {str(e)}"}

# 4. Production Agent Control Loop
def run_agent_loop(goal: str, model_decide: Callable) -> Dict[str, Any]:
    state = AgentState(task_id="tsk_001", goal=goal)
    state.status = "RUNNING"

    while state.current_step < state.max_steps:
        state.current_step += 1
        
        # In production: format prompt with state.history and tools schemas
        decision = model_decide(goal=state.goal, history=state.history)

        if decision["type"] == "FINAL_ANSWER":
            state.status = "COMPLETED"
            return {"status": "COMPLETED", "answer": decision["text"], "steps": state.current_step}

        if decision["type"] == "TOOL_CALL":
            tool_name = decision["name"]
            tool_args = decision["args"]
            
            # Execute with observation
            result = execute_tool(tool_name, tool_args)
            state.history.append({
                "step": state.current_step,
                "thought": decision.get("thought", ""),
                "tool": tool_name,
                "args": tool_args,
                "observation": result
            })
            
            # Check stopping on terminal failure
            if not result.get("success") and not result.get("retryable", True):
                state.status = "FAILED"
                return {"status": "FAILED", "error": result["error"], "steps": state.current_step}

    state.status = "LIMIT_REACHED"
    return {"status": "LIMIT_REACHED", "steps": state.current_step, "error": "Max step ceiling tripped"}