AI Agents in Production: Function Calling, Tool Schemas, Planning, Memory, Permissions & Reliable Loops

In modern enterprise AI systems, we have evolved through distinct architectural generations:

Consider a realistic enterprise incident: “Investigate why today's checkout completion dropped 18% in the UK region.” A passive chatbot can only explain checkout abandonment statistics. An AI Agent actively checks Stripe payment failure logs, inspects recent CDN deployment releases, queries Redis error rates, isolates an expired SSL certificate on the payment webhook, and opens an urgent Jira incident ticket.

The beating heart of every agent is the Control Loop. At each iteration, the model evaluates the accumulated state and makes one of two fundamental decisions: call a tool or produce the final completion.

In clean Python pseudocode, the production loop must never be a simple while True:. It must be bounded by strict limits:

def run_agent_loop(goal: str, user_id: str, max_steps: int = 10) -> AgentResult:
    state = AgentState(goal=goal, user_id=user_id, status="running")
    
    for step in range(max_steps):
        decision = llm_decide_action(state)
        
        if decision.type == "final":
            state.status = "completed"
            return AgentResult(success=True, answer=decision.content)
            
        elif decision.type == "tool_call":
            # 1. Enforce Server-Side Authorization
            if not authorize_tool(state.user_id, decision.tool_name, decision.arguments):
                state.add_error(f"Permission denied for {decision.tool_name}")
                continue
                
            # 2. Execute within Safe Sandbox
            result = execute_sandboxed_tool(decision.tool_name, decision.arguments)
            
            # 3. Observe and append result to working memory
            state.add_observation(decision.tool_name, result)
            
    return AgentResult(success=False, error="Max steps exceeded without resolution")

The Golden Security Axiom: The LLM must never execute tools directly or possess direct database credentials. The LLM only generates a structured JSON request. Your backend application inspects, validates, authorizes, executes the tool, and returns the output:

Every tool exposed to the model must be defined with an explicit JSON Schema:

{
  "name": "issue_customer_refund",
  "description": "Issues a monetary refund to an existing customer for an eligible order.",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": {
        "type": "string",
        "description": "Unique identifier of the order (e.g. 'ORD-98214')"
      },
      "amount_usd": {
        "type": "number",
        "minimum": 0.01,
        "maximum": 5000.00,
        "description": "Refund amount in USD dollars"
      },
      "reason": {
        "type": "string",
        "enum": ["damaged_goods", "shipping_delay", "wrong_item", "customer_satisfaction"]
      }
    },
    "required": ["order_id", "amount_usd", "reason"]
  }
}

In enterprise environments, not all tools carry equal blast radiuses. Production systems categorize tools into 4 risk tiers:

How should an agent sequence its actions? There are two primary paradigms:

An agent without state cannot solve multi-step problems. In production, memory is structured into four distinct layers:

• Working Memory (Scratchpad): Holds the active task state, current loop step index, and immediate tool return payloads.
• Conversational Memory: Recent multi-turn message history between user and assistant, trimmed via rolling window or summarization.
• Semantic Long-Term Memory: Persistent profile facts (e.g. “User prefers invoices in EUR currency”) stored in vector embeddings.
• Episodic Long-Term Memory: Historical logs of previously completed agent tasks, used for few-shot trajectory learning.

When an agent proposes a high-risk operation (such as issuing a $2,000 refund), execution must pause in a WAITING_FOR_APPROVAL state. However, simple boolean approval flags are vulnerable to Time-of-Check to Time-of-Use (TOCTOU) race conditions.

To prevent tampering while awaiting human sign-off, production systems bind the exact tool call parameters with a cryptographic SHA-256 HMAC token:

# Cryptographic Parameter Binding in Python
import hashlib, json, hmac

def generate_approval_token(task_id: str, tool: str, args: dict, secret_key: str) -> str:
    canonical_payload = json.dumps({"task_id": task_id, "tool": tool, "args": args}, sort_keys=True)
    return hmac.new(secret_key.encode(), canonical_payload.encode(), hashlib.sha256).hexdigest()

def verify_approval(token: str, task_id: str, tool: str, current_args: dict, secret_key: str) -> bool:
    expected_token = generate_approval_token(task_id, tool, current_args, secret_key)
    return hmac.compare_digest(token, expected_token)

If the refund amount or recipient is modified while the ticket is pending in the approval queue, the HMAC signature mismatches and the execution engine automatically aborts the transaction.

Agents operate across distributed, flaky networks. If a payment API times out after 10 seconds, the agent cannot tell whether the charge went through. If it blindly retries, the customer may be charged twice.

• Deterministic Idempotency Keys: Every side-effect mutation includes a generated key: idempotency_key = f"refund:{order_id}:{task_id}". The payment gateway guarantees identical repeated requests return the original receipt without double-charging.
• Exponential Backoff with Full Jitter: Retries must wait $t = \min(M, B \times 2^{\text{attempt}}) \times \text{rand}(0.5, 1.5)$ to avoid the thundering herd problem.
• Hard Loop Circuit Breakers: Agents must terminate immediately if:
  • Step count reaches MAX_STEPS = 10.
  • Token consumption reaches MAX_TOKENS = 25,000.
  • Wall-clock duration reaches TIMEOUT = 45s.
  • Identical consecutive tool calls return identical empty results (Loop Trap).

Debugging an agent that went off the rails requires replaying its exact execution trajectory. Every production agent run produces an immutable trace log: