1 The Problem: Can AI Only Talk, or Can It Act?
Imagine you are building an enterprise ERP and CRM SaaS platform for Byteion. Your platform features an AI-powered conversational support assistant. A customer enters the chat and submits an inquiry:
A standard Large Language Model (LLM) is exceptional at producing grammatically elegant prose. But its static training weights contain zero knowledge of your private PostgreSQL records. The model cannot know whether ORD-1001 has been dispatched, is still waiting in warehouse sorting, or has already been delivered.
In your production database, the true record exists:
{
"order_id": "ORD-1001",
"status": "shipped",
"carrier": "Pathao",
"estimated_delivery": "2026-10-16"
}
The only reliable way for the AI to answer accurately is to grant it access to a dedicated backend function:
This function queries the database or inventory microservice, retrieves the live record, and feeds it into the model context. The model can then formulate a factual response:
This is why Function Calling is fundamental: it bridges the reasoning intelligence of LLMs with the deterministic data and operations of real software.
2 What is Function Calling?
Function Calling (also termed Tool Use) is a standardized architectural pattern that enables an LLM to request the execution of predefined software tools, database queries, or external APIs.
Through Function Calling, AI systems can interface with:
- Database Queries: Fetching customer profiles, payment logs, and order histories.
- Internal Microservices: Computation engines, tax calculation services, and inventory levels.
- External REST APIs: Live weather updates, stock pricing, courier tracking webhooks.
- ERP / CRM Actions: Generating leads, opening support tickets, and scheduling reminders.
- Document Retrieval: Semantic search across internal knowledge bases and technical documentation.
Core Engineering Axiom: The AI Model Never Executes Your Code!
Function Calling does not mean the LLM executes Python code on your server or holds direct credentials to your database. The model merely decides which tool is needed and proposes structured arguments. Your backend application intercepts this request, validates the arguments, executes the function in a secure environment, and feeds the resulting payload back to the model.
Client/Backend-Controlled Function Calling Architecture Flow:
[ User Submits Query ]
│
▼
[ Backend Application ] ──(Prompt + Tool Schemas)──> [ AI Model ]
│
[ Tool Call Request (name + args) ] <────────────────────┘
│
▼
[ Backend: Parse & Pydantic Validate Arguments ]
│
▼
[ Backend: Execute Authorized Python Function / DB Query ]
│
▼
[ Tool Result (JSON Payload) ] ──(function_call_output)──> [ AI Model ]
│
[ Final Human-Friendly Text ] <──────────────────────────────┘
│
▼
[ User Receives Final Answer ]
This Client/Backend-Controlled workflow is the foundational architecture recommended in official OpenAI documentation: the model proposes tool calls, the application executes them under strict governance, and returns the result.
3 Functions vs. Tools vs. Tool Calls
While these three terms are often used interchangeably in casual discourse, they represent distinct software artifacts:
| Term | Definition & Role | Concrete Example |
|---|---|---|
| Function | Executable code logic residing in your application that accepts inputs and produces outputs. | def calculate_total(price, qty): ... |
| Tool | The formal JSON Schema interface declaring a function's name, capabilities, and parameters to the LLM. | {"type": "function", "name": "calculate_total", ...} |
| Tool Call | A structured generation emitted by the model requesting the execution of a specific tool with concrete arguments. | {"name": "calculate_total", "arguments": "{\"price\": 200, \"quantity\": 5}"} |
Demonstration:
def calculate_total(price: float, quantity: int) -> float:
return price * quantity
User asks: "Calculate the price of 5 items at \$200 each."
Model generates Tool Call: {"name": "calculate_total", "arguments": {"price": 200, "quantity": 5}}.
Backend executes function: calculate_total(price=200, quantity=5) -> {"total": 1000}.
Model delivers final answer: "The total cost is \$1,000."
4 Standard AI Response vs. Function Calling
Comparing the operational realities of both approaches:
Method A: Free-Form Text Generation
The model receives the user's prompt with no external integrations.
Model: "Your order is likely in transit." (Pure hallucination!)
Method B: Function Calling
The model is equipped with the get_order_status tool.
Backend: Fetches live Shipped status from DB
Model: "Order is Shipped via Pathao." (100% factual!)
| Capability | Standard Text Generation | Function Calling |
|---|---|---|
| Live Application Data | Cannot access (frozen at pretraining cutoff) | Real-time retrieval via backend tools |
| Database Access | None | Secure parameter-bound queries via backend |
| External API Invocations | Impossible | Integrates with any REST or gRPC service |
| Security & Permissions | Unreliable prompt instructions | Hard RBAC and schema validation in Python |
5 Writing Our First Python Function
Before interacting with the LLM API, let us write our core Python business function. It accepts an order identifier and returns structured status data:
def get_order_status(order_id: str) -> dict:
orders = {
"ORD-1001": {
"status": "shipped",
"carrier": "Pathao",
"estimated_delivery": "2026-10-16",
},
"ORD-1002": {
"status": "processing",
"carrier": None,
"estimated_delivery": None,
},
}
order = orders.get(order_id)
if order is None:
return {
"found": False,
"message": f"Order {order_id} was not found in our database.",
}
return {
"found": True,
"order_id": order_id,
**order,
}
# Test verification
print(get_order_status("ORD-1001"))
# Output: {'found': True, 'order_id': 'ORD-1001', 'status': 'shipped', ...}
For pedagogical clarity, we use an in-memory dictionary. In production, this function executes a SQL query or calls an internal gRPC service. The critical engineering pattern is to keep the function interface immutable while abstracting the backing data store.
6 Introducing Functions to the AI Model: Tool Schemas
Writing a Python function does not expose it to the LLM automatically. You must define a formal JSON Schema declaring the tool name, purpose, input parameters, and constraints:
tools = [
{
"type": "function",
"name": "get_order_status",
"description": (
"Look up the current status, logistics carrier, and estimated "
"delivery date of an order using its unique order ID."
),
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "The unique order identifier, such as ORD-1001.",
}
},
"required": ["order_id"],
"additionalProperties": False,
},
"strict": True,
}
]
Tool Schema Architectural Breakdown
type: "function": Declares that this tool is an executable function.name: "get_order_status": The unique identifier used by backend dispatchers to invoke the corresponding Python callable.description: Guides the model's routing logic. The more precise the description, the higher the selection fidelity.parameters: Defines the schema for input arguments. Here,order_idis a required string.required: ["order_id"]: Enforces that the model must supply this argument.additionalProperties: False: Rejects hallucinated extraneous arguments.strict: True: Enables constrained grammar decoding at the inference layer to guarantee 100% schema compliance.
7 Generating Our First Tool Call via OpenAI API
Let us submit a request supplying the tool definition via the OpenAI Responses API:
Script: tool_demo.py
import os
from dotenv import load_dotenv
from openai import OpenAI
load_dotenv()
api_key = os.getenv("OPENAI_API_KEY")
model = os.getenv("OPENAI_MODEL", "gpt-4o-mini")
client = OpenAI(api_key=api_key)
tools = [
{
"type": "function",
"name": "get_order_status",
"description": "Look up order status by order ID.",
"parameters": {
"type": "object",
"properties": {
"order_id": {"type": "string", "description": "Order ID, e.g. ORD-1001."}
},
"required": ["order_id"],
"additionalProperties": False,
},
"strict": True,
}
]
response = client.responses.create(
model=model,
tools=tools,
input="What is the current status of order ORD-1001?",
)
print(response.output)
Did the model run our Python code? No. It generated a tool-call request declaring: "To answer this user inquiry, invoke `get_order_status` with argument `{'order_id': 'ORD-1001'}`."
8 Understanding the Tool Call Response
The Responses API returns an array of output items. When the model requests a tool invocation, an item of type function_call is emitted:
{
"type": "function_call",
"name": "get_order_status",
"arguments": "{\"order_id\":\"ORD-1001\"}",
"call_id": "call_abc123xyz"
}
| Field | Description & Operational Purpose |
|---|---|
| type | Specifies that this item is a function_call rather than final prose text. |
| name | The function identifier requested by the model (get_order_status). |
| arguments | A stringified JSON payload containing the arguments generated by the model. |
| call_id | A unique tracking token used to bind the function's execution result back to this specific request. |
9 Executing Functions on the Backend
The fundamental dispatch routine on the backend:
import json
# Arguments string extracted from the model's function_call item
arguments_json = '{"order_id": "ORD-1001"}'
parsed_args = json.loads(arguments_json)
result = get_order_status(order_id=parsed_args["order_id"])
print("Function Output:", result)
In production systems, model-generated arguments must never be trusted without runtime validation. Adversarial prompt injections or unexpected edge cases can result in malicious inputs.
10 Validating Tool Arguments with Pydantic
We enforce defense-in-depth by wrapping every tool's arguments in a strict Pydantic model:
from pydantic import BaseModel, ConfigDict, Field, ValidationError
class GetOrderStatusInput(BaseModel):
model_config = ConfigDict(extra="forbid")
order_id: str = Field(
min_length=3,
max_length=50,
pattern=r"^ORD-[A-Z0-9-]+$",
description="Must start with ORD- followed by uppercase alphanumerics.",
)
# Valid arguments
valid_data = {"order_id": "ORD-1001"}
validated = GetOrderStatusInput.model_validate(valid_data)
print("Validated ID:", validated.order_id) # 'ORD-1001'
# Invalid or malicious input
try:
invalid_data = {"order_id": "DROP TABLE orders;--"}
GetOrderStatusInput.model_validate(invalid_data)
except ValidationError as e:
print("Rejected by Pydantic:", e)
If validation fails, the backend immediately intercepts the call and returns an error payload back to the model, preventing unsafe execution.
11 Returning Tool Results to the AI Model: The Full Loop
A complete, resilient implementation managing the multi-turn conversational tool loop:
import json
import os
from dotenv import load_dotenv
from openai import OpenAI
from pydantic import BaseModel, ConfigDict, Field, ValidationError
load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
MODEL = os.getenv("OPENAI_MODEL", "gpt-4o-mini")
ORDERS = {
"ORD-1001": {"status": "shipped", "carrier": "Pathao", "estimated_delivery": "2026-10-16"},
"ORD-1002": {"status": "processing", "carrier": None, "estimated_delivery": None},
}
class GetOrderStatusInput(BaseModel):
model_config = ConfigDict(extra="forbid")
order_id: str = Field(min_length=3, max_length=50, pattern=r"^ORD-[A-Z0-9-]+$")
def get_order_status(order_id: str) -> dict:
order = ORDERS.get(order_id)
if not order:
return {"found": False, "message": f"Order {order_id} not found."}
return {"found": True, "order_id": order_id, **order}
TOOLS = [
{
"type": "function",
"name": "get_order_status",
"description": "Look up status and carrier of an order by order ID.",
"parameters": {
"type": "object",
"properties": {
"order_id": {"type": "string", "description": "Order ID like ORD-1001."}
},
"required": ["order_id"],
"additionalProperties": False,
},
"strict": True,
}
]
def execute_tool(name: str, arguments_json: str) -> dict:
if name != "get_order_status":
return {"error": "unsupported_tool", "message": f"Tool {name} is not allowed."}
try:
raw_args = json.loads(arguments_json)
validated = GetOrderStatusInput.model_validate(raw_args)
return get_order_status(validated.order_id)
except (json.JSONDecodeError, ValidationError) as err:
return {"error": "invalid_arguments", "details": str(err)}
except Exception:
return {"error": "execution_failed", "details": "Internal service error."}
def ask_assistant(user_query: str) -> str:
input_items = [{"role": "user", "content": user_query}]
max_rounds = 5
for _ in range(max_rounds):
response = client.responses.create(
model=MODEL,
tools=TOOLS,
input=input_items,
instructions=(
"You are an order support assistant. Use the order lookup tool when needed. "
"Answer based on the tool result. Never invent order details."
),
)
# Preserve the model's output in conversation history
input_items += response.output
tool_calls = [item for item in response.output if item.type == "function_call"]
# Terminate loop when no further tool calls are requested
if not tool_calls:
return response.output_text
# Execute each proposed tool call and inject results
for call in tool_calls:
result = execute_tool(name=call.name, arguments_json=call.arguments)
input_items.append({
"type": "function_call_output",
"call_id": call.call_id,
"output": json.dumps(result, ensure_ascii=False),
})
raise RuntimeError("Exceeded maximum tool rounds without resolution.")
if __name__ == "__main__":
answer = ask_assistant("What is the current status of my order ORD-1001?")
print("AI Answer:", answer)
Loop Mechanics Explained
input_items += response.output retains the model's reasoning state. The backend appends a function_call_output matching the exact call_id. On the subsequent iteration, the model reads this factual output and synthesizes a natural-language response.
12 tool_choice — Controlling When the Model Uses Tools
The tool_choice parameter controls the model's tool invocation behavior:
| Option | Behavior & Effect | Production Use Case |
|---|---|---|
| "auto" | Model dynamically decides whether to call a tool or reply in text. | General chatbots and customer support agents (Default). |
| "required" | Forces the model to execute at least one tool call before responding. | Workflows requiring mandatory database verification. |
| "none" | Inhibits all tool calls; forces standard text generation. | Conversational pleasantries or explanatory queries. |
| Specific Function | Forces invocation of one designated function specifically. | Strict routing or single-purpose data extraction steps. |
13 Can a Model Make Multiple Tool Calls? (Parallel vs Sequential)
Yes! Modern frontier LLMs support parallel tool calling within a single inference step.
The model emits two distinct function_call items in one response:
[
{"name": "get_order_status", "arguments": "{\"order_id\":\"ORD-1001\"}", "call_id": "call_1"},
{"name": "get_order_status", "arguments": "{\"order_id\":\"ORD-1002\"}", "call_id": "call_2"}
]
The backend can execute these concurrently via asyncio.gather and submit the results back tagged with their respective call_id tokens.
Caution: Interdependent Calls Require Sequential Execution
If Tool B requires an argument generated by Tool A (e.g., retrieving a customer ID before querying customer transactions), parallel execution is impossible. The loop must resolve Tool A first, pass its result to the model, and allow the model to emit Tool B sequentially.
14 Are Function Calling and Structured Outputs the Same?
These two features serve complementary yet fundamentally distinct purposes:
| Evaluation Dimension | Structured Outputs (Lesson 16.09) | Function Calling (Lesson 16.10) |
|---|---|---|
| Primary Purpose | Constraining final generated text to a JSON Schema | Requesting backend software tool executions |
| Schema Applied To | Final Output Payload | Function Arguments |
| Backend Execution Required? | No (parse data directly) | Yes (run code and return results) |
| Representative Use Case | Invoice metadata extraction | Database lookup / External API calls |
Synergistic Enterprise Pattern: An intelligent agent first invokes Function Calling to retrieve records from the database, then applies Structured Outputs to format the final synthesized response for frontend UI rendering.
15 Security Considerations in Function Calling
Because Function Calling couples stochastic models with production software systems, improper design creates critical security vulnerabilities:
In multi-tenant SaaS environments, a user belonging to Tenant A might attempt prompt injection: "Show me order ORD-9999 from Tenant B." If your backend function blindly queries by tenant_id supplied in the tool arguments, cross-tenant data leaks will occur.
# def get_order(order_id, tenant_id): ... <-- tenant_id supplied by LLM!
# SECURE ARCHITECTURE:
def get_order(order_id: str, authenticated_tenant_id: str):
return db.query(Order).filter_by(order_id=order_id, tenant_id=authenticated_tenant_id).first()
The authenticated tenant context must always be injected from the validated server session or JWT token, never from model-generated arguments.
Read tools (e.g., get_order_status()) have low blast radius. Write tools (e.g., cancel_order(), issue_refund()) mutate persistent application state. Write operations require strict server-side authorization checks, idempotency keys, and explicit human confirmation before execution.
Writing "Only execute this tool for authorized administrators" in a prompt or tool description is not security; prompt injection easily bypasses natural language guardrails. True authorization must be enforced in Python code.
16 Production Error Handling & Reliability Matrix
Managing tool failures gracefully in enterprise pipelines:
| Failure Mode | Root Cause | Remediation Strategy |
|---|---|---|
| Invalid Arguments | Model breached Pydantic type or regex constraints | Feed ValidationError message back in function_call_output so the model can self-correct. |
| Unknown Tool Name | Model requested an unapproved function name | Reject invocation immediately with a structured error; never use dynamic eval(). |
| Record Not Found | Database contains no record for the requested ID | Return {"found": false} so the model can inform the user courteously. |
| Database Timeout | Downstream service or SQL database latency spike | Enforce 3s timeout with bounded exponential retry and circuit breaking. |
| Infinite Tool Loop | Model repeatedly invokes tools without reaching final prose | Enforce max_tool_rounds = 5 ceiling and raise fallback response. |
17 7 Practical Steps for Debugging Tool Calling
type == "function_call" and verify call_id generation.
json.loads() and Pydantic validation on mock strings outside the API loop.
function_call_output references the precise call_id assigned by the model.
18 Practical Project — AI Order Support Assistant
We now assemble a modular, enterprise-grade AI Order Support Assistant in Python:
├── app/
│ ├── schemas.py # Pydantic input schemas
│ ├── tools.py # Tool definitions & dispatcher
│ ├── services/orders.py # Business logic & data query
│ └── main.py # Conversational agent runner loop
└── tests/
└── test_tools.py # Offline Pytest unit test suite
File 1: app/schemas.py
from pydantic import BaseModel, ConfigDict, Field
class OrderLookupInput(BaseModel):
model_config = ConfigDict(extra="forbid")
order_id: str = Field(
min_length=3,
max_length=50,
pattern=r"^ORD-[A-Z0-9-]+$",
description="Standard Byteion order ID (e.g. ORD-1001).",
)
File 2: app/services/orders.py
from typing import Any, Dict
# Demo data store (replace with production database repository)
DATABASE: Dict[str, Dict[str, Any]] = {
"ORD-1001": {
"status": "shipped",
"carrier": "Pathao",
"tracking_code": "PTH-88419",
"estimated_delivery": "2026-10-16",
},
"ORD-1002": {
"status": "processing",
"carrier": None,
"tracking_code": None,
"estimated_delivery": None,
},
}
def query_order(order_id: str) -> Dict[str, Any]:
record = DATABASE.get(order_id)
if record is None:
return {"found": False, "message": f"Order {order_id} does not exist."}
return {"found": True, "order_id": order_id, **record}
File 3: app/tools.py
import json
from typing import Any, Dict
from pydantic import ValidationError
from schemas import OrderLookupInput
from services.orders import query_order
TOOL_DEFINITIONS = [
{
"type": "function",
"name": "get_order_status",
"description": "Look up order status, shipping carrier, and estimated delivery by order ID.",
"parameters": {
"type": "object",
"properties": {
"order_id": {"type": "string", "description": "Order ID (e.g. ORD-1001)."}
},
"required": ["order_id"],
"additionalProperties": False,
},
"strict": True,
}
]
def dispatch_tool(name: str, arguments_json: str) -> Dict[str, Any]:
if name != "get_order_status":
return {"error": "unknown_tool", "message": f"Tool '{name}' is not recognized."}
try:
raw_args = json.loads(arguments_json)
validated = OrderLookupInput.model_validate(raw_args)
return query_order(validated.order_id)
except (json.JSONDecodeError, ValidationError) as err:
return {"error": "invalid_arguments", "message": str(err)}
except Exception:
return {"error": "internal_error", "message": "Failed to query order service."}
File 4: app/main.py
import json
import os
from dotenv import load_dotenv
from openai import OpenAI
from tools import TOOL_DEFINITIONS, dispatch_tool
load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
MODEL = os.getenv("OPENAI_MODEL", "gpt-4o-mini")
def run_assistant(user_prompt: str) -> str:
conversation = [{"role": "user", "content": user_prompt}]
max_turns = 5
for _ in range(max_turns):
response = client.responses.create(
model=MODEL,
tools=TOOL_DEFINITIONS,
input=conversation,
instructions=(
"You are Byteion's AI Order Support Assistant. "
"Use the get_order_status tool when an order ID is mentioned. "
"Synthesize the tool output clearly. "
"If not found, clearly state that the order was not found."
),
)
conversation += response.output
tool_calls = [item for item in response.output if item.type == "function_call"]
if not tool_calls:
return response.output_text
for tool_call in tool_calls:
result = dispatch_tool(tool_call.name, tool_call.arguments)
conversation.append({
"type": "function_call_output",
"call_id": tool_call.call_id,
"output": json.dumps(result, ensure_ascii=False),
})
raise RuntimeError("Exceeded maximum tool rounds.")
if __name__ == "__main__":
query = "What is the status of my order ORD-1001?"
print("User:", query)
print("Assistant:", run_assistant(query))
File 5: tests/test_tools.py (Offline Pytest Suite)
from tools import dispatch_tool
def test_valid_order_dispatch():
res = dispatch_tool("get_order_status", '{"order_id": "ORD-1001"}')
assert res["found"] is True
assert res["status"] == "shipped"
assert res["carrier"] == "Pathao"
def test_order_not_found():
res = dispatch_tool("get_order_status", '{"order_id": "ORD-9999"}')
assert res["found"] is False
def test_invalid_arguments_rejected():
res = dispatch_tool("get_order_status", '{"order_id": "BAD"}')
assert "error" in res
assert res["error"] == "invalid_arguments"
def test_unknown_tool_rejected():
res = dispatch_tool("delete_database", "{}")
assert res["error"] == "unknown_tool"
19 Knowledge Check: Test Your Understanding
Select the correct answer for each scenario and review the technical explanations to consolidate your knowledge:
What is the core purpose of Function Calling?
Explanation: Function Calling allows the model to propose structured invocations that your backend executes safely.
Upon receiving a Function Call from the model, who executes the actual function logic?
Explanation: Execution control remains strictly with your application backend; the model only proposes arguments.
What is the primary role of the description field in a Tool Schema?
Explanation: Clear descriptions guide the model's semantic routing logic to pick the right tool for user requests.
Why is the call_id parameter critical in tool-calling workflows?
Explanation: When models emit parallel tool requests, matching call_ids ensure deterministic result correlation.
In a Multi-Tenant SaaS platform, where must the tenant_id be sourced from?
Explanation: Sourcing tenant identifiers from model arguments enables cross-tenant data exfiltration via prompt injection.
If the AI requests execution of a write tool like issue_refund(), how should the backend respond?
Explanation: Mutating business actions require rigorous defense-in-depth, idempotency keys, and explicit approval policies.
What is the primary difference between Structured Outputs and Function Calling?
Explanation: Structured Outputs format the model's final response, while Function Calling requests external execution.
20 Lesson Summary & Key Takeaways
In this lesson, we discovered how to empower LLMs with real-world capabilities while keeping security and execution governance strictly under backend control.
Core Engineering Takeaways
- Function Calling as a Bridge: Connects LLM reasoning intelligence with live enterprise data and operational software services.
- Client-Controlled Governance: The model only suggests tool calls; the backend validates arguments and performs the actual execution.
- Strict Tool Schemas: JSON Schema combined with
strict: Trueenforces exact parameter structures and data types. - Pydantic Argument Validation: Every argument string must pass through runtime schema validation before touching internal services.
- Multi-Tenant Isolation: Never trust tenant or user IDs generated by the model; always source them from authenticated sessions.
- Read vs. Write Tools: State-mutating operations require strict authorization, idempotency tracking, and human-in-the-loop gates.
- Bounded Recursive Loops: Cap conversational tool cycles with
max_tool_roundsto prevent runaway loops and token waste.
"The AI model proposes what tool to call and with what arguments; but the authority, security, and execution power remain entirely with your backend application."
Lesson 16.11: Reliable Tool Execution & Side-Effect Control
Engineering patterns for safe tool execution, permission boundaries, idempotency, retries, and database transaction management.