How AI Tools Work: Function Calling and Structured Outputs

TL;DR

An AI tool connects artificial intelligence (AI) to a software capability. A large language model (LLM), the component that generates text, can request a lookup. For the application-owned tool in this episode, server code checks permission, runs the lookup, and returns the result.

• A proposed tool call is a request, not proof that anything happened.
• Predictable fields help the application check data; they do not grant access.
• A failed lookup means “could not check,” not an invented claim status.

Emily now asks, “What is the status of claim EXP-1042?” We will give our fictional support desk one narrow way to read that record, carrying forward the boundaries from the Guardrails episode.

1. A policy is not a claim record

Emily’s dinner cost $45. Travel Policy v3, §4 allows a $50 dinner limit and requires an itemized receipt. No lost-receipt exception was supplied. Our retrieval-augmented generation (RAG) episode showed how finding relevant documents helps explain that rule.

The policy cannot tell us what happened to one particular claim. Emily has since submitted it separately through the company’s existing expense portal. The assistant has never submitted, approved, or paid anything. Answering her new question requires the expense system’s record, rather than a plausible continuation of the earlier conversation.

We propose a read-only tool named get_my_expense_status. It uses the expense system’s application programming interface (API): an agreed way for software to request information from another system. The claim identifier and returned record below are invented teaching examples, not a live lookup.

2. The model asks; the application runs

Function calling lets the model propose a named function and its arguments: the values the function needs. Here, it proposes get_my_expense_status with claim_id set to EXP-1042. The generated proposal does not query the expense system by itself.

Application code checks the tool name and argument, verifies access, then makes the API request. It checks the response and supplies a small result back to the model, which can explain it to Emily. OpenAI’s function-calling guide describes this round trip.

This division describes our application-owned function. Providers also offer hosted tools that run on their infrastructure. Anthropic distinguishes application-executed tools from provider-executed tools.

3. Give the tool a small, clear contract

A contract describes what the tool accepts and returns. Keep this one narrow enough to explain in a few rows:

JSON, short for JavaScript Object Notation, is a common data format with named fields and values. A schema is a checkable description of data: field names, required fields, types, allowed values, and whether extra fields are permitted. Explicitly require the argument and reject unexpected fields. JSON Schema documents required properties and extra-property rules.

The server accepts only an allowlisted tool name: a name it explicitly recognizes. It does not execute arbitrary database queries, web addresses, or shell commands generated by the model. A claim identifier locates a candidate record; it is not a permission slip.

4. Keep permissions on the server

The server derives Emily’s identity and organization from her verified session. It checks that she may access the requested claim on every lookup. Neither “I own it” in chat nor a generated owner field establishes ownership. OWASP recommends least privilege and authorization checks on every request.

Keep uploaded files and tool results untrusted. A returned note saying “ignore access checks” is data, not authority. The application owns the tool definition and schema; retrieved text cannot replace them. OWASP recommends limiting tool privileges and checking calls independently. A model-based warning can help, but cannot replace enforced access.

5. Return fields the application can check

Structured output uses predictable named fields instead of unrestricted prose. Here we use JSON. Our fictional expense system returns this small record:

{
  "claim_id": "EXP-1042",
  "status": "needs_receipt"
}

The application checks that the identifier matches the authorized request and that the status is a value its contract supports. It displays “Receipt required”directly from that validated backend field. The model can explain the status; it must not turn needs_receipt into approval or invent an exception.

Supported strict-output features can constrain generated arguments or responses to a schema. They have provider and schema limits, and applications must handle refusals and incomplete responses. OpenAI documents those conditions for structured outputs. Correct JSON syntax, schema matching, and business truth are separate checks. Formatting a guessed status perfectly would still leave it guessed.

6. Handle missing records and failed lookups

If Emily omits the claim identifier, ask for it. Do not guess from a dinner amount or silently choose another record. If access is denied or no permitted record is found, avoid revealing whether a colleague’s claim exists or what it contains.

A timeout means the status could not be checked. It does not mean “pending.” Keep errors distinct from successful results and match each result to its original call. Anthropic documents result matching, error signaling, and untrusted result content.

Missing fields, an unexpected status, or a mismatched identifier must not appear as verified data. Protect diagnostic logs too: record enough to investigate the failure without copying credentials, private receipts, or entire conversations into routine logs.

7. From one tool to an organized workflow

Our desk can now distinguish a policy question, a status lookup, and an unresolved exception. Each needs a different next step: retrieve a passage, call the narrow tool, or hand off to an authorized person. An orchestrator coordinates those steps and their outcomes. That is the next design problem, not something a tool name solves automatically.

Frequently asked questions

Sources & further reading

The primary research behind the article, for readers who want to explore the details.

1. Function calling OpenAI
2. Tool use with Claude Anthropic
3. Handle tool calls Anthropic
4. Structured model outputs OpenAI
5. object JSON Schema
6. Authorization Cheat Sheet OWASP
7. LLM Prompt Injection Prevention Cheat Sheet OWASP