Tool calling in four steps: the request loop, with working Python

The short version

• The model does not execute anything. It returns JSON naming a function and its arguments; your code runs the function and sends the result back.
• That makes tool calling two requests: one to learn what the model wants to call, one to get an answer that uses the result.
• Three fields carry it: finish_reason is tool_calls, arguments is a JSON string you parse, and id pairs each result with its call.
• Cap the loop, validate the arguments, and pass only the tools a request can use: every definition is sent as prompt tokens on every call.

The four steps

1. Send the request with tools
        |
2. The model returns tool_calls (finish_reason: "tool_calls")
        |
3. Your code runs the function
        |
4. Send the result back as role "tool" and call again; the model answers in text

Step 1: define the tool

Declare each function in tools with a JSON Schema for its parameters:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name"},
                },
                "required": ["city"],
            },
        },
    }
]

The model reads description to decide whether to call the function at all, so write it as when to use the function. The same goes for each parameter's description: a vague one is how arguments get mixed up.

Step 2: read tool_calls

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["FASTMETAL_API_KEY"],
    base_url="https://api.fastmetal.ai/v1",
)

messages = [{"role": "user", "content": "What's the weather in Tokyo?"}]

resp = client.chat.completions.create(
    model="gemini-flash-lite-free",
    max_tokens=300,
    messages=messages,
    tools=tools,
)
print(resp.choices[0].finish_reason)
print(resp.choices[0].message.tool_calls)

We ran this on gemini-flash-lite-free in August 2026, with the question asked in Japanese. Trimmed to the fields that matter, the response was:

{
  "finish_reason": "tool_calls",
  "message": {
    "role": "assistant",
    "tool_calls": [
      {
        "id": "call_247545",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"city\": \"東京\"}"
        }
      }
    ]
  }
}

The argument is "Tokyo", written in Japanese because the question was. Three things to read from it:

• finish_reason is tool_calls: the model wants a function run. If it is stop, the model answered directly and there is nothing to run.
• arguments is a string of JSON. Parse it with json.loads().
• id is what you attach to the result in step 4.

gemini-flash-lite-free costs $0, so this whole post runs without credit. It is shared and rate-limited, and Google may use free-tier prompts for training: check the model's data handling in the catalog before sending anything real.

Step 3: run it in your code

This part is a plain dispatch table:

import json

def get_weather(city: str) -> dict:
    # call a real weather API here
    return {"city": city, "temp_c": 28, "condition": "sunny"}

AVAILABLE = {"get_weather": get_weather}

Don't trust the function name or arguments the model returns. It can name a function that doesn't exist or leave out a required argument. Check the name against the functions you know and validate the arguments before calling anything. Treat them as untrusted input: that is also your defense against prompt injection.

Step 4: return the result and call again

Append the assistant message that carries tool_calls to the history first, then one role: "tool" message per call:

msg = resp.choices[0].message

if msg.tool_calls:
    messages.append(msg)

    for call in msg.tool_calls:
        fn = AVAILABLE.get(call.function.name)
        if fn is None:
            result = {"error": f"unknown function: {call.function.name}"}
        else:
            args = json.loads(call.function.arguments)
            result = fn(**args)

        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result, ensure_ascii=False),
        })

    final = client.chat.completions.create(
        model="gemini-flash-lite-free",
        max_tokens=300,
        messages=messages,
        tools=tools,
    )
    print(final.choices[0].message.content)

tool_call_id must match the id from the first response, or the model can't tell which result belongs to which call. A response can contain several tool calls, so loop over all of them and return every one. Some models reject the request if a single result is missing.

Wrapping it in a loop

In practice you repeat until the model stops asking for tools:

def run(user_message: str, model: str = "gemini-flash-lite-free", max_turns: int = 5):
    messages = [{"role": "user", "content": user_message}]

    for _ in range(max_turns):
        resp = client.chat.completions.create(
            model=model, max_tokens=500, messages=messages, tools=tools,
        )
        msg = resp.choices[0].message

        if not msg.tool_calls:
            return msg.content

        messages.append(msg)
        for call in msg.tool_calls:
            fn = AVAILABLE.get(call.function.name)
            result = fn(**json.loads(call.function.arguments)) if fn else {"error": "unknown"}
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": json.dumps(result, ensure_ascii=False),
            })

    return "Stopped: reached the tool call limit"

Always set max_turns. When a tool keeps returning something the model doesn't expect, the model can call it again and again, and every turn is billed.

The loop runs on another model by changing model. That is what an OpenAI-compatible gateway buys you: one loop, any model in the catalog that supports tool calling.

Tool definitions cost tokens

tools is sent as part of the prompt on every request. With the one-parameter get_weather above, the same short question (asked in Japanese) measured on gemini-flash-lite-free in August 2026:

One small function added 53 prompt tokens. Ten functions add hundreds to every request, and a five-turn loop pays that five times. Pass only the tools that can apply to the request at hand. Paid models bill input per million tokens; rates are on the pricing page.

Common mistakes

• Treating arguments as an object. It is a JSON string; run json.loads() on it.
• Leaving the assistant message out of the history. Before any role: "tool" message, the assistant message with tool_calls must be there, or the model receives results for calls it never made.
• Omitting tool_call_id. It is required, and without it results can't be matched to calls.
• Looping with no limit. An endless loop runs until the balance does.
• Skipping argument validation. Treat model output like user input, and take extra care with functions that delete files, send email or change anything.

Where to go next

If you only want the reply in a fixed shape and have no function to run, structured output is simpler. Decide first whether you need to call a function or just need JSON.

To give an agent a ready-made set of tools, use MCP. FastMetal runs an MCP server at https://mcp.fastmetal.ai/mcp; setup is in the MCP guide.

FAQ

Does the model run my function?

No. It returns JSON naming a function and arguments. Your code runs it and sends the result back as role: "tool", and only then does the model write its final answer.

Does tool calling work on free models?

Yes. The code here was run on gemini-flash-lite-free. Free models are rate-limited, and a loop uses one request per turn.

Do tools add cost?

Yes. Tool definitions are part of every request's prompt. One small function measured 53 extra prompt tokens. Pass only the tools a request can use.

Which models support tool calling?

Most major models do. Check the catalog; the same code runs on each by changing model.

Try it

Create a free account, issue a key, and run one round trip on gemini-flash-lite-free. It costs nothing, and the same loop moves to a paid model by changing one string.