Give Your Local LLM Real Tools: Function Calling with Ollama in Under 50 Lines

Ollama has native tool calling built in — define a schema, wire up a dispatch table, and any Qwen3 or Llama 3.1 model running locally can call your Python functions with validated arguments.

What you'll be able to do after this:

• Define tools (function name, description, typed parameters) that a local LLM can call by name with validated arguments
• Drive a multi-turn agent loop where the model decides when to invoke tools and what arguments to pass — the same pattern that powers cloud-based AI agents, fully offline
• Extend any Ollama-backed chatbot into one that can query databases, call APIs, read files, or run shell commands without sending data to the cloud

The pattern in 45 lines

pip install ollama
ollama pull qwen3:8b

import ollama

# 1. Your real function
def get_current_weather(city: str) -> str:
    return f"72F and sunny in {city}"   # swap for a real weather API call

# 2. Tool schema — what the model reads to know when and how to call it
tools = [{
    "type": "function",
    "function": {
        "name": "get_current_weather",
        "description": "Get current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "The city name"}
            },
            "required": ["city"]
        }
    }
}]

dispatch = {"get_current_weather": get_current_weather}
messages  = [{"role": "user", "content": "What's the weather in Portland?"}]
MODEL     = "qwen3:8b"   # or llama3.1:8b, mistral:7b-instruct

# 3. Multi-turn agent loop — runs until no more tool calls
response = ollama.chat(model=MODEL, messages=messages, tools=tools)

while response.message.tool_calls:
    for tc in response.message.tool_calls:
        result = dispatch[tc.function.name](**tc.function.arguments)
        messages += [
            response.message,
            {"role": "tool", "content": str(result)}
        ]
    response = ollama.chat(model=MODEL, messages=messages, tools=tools)

print(response.message.content)

Key points:

• response.message.tool_calls is a list — the while loop handles parallel tool calls (model fires multiple tools at once) automatically.
• Each tool result goes back as a {"role": "tool", "content": ...} message before the next ollama.chat() call — this is the multi-turn pattern.
• The loop exits when the model returns a response with no tool calls (it has enough information to answer).
• For streaming tool calls (seeing partial arguments as they generate), switch to ollama.chat(..., stream=True) and accumulate content_block_delta events — the Ollama docs cover this pattern.

Models with reliable tool support in 2026: qwen3:8b, qwen3:14b, llama3.1:8b-instruct-q4_K_M, llama3.3:70b-instruct-q4_K_M, mistral:7b-instruct.

The IBM Think tutorial linked below shows the same pattern applied to searching a local filesystem — a Granite 3.2 model picks between a text-search tool and an image-search tool based on query content. Good "real" example beyond a toy weather function.

Sources: Tool Calling — Ollama Docs · Local Tool Calling with Ollama and IBM Granite — IBM Think