Skip to main content
A model running on your own hardware decides when to search the web or read a page, and your code runs each call with the Keenable SDK. Ollama and vLLM both take tools in the format the SDK exports, so you pass TOOLS as is.

How the loop works

  1. Your code sends the conversation and TOOLS to the local server.
  2. If the model wants the web, the assistant message holds tool_calls, each with a function that has a name and arguments.
  3. Your code appends the assistant message, runs each call with run_tool_call, and appends one role: "tool" message per call.
  4. When the assistant message has no tool calls, its content is the answer.
The model must support tool calling. A model without it answers from its own knowledge and never calls the tools.

Prerequisites

  • Ollama or vLLM installed, with a model that supports tool calling.
  • Python 3 or Node.js with tsx to run the program.
  • No Keenable key. The SDK searches without one. To raise the rate limit, set KEENABLE_API_KEY to an API key.

Tool definition (copy verbatim)

Ollama and vLLM take each tool as type: "function" with a nested function object holding name, description and parameters. That is the format of the Keenable SDK’s TOOLS:
Chat Completions format
The loops below pass the SDK’s TOOLS directly, so the definitions always match the SDK version you installed. If you paste the JSON instead, check the SDK column of the arguments table: an older run_tool_call drops any argument its version doesn’t offer.

Run the loop with Ollama

Pull a model that supports tool calling. This page uses qwen3:
Install the Keenable SDK and Ollama’s SDK:
Save this program as ollama-tools.mts or ollama_tools.py. It asks a question, runs every tool call the model makes, and prints the answer:
Run it with npx tsx ollama-tools.mts or python ollama_tools.py while Ollama is running. The program prints the model’s final answer. Ollama returns arguments as an object, which run_tool_call takes as is, and the tool message names the tool with tool_name. If a tool call fails, the loop sends the error message as the tool result so the model can retry. Errors inside a tool call lists what run_tool_call raises.

Run the loop with vLLM

vLLM returns tool calls only when the server starts with --enable-auto-tool-choice and a --tool-call-parser that matches the model. For Qwen2.5 models the parser is hermes:
vLLM’s tool calling docs list the parser for each model family, such as llama3_json for Llama 3.1 and mistral for Mistral models. The server speaks the OpenAI Chat Completions API, so the loop on Tool calling runs against it unchanged. Point it at vLLM with these variables. vLLM ignores the key unless you started it with --api-key:
To run the same loop against Ollama instead, set OPENAI_BASE_URL=http://localhost:11434/v1/ and OPENAI_MODEL=qwen3. Ollama requires a key value but ignores it.

Differences from other providers