TOOLS as is.
How the loop works
- Your code sends the conversation and
TOOLSto the local server. - If the model wants the web, the assistant message holds
tool_calls, each with afunctionthat has anameandarguments. - Your code appends the assistant message, runs each call with
run_tool_call, and appends onerole: "tool"message per call. - When the assistant message has no tool calls, its
contentis the answer.
Prerequisites
- Ollama or vLLM installed, with a model that supports tool calling.
- Python 3 or Node.js with
tsxto run the program. - No Keenable key. The SDK searches without one. To raise the rate limit, set
KEENABLE_API_KEYto an API key.
Tool definition (copy verbatim)
Ollama and vLLM take each tool astype: "function" with a nested function object holding name, description and parameters. That is the format of the Keenable SDK’s TOOLS:
Chat Completions format
TOOLS directly, so the definitions always match the SDK version you installed. If you paste the JSON instead, check the SDK column of the arguments table: an older run_tool_call drops any argument its version doesn’t offer.
Run the loop with Ollama
Pull a model that supports tool calling. This page usesqwen3:
ollama-tools.mts or ollama_tools.py. It asks a question, runs every tool call the model makes, and prints the answer:
npx tsx ollama-tools.mts or python ollama_tools.py while Ollama is running. The program prints the model’s final answer.
Ollama returns arguments as an object, which run_tool_call takes as is, and the tool message names the tool with tool_name. If a tool call fails, the loop sends the error message as the tool result so the model can retry. Errors inside a tool call lists what run_tool_call raises.
Run the loop with vLLM
vLLM returns tool calls only when the server starts with--enable-auto-tool-choice and a --tool-call-parser that matches the model. For Qwen2.5 models the parser is hermes:
llama3_json for Llama 3.1 and mistral for Mistral models.
The server speaks the OpenAI Chat Completions API, so the loop on Tool calling runs against it unchanged. Point it at vLLM with these variables. vLLM ignores the key unless you started it with --api-key:
OPENAI_BASE_URL=http://localhost:11434/v1/ and OPENAI_MODEL=qwen3. Ollama requires a key value but ignores it.