> ## Documentation Index
> Fetch the complete documentation index at: https://docs.keenable.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Every page is listed in https://docs.keenable.ai/llms.txt. To pick a product for a job, read https://docs.keenable.ai/choose.md
> Setup steps for agents, covering the SDKs, the MCP server and the CLI: https://docs.keenable.ai/skill.md
> Keenable search and fetch work without an API key. To add them to code, use the official SDK for the project's language: https://docs.keenable.ai/sdks.md
> Without a key, call POST https://api.keenable.ai/v1/search/public or GET https://api.keenable.ai/v1/fetch/public with an X-Keenable-Title header that names your app.
> With a key, send it in the X-API-Key header and read it from the KEENABLE_API_KEY environment variable. Never ask a user to paste a key into chat. Every access method: https://docs.keenable.ai/auth.md
> To get an API key without the console, use device login: https://docs.keenable.ai/authentication.md#device-login
> MCP server: https://api.keenable.ai/mcp (no key, or X-API-Key; keyless Codex uses https://api.keenable.ai/mcp-public), or https://api.keenable.ai/mcp-oauth for OAuth sign-in. These docs are an MCP server at https://docs.keenable.ai/mcp and the OpenAPI spec is at https://docs.keenable.ai/api-reference/openapi.json
> Keenable searches and fetches public web pages. If a question needs no web information, skip the call.
> On 402 with an API key (code insufficient_credits), stop and ask a human to add credits; don't retry. On 429, wait the number of seconds in Retry-After, then retry. Before you ship, follow https://docs.keenable.ai/production.md
> Every error, with its cause and fix, is in https://docs.keenable.ai/errors.md and moving from another search API is covered in https://docs.keenable.ai/migrate.md
> Before you compare Keenable with another search API, read https://docs.keenable.ai/guides/evaluate-search-apis.md

# Search API benchmarks: speed and open NEEDLE data

> Keenable's speed on the Artificial Analysis Search Index, and NEEDLE, the open benchmark we run: verticals, engines, raw data and how to re-score.

Check how fast Keenable answers on an independent board, then download the raw results of NEEDLE, the open benchmark we run, and score them yourself. To compare providers on your own queries, follow [Evaluate search APIs](https://docs.keenable.ai/guides/evaluate-search-apis.md).

**Updated at:** 19 November 2026, 12:44 UTC, the newest run of NEEDLE, the open benchmark we run, in this page's snapshot. The [run index](https://keenableai.github.io/needle/data/runs.json) lists every run since.

| What | Who runs it | Where to read it |
| - | - | - |
| Time per search call and per agent task | Artificial Analysis, an independent board | [Speed on Artificial Analysis](#speed-on-artificial-analysis) |
| Ranking and answer recall in five verticals | Keenable, as open source with public data | [NEEDLE dashboard](https://keenableai.github.io/needle/) |
| Results on your own queries | You | [Evaluate search APIs](https://docs.keenable.ai/guides/evaluate-search-apis.md) |

## Speed on Artificial Analysis

From the Artificial Analysis Search Index, published 18 August 2026:

* Fastest search call on the Artificial Analysis Search Index: 0.34 s.
* Lowest time per agent task on that board (15.1 s), about 1 s more than running with no search at all.

Artificial Analysis measured Keenable in `realtime` mode. See the [Cost table on Search](https://docs.keenable.ai/products/search.md#cost) for the two modes. Source: [Announcing the Artificial Analysis Search Index](https://artificialanalysis.ai/articles/search-api).

## NEEDLE

We publish NEEDLE, the open benchmark we run, live against 20+ search APIs; anyone can re-score the raw results. Keenable runs NEEDLE, so check any NEEDLE number against the raw data below. The code is MIT licensed and the results data is CC BY 4.0.

Every engine in a run gets the same query text. NEEDLE scores the ranking, titles and snippets each engine returns. It doesn't fetch pages or re-rank, the judge doesn't see engine names, and every engine's evidence is capped at the same snippet length. Queries are generated fresh from public sources, so there is no fixed set to tune for. The [NEEDLE dashboard](https://keenableai.github.io/needle/) has the full comparison protocol and the current results.

### Verticals

| Vertical | Queries | Scored by | Runs | Query file | Newest run in this snapshot |
| - | - | - | - | - | - |
| Scholar | Find a specific paper from a degraded title, a full-text detail or a half-remembered description | recall\@10 and MRR, by arXiv ID, DOI or PMID match | Twice a day | `scholar.jsonl` | [2026-11-19T0927Z](https://huggingface.co/datasets/keenable-ai/needle-results/tree/main/runs/2026-11-19T0927Z) |
| News | Breaking and developing stories, new each hour from public RSS feeds and Google Trends | nDCG\@5 from an LLM judge's 0 to 4 ratings | Every hour | `fresh.jsonl` | [2026-11-19T1201Z](https://huggingface.co/datasets/keenable-ai/needle-results/tree/main/runs/2026-11-19T1201Z) |
| Finance | Company and SEC filing lookups with known answers from public registries | answer-recall\@5 and MRR\@5: is the answer in a top 5 title or snippet | Twice a day | `gold.jsonl` | [2026-11-19T0414Z](https://huggingface.co/datasets/keenable-ai/needle-results/tree/main/runs/2026-11-19T0414Z) |
| AgenticRare | Obscure, long-tail entities, sampled from public agentic search logs | nDCG\@5 from an LLM judge's 0 to 4 ratings | Twice a day | `agentic_rare.jsonl` | [2026-11-19T0534Z](https://huggingface.co/datasets/keenable-ai/needle-results/tree/main/runs/2026-11-19T0534Z) |
| Legal | A specific court opinion or section of the Code of Federal Regulations | recall\@5 and MRR\@5, by citation, docket or URL match | Twice a day | `legal.jsonl` | [2026-11-19T1244Z](https://huggingface.co/datasets/keenable-ai/needle-results/tree/main/runs/2026-11-19T1244Z) |

### Engines

On 19 November 2026, the [NEEDLE dashboard](https://keenableai.github.io/needle/), for the open benchmark we run, listed 25 engines. These are their CLI ids, which `--engines` takes, with the dashboard name where it differs: `keenable-realtime` (shown as Keenable), `perplexity`, `perplexity-fast`, `exa`, `exa-lite`, `brave-llmcontext`, `parallel` (shown as parallel-advanced), `context` (shown as context.dev), `brave`, `parallel-fast`, `you`, `linkup`, `parallel-turbo`, `tinyfish`, `deepseek`, `claude-search`, `kagi`, `octen`, `chatgpt-search`, `firecrawl`, `bing` (shown as bing/searchapi), `google` (shown as google/serper), `tavily`, `jina`, `seltz`.

A vendor with more than one mode or endpoint appears once per mode, such as `exa` and `exa-lite`. The dashboard's Keenable row is `keenable-realtime`, Keenable in `realtime` mode. The NEEDLE CLI also has `keenable`, which runs `pro` mode, and engines the dashboard doesn't show. The [NEEDLE README](https://github.com/keenableai/needle#configuration) lists each engine's API key variable.

## Public data

Everything behind a NEEDLE number is public:

| Data | What it holds | Where |
| - | - | - |
| Per-run reports | Every query and result (title, URL, snippet), judge ratings with reasoning, per-call latencies and the run's settings. Each engine's results are in `<vertical>/<engine>.json`. | [keenable-ai/needle-results](https://huggingface.co/datasets/keenable-ai/needle-results) on Hugging Face, under `runs/<run-id>/` |
| Query files | The exact queries each run sent, with gold answers where the vertical has them | The same run folder |
| Run index | Every run id, its time and the files it produced | [runs.json](https://keenableai.github.io/needle/data/runs.json) |
| Results tables | The current results as CSV, JSON, Markdown and PDF, plus monthly snapshots | [NEEDLE results page](https://keenableai.github.io/needle/results/) |
| Code | Engine clients, the judge prompt and the metric code | [keenableai/needle](https://github.com/keenableai/needle) on GitHub |

## Re-score a run

Run Keenable again on the exact queries of a published run, then compare your report with the one in the run folder. You need [uv](https://docs.astral.sh/uv/) and git.

This re-runs Keenable on the exact queries of the Legal run of 19 November 2026, 12:44 UTC:

```bash Terminal theme={"theme":{"light":"github-light-high-contrast","dark":"dark-plus"}}
git clone https://github.com/keenableai/needle.git
cd needle
uv sync
uvx --from huggingface_hub hf download keenable-ai/needle-results --repo-type dataset \
  --include "runs/2026-11-19T1244Z/*" --local-dir needle-results
uv run needle legal run --queries needle-results/runs/2026-11-19T1244Z/legal.jsonl \
  --engines keenable-realtime --out legal.json
```

You get `legal.json`, a report with recall\@5 and MRR\@5 for `keenable-realtime`, broken down by suite (case law or regulation code), query syntax and court. Compare it with the published `legal.json` in the run folder, or with `legal/keenable-realtime.json` for that engine alone. Your numbers can differ, because the live web changes between runs.

* Without `KEENABLE_API_KEY`, the CLI of NEEDLE, the open benchmark we run, calls Keenable's keyless endpoint, which allows 1,000 requests per hour per IP address. See [Rate limits](https://docs.keenable.ai/rate-limits.md).
* To add other engines, pass them to `--engines` and set each engine's API key. The [NEEDLE README](https://github.com/keenableai/needle#configuration) lists the variables.
* Legal and Scholar score by ID match, so re-running them on a published query file needs no LLM key. News and AgenticRare use an LLM judge, so re-running them needs `OPENROUTER_API_KEY`. So does generating new queries for most verticals.
* To re-score without searching again, read each engine's per-query results from the run folder and apply the metric code in `src/needle/shared/metrics.py`.

## Next steps

* [Evaluate search APIs](https://docs.keenable.ai/guides/evaluate-search-apis.md): build a gold set and compare providers with the same settings.
* [Search](https://docs.keenable.ai/products/search.md): parameters, modes and cost.
* [Migrate to Keenable](https://docs.keenable.ai/migrate.md): map another provider's parameters to Keenable's.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.