Nimble Search

Nimble Search: Search Index Score, Performance & Price

Name: Artificial Analysis Search Index
Creator: Artificial Analysis
License: https://artificialanalysis.ai/docs/legal/Terms-of-Use.pdf

Compare Search APIs

Search Index

Artificial Analysis Search Index for Nimble Search (standard)

Time

23.9s

Average model and search time per task for Nimble Search (standard)

Cost

$0.063

Average model and Search API cost per task (USD) for Nimble Search (standard)

Compared with other Search APIs, Nimble Search is average on the Search Index and somewhat expensive. It's also faster than average. Its best variant is Nimble Search (standard), out of 2 variants tested.

Nimble Search (standard) scores 74 on the Artificial Analysis Search Index, placing it at the median among Search API providers (median: 74).

At 23.9s per task, Nimble Search (standard) is faster than average (median: 31.6s).

On average, it costs $0.063 per task to run Nimble Search (standard) on the Search Index, including the candidate model (somewhat expensive, median: $0.061).

Best variant	Nimble Search (standard)
Candidate model	GPT-5.6 Luna (medium)
Variants tested	2

Results

Search-enabled answer quality across public benchmarks.

Artificial Analysis Search Index

Equal-weighted mean of DeepSearchQA F1, BrowseComp accuracy, and AA-Omniscience accuracy · Higher is better

Model only (33)

Integrated first-party search tool and harness

The equal-weighted mean of each Search API provider's score on DeepSearchQA (average F1), BrowseComp (exact-answer accuracy), and AA-Omniscience (accuracy), shown on a 0-100 scale. Every result runs the same candidate answer model, harness, and settings. The only variable is the Search API provider.

The dashed line represents the candidate answer model, GPT-5.6 Luna (medium), run with no search tools and is considered the baseline. Dotted bars show the same model with an integrated first-party search tool and harness (OpenAI Web Search). See the Search API methodology for the full harness settings.

Cost

Candidate model cost and Search API cost per benchmark task and per search query.

Cost per Task

Average candidate model cost and Search API cost per task (USD) · Lower is better

Model only ($0.0029)

Integrated first-party search tool and harness

Total cost of one benchmark task, split into the candidate answer model's tokens (input, cached, reasoning, and output) and the Search API provider's list price for the searches the model chose to run. The model-only baseline runs no searches, so its cost is token cost alone. Model cost can differ per provider due to differences in returned payloads for a given query, increased numbers of searches performed (due to not getting the right results) as well as increased reasoning token usage when search results are of lower quality.

Artificial Analysis Search Index vs. Cost per Task

Artificial Analysis Search Index vs. total model and search cost per benchmark task (USD)

Most attractive quadrant

Pareto line

Latency

Model time and search time per benchmark task and per search query.

Time per Task

Average derived model time and measured search time per task · Model time is output tokens divided by end-to-end output speed, at the candidate model's own endpoint or the median across providers · Lower is better

Model only (25.6s)

Integrated first-party search tool and harness

The sum of model time and search time for one benchmark task. Model time is derived: answer and reasoning tokens per task divided by the model's end-to-end output speed, at its own endpoint or the median across providers. Search time is the measured time spent in web_search calls. A provider can be fast per call and still add more total time if the model searches against it more often.

For OpenAI Web Search, search time is an upper limit. It includes OpenAI processing time.

Artificial Analysis Search Index vs. Time per Task

Artificial Analysis Search Index vs. average model and search time per benchmark task (seconds) · Model time is output tokens divided by end-to-end output speed, at the candidate model's own endpoint or the median across providers

Most attractive quadrant

Pareto line

For OpenAI Web Search, search time is an upper limit. It includes OpenAI processing time.

Tokens

Candidate model input and output tokens per benchmark task, and quality against total token usage.

Total Tokens per Task

Average candidate model input, reasoning, and answer tokens per benchmark task · Lower is better

Model only (3k)

Integrated first-party search tool and harness

Average total input, reasoning and answer tokens per task. Grader tokens are excluded.

Artificial Analysis Search Index vs. Total Tokens per Task

Artificial Analysis Search Index vs. total candidate model tokens per benchmark task

Most attractive quadrant

Pareto line

Average total input, reasoning and answer tokens per task. Grader tokens are excluded.

Searches

Search queries the candidate model runs per benchmark task.

Search Queries per Task

Average number of search queries the candidate model makes per benchmark task

The average number of web_search calls the candidate answer model makes for one benchmark task, averaged across the three benchmarks. The model decides when to search, so the count reflects how it works with each provider's results. A higher count is not necessarily worse, a provider may return results intentionally designed to encourage more, frequent searches. The model-only baseline runs no searches.

Benchmark Details

Search Index, benchmark scores, cost and time per task for Nimble Search, with the model-only baseline

Provider
Nimble Search (standard)	74	41	76	75	71	$46.14	$16.64	23.9s
Nimble Search (lite)	66	33	61	69	67	$20.71	$9.58	45.2s
Model only	33	baseline	45	17	38	—	$2.92	25.6s

Search $/1k and Model $/1k are the average cost of 1,000 benchmark tasks broken down into search and model token costs. The model-only baseline runs no searches.

For OpenAI Web Search, search time is an upper limit. It includes OpenAI processing time.

Nimble Search: Search Index Score, Performance & Price

Provider summary

Search Index

Time

Cost

Comparison Summary

Technical specifications

Results

Artificial Analysis Search Index

Artificial Analysis Search Index

Model-Only Baseline and First-Party Search

Cost

Cost per Task

Cost per Task

Model-Only Baseline and First-Party Search

Artificial Analysis Search Index vs. Cost per Task

Cost per Task

Latency

Time per Task

Time per Task

Model-Only Baseline and First-Party Search

Artificial Analysis Search Index vs. Time per Task

Time per Task

Tokens

Total Tokens per Task

Total Tokens per Task

Model-Only Baseline and First-Party Search

Artificial Analysis Search Index vs. Total Tokens per Task

Total Tokens per Task

Searches

Search Queries per Task

Search Queries per Task

Benchmark Details

Cost per 1,000 Tasks

Time per Task