Tool search is becoming the default MCP pattern — here’s how to measure when it beats progressive discovery

MCP tool sprawl turns context into a tax you pay on every turn. Tool search (deferred loading) is emerging as the practical workaround, but you should only adopt it if you can show it improves tool-pick accuracy and end-to-end latency, not just token counts.

Why “always inject every tool schema” stopped scaling

If you connect enough MCP servers, the naïve pattern is simple: enumerate every tool, inject every JSON schema, and let the model pick.

It also fails in predictable ways.

• Context-window clutter: every request carries tool definitions the model will never use. GitHub explicitly frames on-demand tool loading (tool search) as a response to context-window clutter from unused tool definitions. (docs.github.com)
• Choice overload: the model has to pick from a long, flat list of tool names whose differences are subtle, especially when servers expose many near-duplicate endpoints.
• Higher steady-state latency: if you attach the full catalogue every turn, you pay the serialisation + tokenisation + model-attention cost every time, even when the user is doing a small follow-up.
• Security surface: more text in the prompt means more places for prompt-injection and “tool description poisoning” to land. MCP’s own spec warns that tool behaviour descriptions and annotations should be treated as untrusted unless coming from a trusted server. (modelcontextprotocol.io)

The key point: you can’t decide this by vibes. You need to measure it on your workload.

The three patterns you should compare

There are only a few realistic strategies teams converge on.

(A) Always-inject schemas (baseline)

You load all tools (names + descriptions + parameter schemas) into the model context at session start, and keep them available.

This is simplest to implement and easiest to reason about. It’s also the first thing that collapses as tool count grows.

(B) Tool search / deferred loading (two-step)

You provide a small “bridge” tool that lets the model search a tool catalogue and then request the full schema only for shortlisted tools.

This is not hypothetical. Claude Code documents this approach for MCP: only tool names load at session start, and tools are deferred and discovered via a search tool when needed. (code.claude.com)

GitHub Copilot’s documentation similarly describes on-demand tool loading to avoid cluttering the context window with unused tool definitions. (docs.github.com)

(C) Grouped / progressive discovery

You expose a curated subset of tools up-front (often per workflow), and progressively reveal more groups based on user intent or a gating decision.

In the MCP community, “progressive discovery” and grouping is actively discussed as a response to tool sprawl and poor client/server coordination, with explicit acknowledgement that tool search is not equally supported in all model/client environments. (github.com)

What to measure (and what not to hand-wave)

If you only measure “tool tokens saved”, you’ll optimise the wrong thing. Treat these as first-class metrics.

• Token footprint per turn: not just the initial prompt. Track the average and p95 “prompt tokens” over a multi-turn task.
• Tool-pick accuracy: did the agent select the correct tool (and correct server) for the step?
• Argument correctness: even with the right tool, did it produce valid, complete parameters?
• End-to-end task latency: wall-clock time from user request to final answer, including extra tool-search turns.
• Recovery cost: when it picks wrong, how many extra turns/tool calls does it take to self-correct?

A practical detail: GitHub Copilot CLI notes that tool calls and tool results sit in the same context window as everything else. That means “just one more search step” is not free; it can increase both latency and context pressure on long tasks. (docs.github.com)

The evaluation harness: how to build it without fooling yourself

You want a harness that can replay the same task set across the three patterns and produce comparable traces.

1) Define a task set that forces tool choice

If the tasks don’t force ambiguity, every approach will look fine.

Include:

• “Single obvious tool” tasks (sanity check)
• “Two plausible tools” tasks (naming collisions, similar capabilities)
• “Multi-step workflow” tasks (where early tool choice affects later feasibility)
• “Follow-up question” tasks (to stress per-turn context growth)

Do not use tasks that can be answered from general model knowledge; you’re evaluating tool routing and execution.

2) Freeze a tool universe and catalogue representation

For each MCP server, snapshot:

• tool name
• short description
• full JSON schema for params
• any server metadata you’ll allow the agent to see

MCP itself is defined via a typed schema and uses JSON Schema for validation throughout the protocol. (modelcontextprotocol.io) That makes it feasible to build a stable, canonical “tool catalogue” representation for experiments.

Important: keep the tool universe fixed across runs. Otherwise you’re measuring churn.

3) Implement the three routing strategies as swappable “policies”

Your agent loop should accept a routing policy:

• inject_all: include the full tool schemas in the system/developer context
• tool_search: include only a lightweight index + a search tool that returns a small shortlist and lets the agent request schemas
• progressive_groups: include only a group’s schemas at first; unlock additional groups via an explicit gate

Make sure policies differ only in tool-availability mechanics, not in prompting style or model.

4) Instrument everything, including the boring bits

At minimum, record per run:

• prompt tokens, completion tokens (per turn)
• number of tool calls
• tool call latency (per tool)
• total wall-clock latency
• chosen tool name + server
• whether tool call succeeded
• whether the final answer matches your expected outcome

If you can, also log the “candidate set size” the model was choosing from at the moment of selection. That’s usually where the signal is.

5) Score tool-pick accuracy with explicit labels, not heuristics

Tool-pick accuracy sounds straightforward until you have near-duplicates.

Use a label scheme:

• correct tool: the intended tool
• acceptable alternative: different tool that still produces the same correct outcome
• wrong tool: produces incorrect outcome or leaks data / violates policy

This also lets you compare progressive discovery fairly: if it hides tools, it may force “acceptable alternatives”, which can still be a win.

6) Make latency comparable across patterns

Tool search usually adds at least one extra model turn:

1. search
2. read schema
3. call tool

That can still win overall if it reduces per-turn prompt size and reduces misroutes.

So measure:

• task latency (user request → final answer)
• model latency (sum of LLM calls)
• tool latency (sum of tool calls)

Then look at where time moves.

Failure modes and “do not do this” guidance

Tool search is not a universal upgrade.

• Do not do tool search if your model/client stack can’t reliably use it. The MCP maintainers explicitly note that some models and agents “do not support tool search well or at all”. (github.com)
• Do not do tool search if most tasks consistently require the same small set of tools. In that world, progressive grouping (or just inject the small set) will likely be faster and simpler.
• Be careful with tool search if your tools are extremely latency-sensitive. Adding an extra round trip can dominate user-perceived performance.
• Be careful with progressive discovery if it creates “tool dead-ends”: the agent commits to a group, gets partial schemas, and later discovers it can’t complete the workflow without re-planning.
• Be careful with always-inject when security and trust boundaries are weak. More tool text increases exposure to indirect prompt injection and tool-description payloads; this risk is actively discussed in recent MCP security work. (coalitionforsecureai.org)

One more failure mode people miss: tool search can create a new kind of “routing hallucination”, where the model confidently selects a tool from a thin index, then never bothers to request the schema detail that would have shown it the mismatch. Your harness should explicitly check “schema requested before call” for tool-search runs.

Interpreting results: when tool search actually wins

On real systems you typically see one of three outcomes.

1. Tool search wins when you have many tools, low reuse of the same tools across tasks, and meaningful naming collisions. It reduces choice overload, cuts per-turn prompt size, and can improve tool-pick accuracy enough to offset the extra step.

1. Progressive grouping wins when your product has clear workflows (sales ops, incident response, code review) and you can curate tool subsets that stay stable. You keep a single-turn selection while avoiding dumping the whole universe.

1. Always-inject wins when the tool set is small, stable, and heavily reused. Don’t over-engineer.

The point of the harness is that you can say, with evidence: “for our workload, pattern B reduced wrong-tool calls and decreased p95 task time” (or: “it made latency worse and didn’t improve accuracy, so we’re not doing it”).

A pragmatic implementation detail: keep the catalogue authoritative

If you adopt tool search, treat the tool catalogue as a product surface, not an emergent property.

That means:

• canonical naming conventions
• descriptions written for disambiguation (not marketing)
• schema hygiene (required vs optional, enums, units)
• explicit deprecation and versioning

MCP’s typed schema and JSON Schema validation conventions make this doable, but they don’t enforce good semantics. (modelcontextprotocol.io)

Where Codeversols fits (briefly)

If you’re planning to ship MCP integrations across multiple internal systems, the hard part is rarely “wire up a tool server”. It’s the evaluation harness, the routing policy, and the operational discipline around catalogues, permissions, and regression testing. Codeversols builds that sort of end-to-end engineering across AI, cloud, and product workflows, typically by standing up the harness early so decisions about tool search vs discovery are measurement-led rather than ideological.

Close

Tool search is becoming the default MCP pattern because it tackles a real scaling limit: context-window clutter and tool sprawl. But it is not automatically better than progressive discovery, and it can absolutely make latency worse.

Build the harness, run the three policies on your real tasks, and let token footprint, tool-pick accuracy, and end-to-end latency decide.