MCP tool discovery is breaking down on large servers: sending a flat list of every tool (schemas included) to the model is turning into context bloat, slow reasoning, and fragile behaviour. Once you cross roughly “dozens” of tools — and especially when teams talk about 160+ — you’re no longer doing “tool calling”; you’re effectively shipping a second API surface into the prompt.
This is not a theoretical concern. The MCP community is actively working on “primitive grouping” (with an Interest Group meeting and published notes in mid‑August 2026) because practitioners are hitting overload and lack of protocol affordances for progressive discovery and coordination. (github.com)
What actually fails when tool lists get large
MCP’s core affordance is simple: a server exposes tools, clients can list them (tools/list) and invoke them. (modelcontextprotocol.io) The failure mode in large deployments isn’t that tools/list stops working; it’s that the interaction design collapses.
Three things tend to go wrong together:
- Context burn: the tool catalogue (names, descriptions, JSON Schemas) becomes a meaningful fraction of the model’s context window, forcing you to choose between tools and user/business context.
- Tool selection degradation: the model has too many near-neighbour tools and starts choosing “plausible” calls rather than correct ones (or it gets stuck in tool browsing loops).
- Policy & access control drift: once tools are numerous and heterogeneous, the difference between “visible”, “callable”, and “safe to reveal” becomes non-trivial — and a flat list encourages accidental disclosure or overly broad visibility.
The MCP spec even calls out deterministic ordering to support caching and prompt cache hit rates when tools are included in the model context — which is telling: we’re already optimising around the fact that tool lists are expensive. (github.com)
The three patterns people are converging on
Most teams trying to operate a “big MCP server” converge on one of these patterns (or a hybrid):
- Protocol-level grouping (the primitive grouping direction)
- Client-side tool search / skills (semantic retrieval over the tool catalogue)
- Progressive discovery (don’t send everything; reveal tools on demand)
AWS’s prescriptive guidance frames the same tension bluntly: conserve context window, and consider semantic search over tools rather than dumping the full list. (docs.aws.amazon.com) The point is not which pattern is fashionable; it’s which failure mode you can tolerate.
Option A: Primitive grouping (protocol-level organisation)
Primitive grouping is trying to give the ecosystem a shared way to say “these tools belong together”, without each server inventing its own taxonomy. The Interest Group notes (Aug 14, 2026) explicitly cite “too many tools, context overload” and “lack of good protocol affordances for progressive discovery”. (github.com) The experimental repository exists precisely to explore how organisation might work at the primitive/protocol level. (github.com)
What grouping buys you
Grouping is mainly about human and model navigation:
- You can keep a single server endpoint while making it legible: “billing”, “incidents”, “deployments”, “reports”.
- You can shape tool selection by giving the model structure: it can narrow to a group before it reasons about specific tools.
- You can apply coarse policy at the group level (visibility, environment restrictions), if the protocol/client supports it.
Where grouping breaks
Grouping does not solve the hardest parts on its own:
- It can become a second API surface: every team bikesheds the taxonomy, and clients have to learn it.
- It doesn’t inherently reduce the token cost if you still ship schemas for every tool in every group.
- It can create false confidence: “tools are grouped, therefore safe”. But access control is still tool- and data-dependent.
Do not do this if your main problem is token cost and you cannot avoid shipping full schemas. Grouping without progressive disclosure often reads well and performs badly.
Option B: Client-side tool search / “skills” (retrieval over tools)
Here the client (or an MCP gateway) keeps the full tool catalogue out of the model’s prompt by default. Instead, it performs a search (often semantic) to select a small candidate set of tools for the current turn.
This is now mainstream enough that both vendor guidance and research literature describe it explicitly: list tools, then do semantic search, then present a small subset. (docs.aws.amazon.com) There’s also recent work on “hybrid semantic tool discovery” in an enterprise MCP gateway context — notably positioning the gateway as a governable chokepoint for auth/policy/observability. (arxiv.org)
What tool search buys you
- Token and attention budgeting: the model sees 5–20 tools, not 160.
- Better ergonomics for multi-server estates: a gateway can unify tooling across teams without forcing a single mega-server.
- A place to implement policy: the retrieval step can incorporate permissions, environment, data sensitivity, and user role.
Where tool search breaks
Tool search can fail in ways that are operationally painful:
- Recall failures become production incidents: the “right” tool exists but doesn’t surface; the agent improvises with the wrong tool.
- Tool description quality becomes a hard dependency: if names/descriptions are inconsistent, retrieval quality collapses. There is active research specifically arguing that MCP tool descriptions are frequently poor and measurably affect agent performance. (arxiv.org)
- Debugging is harder: you need logs that show “tools available”, “tools retrieved”, and “why”. Otherwise it’s indistinguishable from model caprice.
Do not do this if you cannot invest in disciplined tool metadata (names, descriptions, examples), and in observability for retrieval decisions.
Option C: Progressive discovery (reveal tools on demand)
Progressive discovery is the idea that the model should not see every tool up front. Instead, it asks for what it needs: “show me tools in group X”, “show me tools matching Y”, or “describe tool Z in detail now”. The Primitive Grouping IG notes explicitly call out the lack of good protocol affordances for progressive discovery today, which is why many teams are implementing ad-hoc versions of it. (github.com)
In practice, you see patterns like:
- a minimal
tools/listreturning only high-level stubs (or just a few safe tools) - a separate “catalogue” tool that returns subsets
- schema-on-demand (only provide full JSON Schema when a tool is about to be invoked)
What progressive discovery buys you
- The best token economics if done properly: you pay for detail only when the conversation actually needs it.
- A safer default posture: you can avoid revealing sensitive tool names or capabilities to unauthorised sessions.
- A more “agentic” flow: browsing is a tool use, not a prompt dump.
Where progressive discovery breaks
- It is easy to create an infinite browse loop: the agent keeps asking for more catalogue information rather than acting.
- You can accidentally invent an internal protocol: “catalogue tool” semantics become bespoke, and clients/models depend on them.
- You must get caching semantics right. The spec’s emphasis on deterministic ordering and cacheability hints at the operational complexity here. (github.com)
Do not do this if you can’t commit to a stable discovery contract. “We’ll just add a browse tool” becomes technical debt quickly.
Choosing between them: a pragmatic decision rubric
If you are deciding what to build in the next 4–12 weeks, choose based on what you’re optimising for.
If your problem is token cost and latency
- Prefer progressive discovery or client-side tool search.
- Treat grouping as a secondary UX layer, not the core solution.
If your problem is tool selection correctness
- Prefer tool search with strong metadata + evaluation harness.
- Add grouping only if it reflects real domain boundaries (not organisational charts).
If your problem is governance (access control, audit, environment separation)
- Prefer a gateway approach that can enforce policy before tools are even eligible for retrieval. The “enterprise MCP gateway” framing in recent work is exactly this: authentication, policy enforcement, observability at the chokepoint. (arxiv.org)
- Add progressive discovery to reduce information disclosure by default.
If you already have 1 server with 160+ tools
Do not start by refactoring everything into a perfect hierarchy.
- Implement tool search to cap what the model sees per turn.
- Introduce progressive detail (schemas on demand) to cut context burn.
- Add grouping once you have evidence about how tools cluster in real usage.
A concrete “don’t build a second API” checklist
You can keep discovery from turning into its own brittle subsystem if you enforce a few constraints:
- One discovery path: either group→list→invoke, or search→invoke. Avoid having both be “equally valid”.
- Hard caps: set a maximum number of tools returned to the model per turn; force the system to refine the query/group if it wants more.
- Stable identifiers: never make the model rely on display names for selection; use stable tool IDs.
- Schema discipline: keep schemas minimal and consistent; large, optional-heavy schemas are selection poison.
- Observability: log what the model saw, what was eligible, what was retrieved, and what was invoked.
The moment you can’t explain, from logs, why a tool was eligible and why it was chosen, you are operating on vibes.
Where Codeversols fits (briefly)
If you’re building an MCP estate that’s crossing from “a few tools” into “a platform”, the hard part is not wiring up tools/list; it’s designing discovery, permissions, and evaluation so the system stays predictable as tool count grows. That typically means a mix of gateway engineering, retrieval, caching, and disciplined tool design across teams — the kind of cross-cutting work we do at Codeversols across AI, cloud, and product engineering.
Close
Flat MCP tool lists are fine until they aren’t, and the break point arrives suddenly when a server becomes a real product surface. Grouping helps humans and models navigate; tool search controls token cost and improves selection; progressive discovery is the best lever for both cost and disclosure — but it demands a stable contract. Pick the pattern whose failure mode you can contain, then add the others only when you have evidence you need them.
