Google’s announcement Introducing Tool Retrieval in the Gemini API (Nov 19, 2026) adds a feature to the Gemini Interactions API, currently in preview. It targets a problem every agent builder hits: once an agent connects to dozens or hundreds of tools, sending every full parameter schema on every request gets expensive.
This is a reading of the announcement, not a hands-on test. I did not run the API. Facts come from Google’s post. Where I add interpretation, I say so.
The workflow in motion
Example: one Interactions API request in which the model finds and calls a deferred tool.
The problem Google is solving
Google’s framing: sending every full parameter schema on every request inflates input token counts, adds latency, and lowers tool-selection accuracy as the candidate list grows. Tool Retrieval lets the agent keep its full tool library on standby and load a schema only when the current task needs it.
My interpretation: this is context engineering applied to tools. A schema is mostly a description of arguments, and for requests that need only a small subset of tools, the rest of the schemas are dead weight.
How server-side retrieval works
You add a tool_search tool to the request and set defer_loading: true on the tools you want to hold back.
- The model still sees the name and description of every deferred tool, so it knows what exists.
- The parameter schemas stay out of context.
- When the model needs a tool, it asks for it by name. The API loads the full schema and the model emits the function call, all inside the same request.
The three steps (tool_search_call, tool_search_result, function_call) all appear in interaction.steps. Google points out this makes it easy to inspect which tools the model considered when you debug tool names or descriptions.
The minimal request shape from the post:
tools = [
{"type": "tool_search"},
{
"type": "function",
"name": "issue_refund",
"description": "Refund an order to the original payment method.",
"defer_loading": True,
"parameters": {...},
},
# ... 50+ other deferred functions (leave frequent tools undeferred)
]
After that it is normal function calling: your code runs the tool and sends back a function_result.
Google stresses there is no index to build or host. Gemini runs the search on the server, so the only code changes are the new tool and a flag on the tools you defer. You can keep your most-used tools loaded up front and defer the long tail.
Prefix caching and MCP
Two details stand out.
Prefix caching. Google says deferred schemas stay out of the start of the prompt, and the schemas the model loads are added later in the conversation, so the cached prefix stays the same from turn to turn. My interpretation: this is why the design appends schemas rather than rewriting the tool list, since changing tools at the top of a prompt would normally invalidate the cache.
MCP. Tool Retrieval works with remote MCP servers in the Gemini API. Set defer_loading: true on an mcp_server tool and every tool that server exposes is loaded on demand, so you can connect a large server without listing its tools one by one.
Client-side retrieval
If you maintain your own tool registry, embedding index, or per-user permission filter, you can declare tool_search with "execution": "client" and define your own search parameters.
Trade-offs Google states: client-side retrieval adds one round trip per search, because your app runs the lookup. In return, tools do not need to be declared upfront, so you can expose new tools mid-conversation, for example when a user connects an integration.
Pricing and supported models
- There is no separate charge. Tokens are billed at the standard rates for the model you call.
- Turning on
tool_searchadds a small, fixed block of input tokens. - Each schema the model loads is billed as input for the rest of the conversation, the same as a tool declared up front.
- You save tokens when the schemas you never load outweigh the ones you do, which Google says is the usual case for large libraries. It also says to measure against your own traffic.
It works with Gemini 3.6 and later models, including all Gemini 4 models, in the Interactions API. It is in preview.
Design ideas worth borrowing
These are my interpretation of the announcement.
- Index cheap, load expensive. Keep a short catalog (name plus description) always visible, and fetch the heavy detail on demand. The same pattern works for skills, documents and memory.
- Descriptions become the retrieval surface. The model chooses from names and descriptions, so weak descriptions mean missed tools. Google’s step trace is a debugging aid for exactly this.
- Keep hot tools loaded. Deferral is per tool. Frequently used tools avoid an extra search step if left undeferred.
- Let the platform own the search, with an escape hatch. Server mode for zero infrastructure, client mode when you need your own ranking, permissions or dynamic tools.
- Append, don’t rewrite. Adding schemas later in the conversation is friendly to caching.
Where I would push back
- The announcement gives no accuracy or token-saving numbers. Google says to measure against your own traffic.
- Deferral adds a search step. Whether that costs latency in your workload is something I cannot tell from the post, even though the post lists lower latency as a goal.
- Retrieval quality depends on the model picking the right name from a description. A tool with a vague description may never be loaded.
- It is a preview, tied to the Interactions API and to Gemini 3.6 and later.
Takeaways
- Large tool libraries are a context problem before they are a model problem.
- Deferring schemas is a small API change (
tool_searchplusdefer_loading) with a potentially large effect on input tokens. - Write tool descriptions as if they were search keys, and test which tools the model actually loads.
Source: Google, Introducing Tool Retrieval in the Gemini API.
Building an agent with a large tool library? Get in touch.
Click to load Disqus comments