Add Tools warm_up #9856

vblagoje · Oct 7, 2025

Why:

Enables tools and toolsets to perform initialization during Agent/ToolInvoker warmup. This supports tools that need to establish remote connections, load models, initialize connection pools, or perform other setup operations before use.

What:

Added warm_up() method to Tool and Toolset as extension points for initialization
Created warm_up_tools() utility function to warm up lists of tools or Toolsets
Modified ToolInvoker to warm up tools during initialization
Optimized duplicate checking in Toolset.__add__() to avoid redundant validation
Added focused tests covering warmup delegation and integration

How can it be used:

from haystack.tools import Toolset
from haystack.components.agents import Agent

# Custom toolset with initialization needs
class DatabaseToolset(Toolset):
    def __init__(self, connection_string):
        self.connection_string = connection_string
        self.pool = None
        super().__init__([query_tool, update_tool])
        
    def warm_up(self):
        # Initialize connection pool
        self.pool = create_connection_pool(self.connection_string)

How did you test it:

Added comprehensive tests in test/tools/test_toolset_warmup_and_wrapper.py
Verified warmup delegation through _ToolsetWrapper to each wrapped toolset
Manual tests with this PR and mcp-haystack with warm_up implemented in itinerary agent

Notes for the reviewer:

_ToolsetWrapper.warm_up() delegates to each wrapped toolset - critical for preserving individual initialization behavior when combining toolsets
No breaking changes - warm_up() defaults to no-op in base classes
Works with previous versions of Tool implementation "out there". We check if there is warm_up attr and call it only if found.

coveralls · Oct 7, 2025

Pull Request Test Coverage Report for Build 18531976909

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0 changed or added relevant lines in 0 files are covered.
85 unchanged lines in 13 files lost coverage.
Overall coverage increased (+0.02%) to 92.084%

Files with Coverage Reduction	New Missed Lines	%
core/pipeline/pipeline.py	1	89.83%
tools/component_tool.py	2	94.25%
tools/tool.py	2	97.7%
components/converters/csv.py	4	95.45%
components/generators/azure.py	4	91.84%
components/generators/chat/azure.py	4	93.55%
components/generators/openai.py	4	94.2%
components/generators/chat/openai.py	6	96.76%
core/pipeline/breakpoint.py	7	88.76%
components/tools/tool_invoker.py	9	96.02%

Totals
Change from base Build 18340248289:	0.02%
Covered Lines:	13297
Relevant Lines:	14440

💛 - Coveralls

vblagoje · Oct 8, 2025

Note to @anakin87 @julian-risch - @sjrl and I talked about invoking tool warm up in all chat generators via their warm_up method. Implementation in this PR will make Agent work because now ToolInvoker warms up tools. But to have pipelines working with tools warmup (ChatGenerator + ToolInvoker pipeline without Agent) we need to add warm_up to all ChatGenerators. However, that is scope for another PR.

vblagoje · Oct 10, 2025

@sjrl @julian-risch @anakin87 can I 🙏 have some feedback on this PR so we can push it forward!

julian-risch

Left some comments. My my concern is the renaming of serde_utils.py to utils.py, which is a breaking change. Other than that looks quite good to me already.

haystack/tools/__init__.py

haystack/tools/tool.py

haystack/tools/toolset.py

haystack/tools/utils.py

haystack/components/tools/tool_invoker.py

sjrl · Oct 13, 2025

@vblagoje I believe we should also update the Agent's warm up method to warm up it's tools as well right? Possibly I think would also make sense to update the ChatGenerators in main to do so as well. What do you think?

test/tools/test_toolset_wrapper.py

sjrl · Oct 13, 2025

haystack/tools/toolset.py

+class _ToolsetWrapper(Toolset):
+    """
+    A wrapper that holds multiple toolsets and provides a unified interface.
+
+    This is used internally when combining different types of toolsets to preserve
+    their individual configurations while still being usable with ToolInvoker.
+    """


@vblagoje is it possible that this wrapper could also solve this issue deepset-ai/haystack-core-integrations#2369 if we add to_dict and from_dict methods?

haystack/tools/toolset.py

sjrl · Oct 13, 2025

And letting ToolInvoker warm_up is enough.

Agent doesn't call ToolInvoker.warm_up in it's warm_up method so when is the Agent warming up the tools exactly? From looking where warm_up_tools was placed in ToolInvoker (inside of ToolInvoker._validate_and_prepare_tools) it seems we are still warming up Tools at init time since ToolInvoker calls _validate_and_prepare_tools in it's init method. Wouldn't this still cause the issue we were having before?

Did you check that this branch would cause the test added here deepset-ai/haystack-core-integrations#2357 to work as expected?

vblagoje · Oct 13, 2025

And letting ToolInvoker warm_up is enough.

Agent doesn't call ToolInvoker.warm_up in it's warm_up method so when is the Agent warming up the tools exactly? From looking where warm_up_tools was placed in ToolInvoker (inside of ToolInvoker._validate_and_prepare_tools) it seems we are still warming up Tools at init time since ToolInvoker calls _validate_and_prepare_tools in it's init method. Wouldn't this still cause the issue we were having before?

Did you check that this branch would cause the test added here deepset-ai/haystack-core-integrations#2357 to work as expected?

Yes you are correct we talked about doing it there in _validate_and_prepare_tools but we obviously can't. That makes me conclude we need:

a mandatory ToolInvoker warm_up (needed for regular, no Agent pipelines)
We can call either CG or ToolInvoker warm_up in Agent warm_up and be ok because Tools are not split into multiple instances.

Is that correct? If so I'll adjust this PR to move tools warm_up to ToolInvoker and we can then call it in Agent warm_up
I'd leave a change to warm_up CGs to another PR. With this PR we cover Agent and in the next one non-Agent pipelines?
Do you agree? @sjrl

sjrl · Oct 13, 2025

Yes you are correct we talked about doing it there in _validate_and_prepare_tools but we obviously can't. That makes me conclude we need:

a mandatory ToolInvoker warm_up (needed for regular, no Agent pipelines)

We can call either CG or ToolInvoker warm_up in Agent warm_up and be ok because Tools are not split into multiple instances.

Is that correct? If so I'll adjust this PR to move tools warm_up to ToolInvoker and we can then call it in Agent warm_up I'd leave a change to warm_up CGs to another PR. With this PR we cover Agent and in the next one non-Agent pipelines? Do you agree? @sjrl

Let's add a ToolInvoker.warm_up method and for safety let's also call ToolInvoker.warm_up inside of Agent.warm_up. We already call CG.warm_up in Agent.

I'd leave a change to warm_up CGs to another PR

That's fine we can leave the update to ChatGenerator warm ups to another PR.

vblagoje · Oct 14, 2025

I've tested this version of haystack with mcp and warmup and it works ok. However, as we discussed above, Agent warm_up needs to be called to warm up tools. Is that ok for no code envs? cc @julian-risch @sjrl

sjrl · Oct 14, 2025

@vblagoje how do you mean by this comment?

Agent warm_up needs to be called to warm up tools. Is that ok for no code envs?

I think this is fine. Not entirely sure how this applies for no code envs

haystack/components/tools/tool_invoker.py

haystack/tools/tool.py

sjrl · Oct 14, 2025

Looking good @vblagoje I have a one more comment in addition to the few above

Could we create a warm_up method for ComponentTool to call the underlying component's warm_up method?

haystack/tools/component_tool.py

releasenotes/notes/tools-warm-up-support-e16cc043fed3653f.yaml

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

sjrl

@vblagoje looks good to me!

Sebastian re-reviewed and approved. My earlier comments have been addressed.

vblagoje added the ignore-for-release-notes PRs with this flag won't be included in the release notes. label Oct 7, 2025

github-actions bot added the type:documentation Improvements on the docs label Oct 7, 2025

vblagoje added 6 commits October 8, 2025 11:30

Tools warmup initial

74a3e95

Fix lint

27967e2

Improve pydocs for warm_up

5bc77fe

Further improve pydocs for warm_up

bda2195

No need to warm_up tools in Agent as they are warmed up by ToolInvoker

f9b947c

Simplify Toolset __add__ logic

5b4fdea

vblagoje force-pushed the tools_warmup_new branch from 50c39a2 to 5b4fdea Compare October 8, 2025 09:31

vblagoje added 2 commits October 8, 2025 11:36

Simplify _ToolsetWrapper

335ced9

Add unit tests

d294485

github-actions bot added the topic:tests label Oct 8, 2025

vblagoje marked this pull request as ready for review October 8, 2025 10:03

vblagoje requested a review from a team as a code owner October 8, 2025 10:03

vblagoje requested review from anakin87, julian-risch and sjrl and removed request for a team October 8, 2025 10:03

vblagoje changed the title ~~Tools warmup~~ Add Tools warm_up Oct 8, 2025

vblagoje mentioned this pull request Oct 8, 2025

Add Tool/Toolset warm_up #9849

Closed

julian-risch previously requested changes Oct 10, 2025

View reviewed changes

haystack/tools/__init__.py Outdated Show resolved Hide resolved

haystack/tools/tool.py Show resolved Hide resolved

haystack/tools/toolset.py Show resolved Hide resolved

haystack/tools/toolset.py Show resolved Hide resolved

julian-risch reviewed Oct 10, 2025

View reviewed changes

haystack/tools/utils.py Show resolved Hide resolved

sjrl reviewed Oct 13, 2025

View reviewed changes

haystack/components/tools/tool_invoker.py Outdated Show resolved Hide resolved

sjrl reviewed Oct 13, 2025

View reviewed changes

test/tools/test_toolset_wrapper.py Outdated Show resolved Hide resolved

sjrl reviewed Oct 13, 2025

View reviewed changes

haystack/tools/toolset.py Show resolved Hide resolved

vblagoje added 10 commits October 13, 2025 14:51

ToolInvoker warm_up

498066a

Improve Tool pydoc

e1a6e38

Resurrect serde_utils.py

a9d0ee2

Update tests

9aa752e

Call ToolInvoker warm_up in agent warm_up

09b4f8c

Lint

075dec3

Move warm_up tests to ToolInvoker

0dc218c

Update tests

a187d6e

Remove tests

69b6113

Pydoc nit

b24149a

sjrl reviewed Oct 14, 2025

View reviewed changes

haystack/components/tools/tool_invoker.py Outdated Show resolved Hide resolved

sjrl reviewed Oct 14, 2025

View reviewed changes

haystack/tools/tool.py Outdated Show resolved Hide resolved

vblagoje added 3 commits October 14, 2025 14:20

PR feedback

80b154c

ToolInvoker's warm_up is idempotent

8f65bc1

Add reno note

c1fed85

vblagoje removed the ignore-for-release-notes PRs with this flag won't be included in the release notes. label Oct 14, 2025

vblagoje mentioned this pull request Oct 14, 2025

Add MCPTool/MCPToolset warm_up deepset-ai/haystack-core-integrations#2384

Draft

sjrl reviewed Oct 15, 2025

View reviewed changes

haystack/tools/component_tool.py Outdated Show resolved Hide resolved

sjrl reviewed Oct 15, 2025

View reviewed changes

releasenotes/notes/tools-warm-up-support-e16cc043fed3653f.yaml Outdated Show resolved Hide resolved

vblagoje and others added 2 commits October 15, 2025 16:09

Update releasenotes/notes/tools-warm-up-support-e16cc043fed3653f.yaml

3edc510

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

Make ComponentTool warm_up idempotent

45240be

sjrl approved these changes Oct 16, 2025

View reviewed changes

vblagoje mentioned this pull request Oct 17, 2025

feat: Update tools param to Optional[Union[list[Union[Tool, Toolset]], Toolset]] #9886

Open

Search code, repositories, users, issues, pull requests...

Add Tools warm_up #9856

Are you sure you want to change the base?

Add Tools warm_up #9856

Uh oh!

Conversation

vblagoje commented Oct 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why:

What:

How can it be used:

How did you test it:

Notes for the reviewer:

Uh oh!

coveralls commented Oct 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 18531976909

Warning: This coverage report may be inaccurate.

Details

💛 - Coveralls

Uh oh!

vblagoje commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vblagoje commented Oct 10, 2025

Uh oh!

julian-risch left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sjrl commented Oct 13, 2025

Uh oh!

Uh oh!

sjrl Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sjrl commented Oct 13, 2025

Uh oh!

vblagoje commented Oct 13, 2025

Uh oh!

sjrl commented Oct 13, 2025

Uh oh!

vblagoje commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sjrl commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sjrl commented Oct 14, 2025

Uh oh!

Uh oh!

Uh oh!

sjrl left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vblagoje commented Oct 7, 2025 •

edited

Loading

coveralls commented Oct 7, 2025 •

edited

Loading

vblagoje commented Oct 8, 2025 •

edited

Loading

vblagoje commented Oct 14, 2025 •

edited

Loading

sjrl commented Oct 14, 2025 •

edited

Loading