DRAFT ONLY FOR DISCUSSION: DO NOT MERGE: Tools Alpha: See NOTES.md #2833
Conversation
|
…e those restrictions. All tools are still included with every completion request, but the `tool_choice` field is populated with allowed tools dynamically based on mode, and `tool_choice` is configured to require the model to respond with a `tool_call` rather than `stop`. This technique (rather than filtering the tools) keeps cache hit rates high for the function definitions, while still maintaining the needed controls
…s dont follow the darn spec. So now, tools are filtered at generation time, and only acceptable tools are sent with the request. This likely does have a small negative hit on cache utilization, but we'll see.
…alling providers The ToolUseControl component is now rendered only when the selected provider supports native function calling, as defined in the nativeFunctionCallingProviders array. This ensures that tool use options are only available for compatible providers, preventing potential errors or unexpected behavior in the UI.
Extract native tool call handling logic into reusable helper functions (addNativeToolCallsToParams, processNativeToolCallsFromDelta) to reduce code duplication in OpenRouter provider. Re-enable Morph fast apply feature which conditionally disables traditional editing tools when fast apply is available. Pass full providerState to getAllowedJSONToolsForMode to support fast apply availability checking.
013a059 to
171323c
Compare
|
|
||
| **Tidbits**: | ||
|
|
||
| - The cache hit rates are kinda crazy (often 90%): |
There was a problem hiding this comment.
Is that for GPT-5 or Sonnet?
|
I see it says DO NOT MERGE, but is there an objection to merging if we ensure everything is behind an experimental flag and there's no regressions in the XML tool flow? We also have: #2866, which has the advantage of showing partial tool calls (e.g. streaming diffs). Could that eventually also be implemented in the approach of this PR? This PR has the advantage of having a smaller diff with upstream in existing code I think. |
I have absolutely no objection to including this at all. I just didnt want anything to happen accidently I did confirm for every tool that it works both via XML and JSON calls. It just has no tests or anything but I'd love to have more folks using it. My results, and that of others, are very positive. I do believe that the technique @NaccOll built in his PR for streaming the tool calls could be incorporated here. |
No description provided.