Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

feat(Mocking): pass execution trace history into LLM mocker#666

Merged
bai-uipath merged 1 commit intomainUiPath/uipath-python:mainfrom
bai/llm-mocker-parityUiPath/uipath-python:bai/llm-mocker-parityCopy head branch name to clipboard
Oct 13, 2025
Merged

feat(Mocking): pass execution trace history into LLM mocker#666
bai-uipath merged 1 commit intomainUiPath/uipath-python:mainfrom
bai/llm-mocker-parityUiPath/uipath-python:bai/llm-mocker-parityCopy head branch name to clipboard

Conversation

@bai-uipath
Copy link
Collaborator

@bai-uipath bai-uipath commented Oct 9, 2025

Add a span processor collect in-progress OpenTelemetry traces during execution and pass them to the LLM mocker. For example, in a calculator evaluation run that calls the get_random_operator tool twice, the LLM mocker gets the following run history added to its context.

First LLM mock:

CURRENT AGENT RUN SO FAR:
(empty)

Second LLM mock:

CURRENT AGENT RUN SO FAR:
get_random_operator
  Input: {}
  Output: {"result": "+"}

@github-actions github-actions bot added test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-llamaindex Triggers tests in the uipath-llamaindex-python repository labels Oct 9, 2025
@bai-uipath bai-uipath changed the title [WIP] feat(Mocking): Pass execution trace history into LLM mocker feat(Mocking): Pass execution trace history into LLM mocker Oct 10, 2025
@bai-uipath bai-uipath marked this pull request as ready for review October 10, 2025 00:02
@bai-uipath bai-uipath requested a review from akshaylive October 10, 2025 00:02
@bai-uipath bai-uipath force-pushed the bai/llm-mocker-parity branch from 7f940b0 to e34be3f Compare October 10, 2025 19:07
@bai-uipath bai-uipath changed the title feat(Mocking): Pass execution trace history into LLM mocker feat(Mocking): pass execution trace history into LLM mocker Oct 10, 2025
@bai-uipath bai-uipath force-pushed the bai/llm-mocker-parity branch from e34be3f to 35fa5f6 Compare October 10, 2025 19:12
src/uipath/_cli/_evals/_runtime.py Outdated Show resolved Hide resolved
src/uipath/_cli/_evals/_runtime.py Outdated Show resolved Hide resolved
src/uipath/tracing/_utils.py Show resolved Hide resolved
src/uipath/_cli/_evals/mocks/mocks.py Outdated Show resolved Hide resolved
@bai-uipath bai-uipath force-pushed the bai/llm-mocker-parity branch from cc56100 to 2ac9da1 Compare October 10, 2025 23:09
src/uipath/tracing/_utils.py Outdated Show resolved Hide resolved
@bai-uipath bai-uipath force-pushed the bai/llm-mocker-parity branch from a3cbd6d to 1d31e7c Compare October 13, 2025 17:50
@bai-uipath bai-uipath requested a review from akshaylive October 13, 2025 18:05
Copy link
Collaborator

@akshaylive akshaylive left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢

@bai-uipath bai-uipath merged commit cd8fe88 into main Oct 13, 2025
59 checks passed
@bai-uipath bai-uipath deleted the bai/llm-mocker-parity branch October 13, 2025 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-llamaindex Triggers tests in the uipath-llamaindex-python repository

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.