Kitaru by ZenML Open source · Apache 2.0

Traces you can run, not just read

Score what happened. Replay what could have.

Your agent’s real traces become frozen, replayable worlds for your real code — the test suite production already wrote.

Lightweight, wraps your existing agent

uv add kitaru

Book a demo Star on GitHub

replay · support-agent case

# import existing traces from anywhere

kitaru executions import trace://8f3a91c2

# score what happened — the agent never runs

executions.evaluate(scorers=[resolved])

# replay against your real code

kagent.replay(exec, repeats=10)

# fork the failing step, swap the tool

kagent.replay(exec, tools={"refund": stub_ok})

# fix — then verify on every failure like it

matches = executions.where(resolved < 0.5)

kagent.replay(matches, name="refund-errors")

Step Outcome

Score every trace 23 fail the same way

Replay ×10 failure reproduces 10 / 10

Fork · healthy tool agent recovers — fault isolated

Fix · all 23 PASS — guards every PR

The complaint is now a regression test that runs on every PR.

THE SAME MOVE, EVERY TIME

One recorded execution.
A different question each time.

Replay the execution. Change one thing. Read the answer off a diff. Nothing reruns in production.

debug

Why did it do that?

Reproduce the failure on your desk — same code, same world, every time.

10 / 10 runs reproduce it

verify

Is the fix safe to merge?

Replay it against every recorded run that failed the same way.

23 / 23 pass after the fix

decide

Can we ship the cheaper model?

Same conversations, cheaper model — read the answer off a diff.

−84% cost · 192/200 answers identical

gate

Did the new prompt regress?

Replay last week's traffic against the branch before it merges.

0 regressions reach production

ONE WRAP, EVERY CALL RECORDED

Keep your agent SDK. Make its runs replayable.

Wrap the calls and steps your SDK exposes, and Kitaru stores them as replayable checkpoints. That recording is what replay reads back, so a what-if is faithful, not a guess.

Before agent.py

from agents import Runner
from agents.sandbox import SandboxAgent, Manifest

agent = SandboxAgent(
    name="Compliance Reviewer",
    model="gpt-5-mini",
    default_manifest=manifest,
)

result = await Runner.run(agent, task)

With Kitaru agent.py

from kitaru.adapters.openai_agents import KitaruRunner, OpenAIRunRequest
from agents.sandbox import SandboxAgent, Manifest

agent = SandboxAgent(
    name="Compliance Reviewer",
    model="gpt-5-mini",
    default_manifest=manifest,
)

runner = KitaruRunner(agent, checkpoint_strategy="calls")
result = await runner.run(OpenAIRunRequest.start(task))

Before agent.py

from claude_agent_sdk import query, ClaudeAgentOptions

options = ClaudeAgentOptions(
    system_prompt="You are a compliance reviewer.",
    allowed_tools=[],
)

async for msg in query(prompt=task, options=options):
    process(msg)

With Kitaru agent.py

from kitaru import flow
from claude_agent_sdk import ClaudeAgentOptions
from kitaru.adapters.claude_agent_sdk import ClaudeRunRequest, ClaudeRunResult, KitaruClaudeRunner

runner = KitaruClaudeRunner(
    name="compliance_review",
    options_factory=lambda request: ClaudeAgentOptions(
        system_prompt="You are a compliance reviewer.",
        allowed_tools=[],
        max_turns=request.max_turns,
    ),
)

@flow
def review(task: str) -> ClaudeRunResult:
    return runner.run_sync(ClaudeRunRequest.start(task, max_turns=3))

Before agent.py

from pydantic_ai import Agent

agent = Agent(
    "openai:gpt-5-mini",
    system_prompt="You are a compliance reviewer.",
    tools=[search_docs, fetch_policy],
)

result = await agent.run(task)

With Kitaru agent.py

from kitaru.adapters.pydantic_ai import KitaruAgent
from pydantic_ai import Agent

agent = KitaruAgent(Agent(
    "openai:gpt-5-mini",
    system_prompt="You are a compliance reviewer.",
    tools=[search_docs, fetch_policy],
))

result = await agent.run(task)

Before agent.py

from google import genai

client = genai.Client()

interaction = client.interactions.create(
    model="gemini-3.5-flash",
    input=task,
)
print(interaction.output_text)

With Kitaru agent.py

from kitaru import flow
from kitaru.adapters.gemini import (
    GeminiInteractionRequest, KitaruGeminiInteractionsRunner,
)

runner = KitaruGeminiInteractionsRunner(name="review")

@flow
def review(task: str):
    request = GeminiInteractionRequest.start(
        task, model="gemini-3.5-flash",
    )
    return runner.run_sync(request)

Before agent.py

from google.adk.runners import Runner

runner = Runner(
    agent=agent,
    app_name="support",
    session_service=sessions,
)

result = await runner.run_async(
    user_id="user-123",
    session_id="s-456",
    new_message=msg,
)

With Kitaru agent.py

from kitaru.adapters.google_adk import ADKRunRequest, KitaruADKRunner

kitaru_runner = KitaruADKRunner(
    adk_runner,
    name="support_agent",
    checkpoint_strategy="runner_call",
)

result = await kitaru_runner.run(ADKRunRequest(
    user_id="user-123",
    session_id="s-456",
    message=msg,
))

Before agent.py

from anthropic import Anthropic

client = Anthropic()

def my_agent(task: str) -> str:
    plan = analyze(client, task)
    # nothing recorded — nothing to replay or test against.
    result = execute(client, plan)
    return result

With Kitaru agent.py

from kitaru import flow, checkpoint
from anthropic import Anthropic

client = Anthropic()

@flow
def my_agent(task: str) -> str:
    plan = checkpoint(analyze)(client, task)
    result = checkpoint(execute)(client, plan)
    return result

THE WHAT-IF WORKFLOW

Ask anything of the runs you already have.

Cheaper model? Flaky tool result? New prompt? Replay real executions with one thing changed, read the diff — and never touch production.

Start from Kitaru-recorded executions

Wrap your agent so model/tool calls and steps exposed through Kitaru primitives or adapters become replayable checkpoints.

Pydantic AI

LangGraph

OpenAI Agents SDK

Claude Agent SDK

Gemini

Google ADK

Raw Python

01 Replay

Start from execution IDs

200 real executions whose Kitaru-visible steps and model/tool calls are already checkpointed. Production stays untouched.

exec_ids = cohort.exec_ids

02 Change one thing

Cheaper model, flaky tool

Replay from the decision point with a cheaper model, or replace a recorded lookup result. Production stays exactly as it was.

replay(id, from_="decide", model="fast")

03 Diff

See what would have happened

Compare each candidate to its source execution: cost, changed outputs, first divergence. The decision now has evidence.

cheap.llm_usage_summary

The diff

Same 200 executions. Cheap model. Order lookup result replaced.

−61% median cost

196 / 200 outputs identical

4 to review

Illustrative numbers. Tool-level detail depends on the primitives or adapter boundaries your agent exposes to Kitaru.

SEE THE CODE

Every counterfactual is one replay call.

Swap a model, replace a checkpoint result, fail a call — each a simulation against the recorded world, one change at a time. Hover any line to see what it does.

whatif.py

import kitaru
 
client = kitaru.KitaruClient()
# your agent exposes replayable checkpoints through Kitaru
source = support_agent.run(ticket)
result = source.wait()
 
# reproduce faithfully, then change ONE thing
baseline = client.executions.replay(
source.exec_id, from_="decide",
)
cheaper = client.executions.replay(
source.exec_id, from_="decide", model="glm-5.4",
)
faked = client.executions.replay(
source.exec_id, from_="lookup_order",
overrides={"checkpoint.lookup_order": stale_order},
)
timeout = client.executions.replay(
source.exec_id, from_="lookup_order", lookup_mode="timeout",
)
 
usage = cheaper.llm_usage_summary

support_agent.run()

Run in production. Keep the returned handle: it carries the execution ID that replay needs.

replay(from_="decide")

Reproduce from a checkpoint with no change. Earlier checkpoints reuse recorded outputs. The faithful baseline.

model="glm-5.4"

Pass a different flow input for this replay. Same execution history, cheaper model choice.

overrides=checkpoint...

Replace one recorded checkpoint result. Kitaru replays the consumers of that changed value.

lookup_mode="timeout"

Use a flow input your agent already understands to run the replay against a timeout path.

llm_usage_summary

Inspect shipped execution metadata, checkpoints, and artifacts to compare the source execution and replay.

AGENT-DRIVEN EXPERIMENTATION

Don’t run the experiments. Let your agent hill-climb.

Kitaru exposes replay and diff over an MCP server and a CLI. Point your coding agent at it: it replays a run, changes one thing, reads the diff, and tries the next idea, climbing toward a cheaper, safer version while you watch.

Drive it from

Claude Code

Codex

Cursor

Gemini CLI or any harness that speaks MCP

agent · kitaru mcp

replay #4127 · from_="decide" · model="gpt-5-mini"

81% cheaper, but skipped the escalation ✗

replay #4127 · from_="decide" · model="fast"

78% cheaper, decisions match ✓

kept this variant, opening a PR

A few replays, no human in the loop. The diff is the agent’s reward signal.

What arrives as a complaint
leaves as a regression test.

uv add kitaru

Book a demo Star on GitHub

Open source (Apache 2.0). Replay real runs against your real code. Nothing reruns in production.

Traces you can run, not just read

One recorded execution.A different question each time.

Keep your agent SDK. Make its runs replayable.

Ask anything of the runs you already have.

Start from execution IDs

Cheaper model, flaky tool

See what would have happened

Every counterfactual is one replay call.

Don’t run the experiments. Let your agent hill-climb.

Foundations and proof for Kitaru

From ZenML to Kitaru

Why agents need durable execution

Agents need more than traces

What arrives as a complaint leaves as a regression test.

One recorded execution.
A different question each time.

What arrives as a complaint
leaves as a regression test.