Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

feat(channels): render arbitrary JSX as images via Takumi + static chart components#6146

Open
AlemTuzlak wants to merge 35 commits into
mainCopilotKit/CopilotKit:mainfrom
feat/channels-jsx-imageCopilotKit/CopilotKit:feat/channels-jsx-imageCopy head branch name to clipboard
Open

feat(channels): render arbitrary JSX as images via Takumi + static chart components#6146
AlemTuzlak wants to merge 35 commits into
mainCopilotKit/CopilotKit:mainfrom
feat/channels-jsx-imageCopilotKit/CopilotKit:feat/channels-jsx-imageCopy head branch name to clipboard

Conversation

@AlemTuzlak

Copy link
Copy Markdown
Contributor

What & why

Channels' JSX vocabulary (<Message>, <Button>, …) renders to native platform components (Slack Block Kit, Teams Adaptive Cards). But arbitrary app JSX — a shadcn-styled card, a dashboard tile — had no path to a channel. This adds one: thread.post(<AnyStaticJSX/>) now auto-renders arbitrary JSX to a PNG via Takumi (browser-free, in-process) and uploads it through the existing postFile, while channel-UI components keep rendering natively. No browser anywhere.

How it works

  • Auto-routing by peeking at output (render/detect.tsresolveArbitraryElement): a React element, or an unbranded component that returns a React element → image; channels-ui components (branded via CHANNEL_COMPONENT) and user components that return channel nodes → native. This peek-at-output approach is what lets a developer's own native channel components and arbitrary app JSX coexist with zero call-site ceremony. channels-ui's empty IntrinsicElements + a JSX.ElementType declaration make React-returning components valid JSX tags in channels-pragma files.
  • Thread.post routes; update/postEphemeral/awaitChoice reject arbitrary JSX with a clear error (an uploaded image can't be edited in place / can't dispatch a click).
  • Fonts & CSS: createChannel({ render: { fonts, stylesheets, width, height } }), overridable per-post. Takumi resolves var()/oklch in compiled CSS for arbitrary cards (shadcn theming works).
  • react + takumi-js are optional peer deps of channels-core/channels, imported only from the lazy render/takumi.ts — a consumer who never posts an image never loads them.

Built-in charts — @copilotkit/channels/charts

Browser-free static chart components (geometry computed in TS, no Recharts/Chart.js/Mermaid): BarChart, StackedBar, Sparkline, Meter (box-model) and LineChart, PieChart, Scatter (inline SVG). Styleable via colors/className/style/gridColor/labelClassName; fixed default palette. A real-Takumi render-smoke test asserts each chart renders its data in a visible palette color.

examples/slack migration

Removes the Playwright/headless-Chromium render path (render/{browser,chart,diagram}.ts, the Chart.js/Mermaid tools). render_mrr now posts an arbitrary-JSX MrrCard + a <BarChart> as images via Takumi. playwright is kept only as a devDependency for the e2e Slack-login harness. (Mermaid diagram rendering is dropped — no browser-free equivalent.)

Docs

New guide showcase/shell-docs/.../channels/posting-jsx-as-images.mdx (auto-routing, render config, per-chart styling/behavior table, peer-install prerequisite, and the plain limitations: static SSR only — no Recharts/Chart.js/Mermaid).

Notes

  • Verified via an isolated Takumi spike: it ingests inline SVG (rect/polyline/path-arc/circle) and resolves var() fallbacks in backgroundColor but not in SVG fill/stroke — so the chart palette is concrete hex, not CSS vars.
  • One pre-existing flaky @copilotkit/vue SSR test had its timeout raised (it times out under the pre-commit full-suite load, unrelated to this change).
  • Went through an extensive review-and-fix loop; deferred, pre-existing, out-of-subject follow-ups (e.g. a repo-wide createBotcreateChannel docs rename, an awaitChoice waiter-leak on handler-throw, an MCP client leak in the example's connectMcp) are intentionally left for separate PRs.

🤖 Generated with Claude Code

Alem Tuzlak added 28 commits July 23, 2026 12:18
Adds render_mrr, replacing the removed Playwright render_chart tool: posts
an MrrCard (plain createElement-authored React component) and, when a
signups series is passed, a BarChart from @copilotkit/channels/charts --
both routed to the Takumi image path automatically via thread.post's
arbitrary-JSX detection, no wrapper needed.

Also widens channels-ui's JSX.ElementType (TS 5.1 tag/element decoupling)
so a function component that returns a real ReactElement (e.g. BarChart)
can be used directly as a JSX tag under the channels pragma without
tripping TS2786 -- the type checker previously required every tag's
return type to be assignable to JSX.Element (ChannelNode), which broke
exactly the "arbitrary app JSX" pattern this demo exercises. Adds
@types/react to examples/slack so React types actually resolve there.
… render-smoke color assertion

Takumi does not resolve CSS var()/theme tokens for SVG stroke/fill, so
DEFAULT_CHART_COLORS is now a fixed hex palette instead of shadcn
--chart-N custom properties. PieChart returns an empty <svg> for
all-non-positive data instead of a misleading full disk, and colors a
single positive slice by its original index in `data`. Meter clamps NaN
input to 0. render-smoke.test.ts decodes rendered PNGs and asserts a
palette color is actually visible (not just a non-empty file), guarding
against the invisible-var() failure mode. Docs corrected to match:
charts theme via the `colors` prop, not automatic shadcn token
adoption, and the per-chart styling-prop support matrix is documented.
…rds; correct Meter height docs; guard-branch tests
…harts; readonly colors; wider render-smoke
…generate centering; coverage + empty-id warn
@github-actions

github-actions Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Production Digest-Pinning Audit

All 41 services digest-pinned.

Run 2026-07-24 09:09:17 PDT — 0 finding(s).

Alem Tuzlak and others added 6 commits July 24, 2026 12:07
… standup

Three "run it in our own Slack" showcase features under app/showcase/, each a
shadcn-styled card + charts rendered to images via Takumi, and each triggerable
BOTH by a slash command and by prompt (a defineChannelTool the agent calls) via
one shared render* fn:

- /prs      render_pr_radar     - open PRs oldest-first, age-coloured badges +
                                  PRs-by-age bar chart (public GitHub, no token)
- /pulse    render_weekly_pulse - KPI card (stars/downloads/open issues) +
                                  downloads line chart + issues bar chart
                                  (public GitHub + npm)
- /standup  render_standup      - progress card + status pie (with legend) +
                                  load-per-assignee bar (Linear, LINEAR_API_KEY)

The shadcn look is one token stylesheet fed via render.stylesheets; text is
Geist (Takumi's built-in font). Every feature reads live data and falls back to
labelled sample data (never throws), with per-request timeouts so a stalled
upstream still degrades. Wires showcaseTools/showcaseCommands into the channel,
declares the commands in both Slack manifests, and documents them in the README
and the posting-jsx-as-images guide.
… per-team standup

Addresses testing feedback on the Slack showcase features.

charts (@copilotkit/channels/charts):
- Takumi does not rasterize SVG <text>, so LineChart's title/axis labels never
  rendered. Rework LineChart to keep the line/grid/points in SVG but draw the
  title, a 5-tick y-axis scale, and per-point x labels as HTML around it.
- BarChart/StackedBar now print each bar's value (compacted, e.g. "1.2k") above
  it, via a shared formatCompact helper.

examples/slack:
- Stop the agent re-posting earlier turns' images: history is rebuilt from Slack
  message text each turn and image posts leave none, so the model re-satisfied
  every past request. Each render now leads with a one-line text summary (which
  survives reconstruction), and the runtime prompt gains a "latest request only"
  rule plus the three render_* tools.
- Size the SVG charts (LineChart) to the post canvas so they fill the image
  instead of sitting in a corner.
- Redesign the standup: query all Linear teams, compute per-team cycle
  completion, and render a per-team progress card (a meter per team) plus a
  done-vs-remaining stacked bar.

Docs + tests updated to match.
…createElement

Let apps author image cards as plain JSX (`<div className="card">…`) instead of
`createElement`, while native channel components keep working unchanged.

channels-ui:
- The JSX runtime now compiles host/intrinsic tags (<div>, <span>, <svg>) to
  real React elements, and component tags (<Message>, <BarChart>, <Meter>, app
  cards) to ChannelNodes. A React element's $$typeof distinguishes host markup
  (image path) from the string-typed channel vocabulary (`{type:"section"}`,
  native) — no ambiguity. `react` is an OPTIONAL peer, lazily required only when
  a host tag is used, so channel-only/react-free bots are unaffected.
- JSX.IntrinsicElements now types host tags (loose CSS map so channels-ui stays
  type-level react-free).

channels-core:
- render/takumi.ts gains a `toReact` converter (image path only, where react is
  present): it materializes the tree — rebuilding React elements with converted
  children and INVOKING nested component nodes — so a component nested inside a
  host card (`<div><Meter/></div>`) renders. detect.ts routing is unchanged.

examples/slack: rewrite the image cards (mrr, pr-radar, weekly-pulse,
cycle-standup) from createElement to JSX. Docs note the authoring model.
…d cards

Adds ready-made data-viz tools and a CopilotKit-branded look.

channels-core/charts:
- chartTool (render_chart) + diagramTool (render_diagram): ready-made channel
  tools any bot registers. render_chart turns structured data (e.g. a parsed
  CSV) into a bar/line/pie/stacked/scatter image; render_diagram turns nodes +
  edges into a layered flow diagram. Both validate input and return a helpful
  string instead of throwing.
- FlowDiagram: layered longest-path layout (cycle-safe), HTML boxes + SVG arrows.
- DEFAULT_CHART_COLORS is now the CopilotKit brand data-viz palette.

examples/slack:
- Style cards with Tailwind (brand tokens in styles/tailwind.css → compiled to
  styles/brand.css via `pnpm build:css`), fed to createChannel with the Plus
  Jakarta Sans brand font (assets/fonts) via app/render/brand.ts. Takumi resolves
  the Tailwind classes at render time. Cards rewritten to the light OG brand look;
  the old shadcn theme is removed.
- Register chartTool + diagramTool so you can ask for an ad-hoc chart/diagram.

Docs (guide + README) cover the tools, branding, and build:css workflow.
…rner reveal)

- FlowDiagram root fills the post canvas (width/height 100% + white bg), and
  render_diagram sizes the canvas to the flow direction (portrait for down,
  landscape for right) — so the diagram spans the image instead of sitting in the
  top-left with a large blank/gray area.
- Drop the outer rounded corners on the showcase cards (mrr, pr-radar, pulse,
  standup): the rounded edge exposed the canvas behind the card. Cards are now
  full-bleed; inner badges/tiles keep their rounding. Recompiled styles/brand.css.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Morty Proxy This is a proxified and sanitized view of the page, visit original site.