Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

ui: rendering performance follow-up#26097

Open
allozaur wants to merge 5 commits into
ggml-org:masterggml-org/llama.cpp:masterfrom
allozaur:allozaur/ui-rendering-performance-refactorallozaur/llama.cpp:allozaur/ui-rendering-performance-refactorCopy head branch name to clipboard
Open

ui: rendering performance follow-up#26097
allozaur wants to merge 5 commits into
ggml-org:masterggml-org/llama.cpp:masterfrom
allozaur:allozaur/ui-rendering-performance-refactorallozaur/llama.cpp:allozaur/ui-rendering-performance-refactorCopy head branch name to clipboard

Conversation

@allozaur

Copy link
Copy Markdown
Contributor

Overview

Follow-up PR with architectural improvements after #26053

Additional information

Requirements

Copilot AI review requested due to automatic review settings July 24, 2026 20:25
@allozaur
allozaur requested a review from a team as a code owner July 24, 2026 20:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Follow-up UI performance work focused on reducing per-render CPU cost during streaming by memoizing expensive parsing/formatting paths and improving tool-result lookup behavior in the chat message renderer.

Changes:

  • Add bounded memoization caches for frequently re-run helpers (tool-call JSON parsing, partial JSON scanning, syntax highlighting, LaTeX preprocessing, search-results parsing, tool-result classification/line splitting).
  • Replace per-tool-call linear search of tool result messages with an indexed lookup to avoid O(n^2) behavior across many tool calls.
  • Add unit tests intended to prevent regressions in tool-call parsing and lookup performance.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tools/ui/tests/unit/parse-toolcalls-memo.test.ts Adds unit tests for tool-call parsing memoization and tool-message lookup behavior.
tools/ui/src/lib/utils/search-results.ts Memoizes parsing of tool-result search output and extraction of query from tool args.
tools/ui/src/lib/utils/parse-partial-json-args.ts Adds a bounded cache around partial JSON arg parsing/scanning.
tools/ui/src/lib/utils/latex-protection.ts Refactors LaTeX processing to reuse constants and memoizes preprocessing results.
tools/ui/src/lib/utils/code.ts Adds a bounded cache for highlight.js output keyed by language/autoDetect/code.
tools/ui/src/lib/utils/agentic.ts Uses Map/Set for faster tool message lookup; memoizes tool result parsing/classification and toolCalls JSON parsing.
tools/ui/src/lib/constants/latex-protection.ts Introduces reusable LaTeX-related constants/regexes used by the preprocessing pipeline.
tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockSearchResults.svelte Minor cleanup of derived variable typing.
tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockEditFile.svelte Removes an extra blank line.
tools/ui/src/lib/components/app/chat/ChatMessages/ChatMessage/ChatMessageToolCall/ChatMessageToolCallBlockDefault.svelte Removes unused type import and reorders derived declarations.
Comments suppressed due to low confidence (2)

tools/ui/tests/unit/parse-toolcalls-memo.test.ts:31

  • The "returns the same array reference" test doesn't actually assert memoization (or reference identity) and would pass even if JSON.parse runs every time. Consider asserting that JSON.parse is only invoked once for the same toolCalls JSON string across repeated deriveAgenticSections calls.
	it('returns the same array reference for the same JSON string', () => {
		// parseToolCalls is not exported, but deriveAgenticSections uses it
		// internally. We verify memoization through behavior: calling
		// deriveAgenticSections twice with the same toolCalls should not
		// re-parse (which we verify by checking the returned sections

tools/ui/tests/unit/parse-toolcalls-memo.test.ts:124

  • This test uses a wall-clock timing assertion (elapsed < 100ms), which is prone to flakes across CI environments and doesn't directly assert the intended regression (avoiding per-tool-call Array#find). A deterministic check is to override toolMessages.find to throw and ensure deriveAgenticSections still succeeds.
		// If the lookup were still O(n^2), this would be noticeably slow
		const start = Date.now();
		const sections = deriveAgenticSections(msg, toolMessages, [], false);
		const elapsed = Date.now() - start;


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tools/ui/src/lib/utils/latex-protection.ts Outdated
Comment thread tools/ui/src/lib/utils/agentic.ts Outdated
Comment thread tools/ui/tests/unit/parse-toolcalls-memo.test.ts Outdated
@allozaur allozaur changed the title ui: rendering performance follow-uo ui: rendering performance follow-up Jul 24, 2026
@allozaur
allozaur requested a review from ServeurpersoCom July 24, 2026 20:50
@zachwinter

Copy link
Copy Markdown
Contributor

lovely! probably makes sense to push this through and rebase #26098 onto it once this lands? they're pretty complimentary. either way works for me – just holler.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.