Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Conversation

maxdeichmann
Copy link
Member

@maxdeichmann maxdeichmann commented Oct 13, 2025

Important

Rewrites observation CSV exports by introducing a streaming mechanism with model data enrichment, enhancing performance and maintainability.

  • Behavior:
    • Introduces getObservationStream in observation-stream.ts for streaming observation data with model enrichment.
    • Updates handleBatchExportJob and handleBatchActionJob to use getObservationStream for observations.
    • Increases totalObservations from 1000 to 10000 in seed-clickhouse.ts.
  • Refactoring:
    • Moves model enrichment logic to enrichObservationWithModelData in observations_converters.ts.
    • Refactors tests in batchExport.test.ts to use getObservationStream.
  • Misc:
    • Adds clickhouseSearchCondition export in queries/index.ts.
    • Minor logging improvements in observation-stream.ts.

This description was created by Ellipsis for 63d697b. You can customize this summary. It will automatically update as commits are pushed.

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Oct 13, 2025
@maxdeichmann maxdeichmann changed the title chore: reqrite observations csv exports chore: rewrite observations csv exports Oct 16, 2025
@langfuse langfuse deleted a comment from ellipsis-dev bot Oct 16, 2025
worker/src/features/batchAction/handleBatchActionJob.ts Outdated Show resolved Hide resolved
@maxdeichmann maxdeichmann added this pull request to the merge queue Oct 17, 2025
@dosubot dosubot bot added the auto-merge This PR is set to be merged label Oct 17, 2025
Merged via the queue into main with commit 24eb365 Oct 17, 2025
31 checks passed
@maxdeichmann maxdeichmann deleted the max/lfe-6371 branch October 17, 2025 09:36
@dosubot dosubot bot removed the auto-merge This PR is set to be merged label Oct 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Morty Proxy This is a proxified and sanitized view of the page, visit original site.