Describe the feature

livekit-client can publish frame-metadata trailers with user_timestamp and frame_id, and (since #1983, shipped in livekit-client@2.20.1) it can now parse a user_data trailer (tag 0x03) on the receive side. However, there is still no way to write user_data on the publish side from the JS SDK.

• FrameMetadataPublishOptions only exposes timestamp and frameId.
• The encode transform (appendPacketTrailerToEncodedFrame in src/frameMetadata/frameMetadata.ts) only ever writes the timestamp + frameId TLVs; there is no plumbing to attach an arbitrary per-frame user_data payload.
• By contrast, the Rust SDK supports this today via VideoFrame.frame_metadata.user_data passed to capture_frame (Add support for frame level user timestamp rust-sdks#890), and the blog post (https://livekit.com/blog/frame-metadata) advertises user_data as a first-class capability across SDKs.

So JS clients can read user_data published by Rust/other SDKs, but a JS publisher cannot emit it.

Use case

Frame-accurate correlation of application events with the published video: attach a small user_data payload (e.g. an event id / tag) to the frame that was live when the event fired, so a downstream recording/analysis pipeline can line up video with application state with no separate sync channel — exactly the "frame-accurate correlation" use case from the frame-metadata blog post.

Today this requires reimplementing the SDK's encoded-frame transform in a custom worker just to add the user_data TLV (reusing the exact LKTS XOR-TLV wire format), which duplicates SDK internals and risks drifting from the wire format across releases.

Proposal

Expose a publish-side user_data mechanism, e.g.:

• a userData flag in FrameMetadataPublishOptions (to advertise the feature on the track), and
• an API to set the user_data for upcoming frame(s), e.g. LocalVideoTrack.setFrameUserData(bytes: Uint8Array) (attach to the next encoded frame, or valid until replaced), mirroring the Rust capture_frame model.

Questions

• Was the publish-side user_data writer intentionally deferred, or is it planned as a follow-up to Add support for user data frame metadata trailer type #1983?
• Is there an issue/PR already tracking this that I missed?

SDK

livekit-client@2.20.1