Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Shaka playback: micro-stalls on 1080p+ ABR HEVC due to batched transmuxer output #126

Copy link
Copy link

Description

@privaloops
Issue body actions

Symptom

When playing an ABR HEVC DASH stream at 1080p through @hevcjs/shaka-plugin,
playback exhibits regular ~3-4s pauses on hardware that decodes HEVC in
WASM noticeably below real-time. The pauses freeze both audio and video
in sync (no Shaka buffering spinner) and recur for the duration of the
stream. The same content plays smoothly via @hevcjs/dashjs-plugin on
the exact same machine.

Repro: https://www.hevcjs.dev/demo/shaka.html → load the
ABR 480p/720p/1080p + audio (30s) preset on a Windows/Edge box with
modern NVIDIA GPU (NVENC available for H.264). Cuts begin within a few
seconds of playback start.

Diagnosis

Per-segment instrumentation on the affected device:

Step Time Note
Demux fMP4 ~5 ms n/a
HEVC decode (WASM, single-thread) ~3120 ms 48 frames @ 1080p, ~15 fps
H.264 encode (WebCodecs / NVENC) ~470 ms ~100 fps, HW-accelerated
Total ~3600 ms for 2000 ms of content speedX ≈ 0.55

video.buffered shows a single contiguous range that grows in 2-second
bumps each time a segment is produced — no gap, no spec violation. The
playback head simply skirts the buffered-range edge because transcoding
is borderline slower than real time on this hardware, and Shaka's
Transmuxer.transmux() API (Shaka 4.x) returns a single Uint8Array
per segment, so the buffer can only grow in jumps.

dash.js doesn't show this because the MSE-intercept path uses
SegmentTranscoder.processMediaSegmentStreaming which appends H.264
chunks to MSE progressively as the encoder emits them, keeping the
buffered range growing continuously rather than in bumps.

Proposed mitigations

In priority order:

  1. Recommend / surface a higher Shaka buffer config
    (streaming.bufferingGoal: 30, streaming.rebufferingGoal: 5). Add
    a "Performance & tuning" section in the Shaka plugin docs explaining
    why and how. Lowest cost, highest user-visible win on slow hardware.
    Optionally export a recommendedBufferConfig() helper from the
    plugin so users get the right defaults in one call.

  2. Pipeline decode + encode inside SegmentTranscoder.processMediaSegment.
    Replace the "feed all → drain all → encode all → flush" sequence
    with per-sample feed() → drain() and immediate _encoder.encode()
    on each newly-displayable frame. Safe in memory ownership because
    H264Encoder.encode() already copies YUV planes into a fresh I420
    buffer before the source HEVC frame's WASM backing can be
    invalidated by the next feed(). Expected wall-clock saving:
    ~15-25% per segment (only the encode time overlaps decode, and the
    first frames are still stuck behind the DPB bumping). Not enough on
    its own to clear the 0.55x → 1.0x gap, but bumps headroom for free.

  3. WASM decoder throughput. The biggest factor. Out of scope for a
    quick win; track separately. Bench data point: 1080p @ 61 fps on
    beefier reference hardware, ~15 fps on the affected Windows box.
    Real BBB 1080p content appears to be ~4x slower than the synthetic
    benchmark.

Acceptance criteria

  • Documented streaming.bufferingGoal / rebufferingGoal recipe in
    the Shaka plugin docs.
  • Demo (demo/shaka.html) applies it so the live page no longer shows
    cuts on hardware where transcoding is real-time-borderline.
  • Optional: pipelined processMediaSegment with a unit test asserting
    byte-identical output vs the current sequential implementation on a
    fixture HEVC segment, plus an e2e test confirming the change doesn't
    regress the dash.js path.

Out of scope

  • WASM decoder optimization (separate issue).
  • Streaming output from the Shaka path (Shaka 4.x Transmuxer.transmux()
    contract is one-Uint8Array-per-segment; possibly revisit when
    upgrading the plugin to target Shaka 5+).

Context

Diagnostic surface added in #124 was removed before merge. The
correctness fix in that PR (SegmentTranscoder.prepareInit reset +
HevcTransmuxer init-bytes cache) is unrelated and orthogonal to this
throughput concern — it fixed image corruption at ~14s but the
micro-stall pattern was always present, just masked by the earlier
crash.

Reactions are currently unavailable

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    Morty Proxy This is a proxified and sanitized view of the page, visit original site.