VideoSink

Artifact: androidx.media3:media3-exoplayer

Kotlin |Java

@UnstableApi
interface VideoSink

A sink that consumes decoded video frames and images.

Summary

Nested types
`@Documented @Retention(value = RetentionPolicy.SOURCE) @Target(value = ElementType.TYPE_USE) @UnstableApi @IntDef(value = ) annotation VideoSink.FirstFrameReleaseInstruction` The instruction provided when the stream changes for releasing the first frame.
`@Documented @Retention(value = RetentionPolicy.SOURCE) @Target(value = ElementType.TYPE_USE) @IntDef(value = ) annotation VideoSink.InputType` Specifies how the input frames are made available to the video sink.
`interface VideoSink.Listener` Listener for `VideoSink` events.
`interface VideoSink.VideoFrameHandler` Handler for a video frame.
`class VideoSink.VideoSinkException : Exception` Thrown by `VideoSink` implementations.

Constants
`const Int`	`INPUT_TYPE_BITMAP = 2` Input frames come from a `Bitmap`.
`const Int`	`INPUT_TYPE_SURFACE = 1` Input frames come from a `surface`.
`const Int`	`RELEASE_FIRST_FRAME_IMMEDIATELY = 0` Instructs to release the first frame as soon as possible.
`const Int`	`RELEASE_FIRST_FRAME_WHEN_PREVIOUS_STREAM_PROCESSED = 2` Instructs to release the first frame when the playback position reaches the stream start position.
`const Int`	`RELEASE_FIRST_FRAME_WHEN_STARTED = 1` Instructs to release the first frame when rendering starts.

Public functions
`Unit`	`allowReleaseFirstFrameBeforeStarted()` Allows the sink to release the first frame even if rendering is not `started`.
`Unit`	`clearOutputSurfaceInfo()` Clears the set output surface info.
`Unit`	`flush(resetPosition: Boolean)` Flushes the video sink.
`Surface!`	`getInputSurface()` Returns the input `Surface` where the video sink consumes input frames from.
`Boolean`	`handleInputBitmap( inputBitmap: Bitmap!, bufferTimestampIterator: TimestampIterator! )` Handles an input `Bitmap`.
`Boolean`	`handleInputFrame( bufferPresentationTimeUs: Long, videoFrameHandler: VideoSink.VideoFrameHandler! )` Handles a video input frame.
`Boolean`	`initialize(sourceFormat: Format!)` Initializes the video sink.
`Boolean`	`isEnded()` Returns whether all the data has been rendered to the output surface.
`Boolean`	`isInitialized()` Returns whether the video sink is `initialized`.
`Boolean`	`isReady(otherwiseReady: Boolean)` Returns whether the video sink is able to immediately render media to its output surface from the current position.
`Unit`	`join(renderNextFrameImmediately: Boolean)` Joins the video sink to a new stream.
`Unit`	`onInputStreamChanged( @VideoSink.InputType inputType: Int, format: Format!, startPositionUs: Long, @VideoSink.FirstFrameReleaseInstruction firstFrameReleaseInstruction: Int, videoEffects: (Mutable)List<Effect!>! )` Informs the video sink that a new input stream will be queued with the given effects.
`Unit`	`redraw()` Redraws the `video effects` immediately.
`Unit`	`release()` Releases the sink.
`Unit`	`render(positionUs: Long, elapsedRealtimeUs: Long)` Incrementally renders processed video frames to the output surface.
`Unit`	`setBufferTimestampAdjustmentUs(bufferTimestampAdjustmentUs: Long)` Sets the buffer timestamp adjustment.
`Unit`	`setChangeFrameRateStrategy( @C.VideoChangeFrameRateStrategy changeFrameRateStrategy: Int )` Changes the `C.VideoChangeFrameRateStrategy` used when calling setFrameRate.
`Unit`	`setListener(listener: VideoSink.Listener!, executor: Executor!)` Sets a `Listener` on this sink.
`Unit`	`setOutputSurfaceInfo(outputSurface: Surface!, outputResolution: Size!)` Sets the output surface info.
`Unit`	`setPlaybackSpeed(speed: @FloatRange(from = 0, fromInclusive = false) Float)` Sets the playback speed.
`Unit`	`setVideoEffects(videoEffects: (Mutable)List<Effect!>!)` Sets `video effects` to apply immediately.
`Unit`	`setVideoFrameMetadataListener( videoFrameMetadataListener: VideoFrameMetadataListener! )` Sets the `VideoFrameMetadataListener`.
`Unit`	`signalEndOfCurrentInputStream()` Signals the end of the current input stream.
`Unit`	`signalEndOfInput()` Signals the end of the last input stream.
`Unit`	`startRendering()` Starts rendering to the output surface.
`Unit`	`stopRendering()` Stops rendering to the output surface.

Constants

INPUT_TYPE_BITMAP

const val INPUT_TYPE_BITMAP = 2: Int

Input frames come from a Bitmap.

INPUT_TYPE_SURFACE

const val INPUT_TYPE_SURFACE = 1: Int

Input frames come from a surface.

RELEASE_FIRST_FRAME_IMMEDIATELY

const val RELEASE_FIRST_FRAME_IMMEDIATELY = 0: Int

Instructs to release the first frame as soon as possible.

RELEASE_FIRST_FRAME_WHEN_PREVIOUS_STREAM_PROCESSED

const val RELEASE_FIRST_FRAME_WHEN_PREVIOUS_STREAM_PROCESSED = 2: Int

Instructs to release the first frame when the playback position reaches the stream start position.

RELEASE_FIRST_FRAME_WHEN_STARTED

const val RELEASE_FIRST_FRAME_WHEN_STARTED = 1: Int

Instructs to release the first frame when rendering starts.

Public functions

allowReleaseFirstFrameBeforeStarted

fun allowReleaseFirstFrameBeforeStarted(): Unit

Allows the sink to release the first frame even if rendering is not started.

This is used to update the FirstFrameReleaseInstruction of the stream that is currently being processed.

clearOutputSurfaceInfo

fun clearOutputSurfaceInfo(): Unit

Clears the set output surface info.

flush

fun flush(resetPosition: Boolean): Unit

Flushes the video sink.

After calling this method, any frames stored inside the video sink are discarded.

Parameters
`resetPosition: Boolean`	Whether to reset the current position.

getInputSurface

fun getInputSurface(): Surface!

Returns the input Surface where the video sink consumes input frames from.

Must be called after the sink is initialized.

handleInputBitmap

fun handleInputBitmap(
    inputBitmap: Bitmap!,
    bufferTimestampIterator: TimestampIterator!
): Boolean

Handles an input Bitmap.

Must be called after the corresponding stream is signaled.

Parameters
`inputBitmap: Bitmap!`	The `Bitmap` to queue to the video sink.
`bufferTimestampIterator: TimestampIterator!`	The buffer presentation times within the current stream that the bitmap should be shown at. The timestamps should be monotonically increasing.

Returns
`Boolean`	Whether the bitmap was queued successfully. If `false`, the caller can try again later.

handleInputFrame

fun handleInputFrame(
    bufferPresentationTimeUs: Long,
    videoFrameHandler: VideoSink.VideoFrameHandler!
): Boolean

Handles a video input frame.

Must be called after the corresponding stream is signaled.

Parameters
`bufferPresentationTimeUs: Long`	The buffer presentation time, in microseconds.
`videoFrameHandler: VideoSink.VideoFrameHandler!`	The `VideoFrameHandler` used to handle the input frame.

Returns
`Boolean`	Whether the frame was handled successfully. If `false`, the caller can try again later.

initialize

fun initialize(sourceFormat: Format!): Boolean

Initializes the video sink.

Parameters
`sourceFormat: Format!`	The format of the first input video or image.

Returns
`Boolean`	Whether initialization succeeded. If `false`, the caller should try again later.

Throws
`androidx.media3.exoplayer.video.VideoSink.VideoSinkException`	If initializing the sink failed.

isEnded

fun isEnded(): Boolean

Returns whether all the data has been rendered to the output surface.

This method returns true if the end of the last input stream has been signaled and all the input frames have been rendered. Note that a new input stream can be signaled even when this method returns true (in which case the sink will not be ended anymore).

isInitialized

fun isInitialized(): Boolean

Returns whether the video sink is initialized.

isReady

fun isReady(otherwiseReady: Boolean): Boolean

Returns whether the video sink is able to immediately render media to its output surface from the current position.

The caller should be ready if and only if the video sink is ready.

Parameters
`otherwiseReady: Boolean`	Whether the caller is ready except for the video sink.

join

fun join(renderNextFrameImmediately: Boolean): Unit

Joins the video sink to a new stream.

The sink will mask isReady as true for a short time to avoid interrupting an ongoing playback, even if the first frame hasn't yet been rendered to the output surface.

Parameters
`renderNextFrameImmediately: Boolean`	Whether the next frame should be rendered as soon as possible or only at its preferred scheduled release time.

onInputStreamChanged

fun onInputStreamChanged(
    @VideoSink.InputType inputType: Int,
    format: Format!,
    startPositionUs: Long,
    @VideoSink.FirstFrameReleaseInstruction firstFrameReleaseInstruction: Int,
    videoEffects: (Mutable)List<Effect!>!
): Unit

Informs the video sink that a new input stream will be queued with the given effects.

Must be called after the sink is initialized.

Parameters
`@VideoSink.InputType inputType: Int`	The `InputType` of the stream.
`format: Format!`	The `Format` of the stream.
`startPositionUs: Long`	The start position of the buffer presentation timestamps of the stream, in microseconds.
`@VideoSink.FirstFrameReleaseInstruction firstFrameReleaseInstruction: Int`	The `FirstFrameReleaseInstruction` indicating when to release the stream's first frame.
`videoEffects: (Mutable)List<Effect!>!`	The `List` to apply to the new stream.

redraw

fun redraw(): Unit

Redraws the video effects immediately.

release

fun release(): Unit

Releases the sink.

render

fun render(positionUs: Long, elapsedRealtimeUs: Long): Unit

Incrementally renders processed video frames to the output surface.

Parameters
`positionUs: Long`	The current playback position, in microseconds.
`elapsedRealtimeUs: Long`	`elapsedRealtime` in microseconds, taken approximately at the time the playback position was `positionUs`.

Throws
`androidx.media3.exoplayer.video.VideoSink.VideoSinkException`	If an error occurs during rendering.

setBufferTimestampAdjustmentUs

fun setBufferTimestampAdjustmentUs(bufferTimestampAdjustmentUs: Long): Unit

Sets the buffer timestamp adjustment.

Parameters
`bufferTimestampAdjustmentUs: Long`	The timestamp adjustment to add to the buffer presentation timestamps to convert them to frame presentation timestamps, in microseconds.

setChangeFrameRateStrategy

fun setChangeFrameRateStrategy(
    @C.VideoChangeFrameRateStrategy changeFrameRateStrategy: Int
): Unit

Changes the C.VideoChangeFrameRateStrategy used when calling setFrameRate.

The default value is VIDEO_CHANGE_FRAME_RATE_STRATEGY_ONLY_IF_SEAMLESS.

setListener

fun setListener(listener: VideoSink.Listener!, executor: Executor!): Unit

Sets a Listener on this sink. Callbacks are triggered on the supplied Executor.

Parameters
`listener: VideoSink.Listener!`	The `Listener`.
`executor: Executor!`	The `Executor` to dispatch the callbacks.

setOutputSurfaceInfo

fun setOutputSurfaceInfo(outputSurface: Surface!, outputResolution: Size!): Unit

Sets the output surface info.

setPlaybackSpeed

fun setPlaybackSpeed(speed: @FloatRange(from = 0, fromInclusive = false) Float): Unit

Sets the playback speed.

setVideoEffects

fun setVideoEffects(videoEffects: (Mutable)List<Effect!>!): Unit

Sets video effects to apply immediately.

setVideoFrameMetadataListener

fun setVideoFrameMetadataListener(
    videoFrameMetadataListener: VideoFrameMetadataListener!
): Unit

Sets the VideoFrameMetadataListener.

signalEndOfCurrentInputStream

fun signalEndOfCurrentInputStream(): Unit

Signals the end of the current input stream.

signalEndOfInput

fun signalEndOfInput(): Unit

Signals the end of the last input stream.

startRendering

fun startRendering(): Unit

Starts rendering to the output surface.

stopRendering

fun stopRendering(): Unit

Stops rendering to the output surface.

VideoSink

Summary

Nested types

Constants

Public functions

Constants

INPUT_TYPE_BITMAP

INPUT_TYPE_SURFACE

RELEASE_FIRST_FRAME_IMMEDIATELY

RELEASE_FIRST_FRAME_WHEN_PREVIOUS_STREAM_PROCESSED

RELEASE_FIRST_FRAME_WHEN_STARTED

Public functions

allowReleaseFirstFrameBeforeStarted

clearOutputSurfaceInfo

flush

getInputSurface

handleInputBitmap

handleInputFrame

initialize

isEnded

isInitialized

isReady

join

onInputStreamChanged

redraw

release

render

setBufferTimestampAdjustmentUs

setChangeFrameRateStrategy

setListener

setOutputSurfaceInfo

setPlaybackSpeed

setVideoEffects

setVideoFrameMetadataListener

signalEndOfCurrentInputStream

signalEndOfInput

startRendering

stopRendering