Custom Video Sources - Android

To deliver high-quality livestreams, it's essential to fine-tune the video tracks being broadcasted. Whether you’re hosting a webinar, or streaming an event, using custom video tracks gives you better control over stream quality and performance.

Custom Video Track

A custom video track lets you choose the capture resolution, the camera and multiStream for the video you send. Background removal and video filters from an external SDK (e.g., Banuba) are applied with a video processor, as described below.

How to Create a Custom Video Track ?

• Create the track with VideoSDK.createCameraVideoTrack(). Pass the resolution as encoderConfig and the camera as facingMode. It returns a CustomStreamTrack.
• Every argument is optional. Pass them by name. The optional cameraId, maxLayer, bitrateMode and codec arguments are described in the API reference.
• It is a suspend function, so call it from a coroutine. It suspends until the camera is open and runs the work on a background thread, so it is safe to call from the main thread.
• The camera opens when the track is created. Call stop() on the track to release the camera if you don't publish it.
• It throws VideoSDKException when the track can't be created, for example with 3017 (ERROR_CAMERA_ACCESS_DENIED_OR_DISMISSED) when the CAMERA permission is not granted, 3023 (ERROR_CAMERA_IN_USE) when another app holds the camera, or 3026 (ERROR_VIDEO_SOURCE_INITIATION_FAILED) for any other open failure. The method doesn't ask for the permission, so request it first.
• A camera that fails after the track is created ends the track. Set onEnded on the track to find out.

Example

lifecycleScope.launch {
  try {
    val videoCustomTrack = VideoSDK.createCameraVideoTrack(

      // This will accept the resolution (height x width) of video you want to capture.
      encoderConfig = CustomCameraTrackConfig.H720P_W960P, // H480P_W640P | H720P_W1280P ... // Default : H720P_W1280P

      // It will specify whether to use front or back camera for the video track.
      facingMode = FacingMode.front, // FacingMode.back | Default : FacingMode.front

      // multiStream - we will discuss this parameter in next step.
      multiStream = false, // true | Default : true
    )
  } catch (e: VideoSDKException) {
    Log.e("VideoSDK", "Camera track failed: ${e.code} ${e.name}")
  }
}

caution

The behavior of custom track configurations is influenced by the capabilities of the device. For example, if you set the encoder configuration to 1080p but the webcam only supports 720p, the encoder configuration will automatically adjust to the highest resolution that the device can handle, which in this case is 720p.

What is multiStream?

• By enabling multiStream, your livestream will broadcast multiple resolutions (e.g., 720p, 360p, 180p), allowing viewers to receive the best stream quality based on their network. The multiStream = true configuration indicates that VideoSDK, by default, sends multiple resolution video streams to the server. For example, for an H720P_W1280P track, VideoSDK sends streams in 1280x720, 640x360 and 320x180 resolution. This enables VideoSDK to deliver the appropriate stream to each participant based on their network bandwidth.

Setting multiStream = false restricts VideoSDK to send only one stream, helping to maintain quality by focusing on a single resolution.

The number of streams depends on the resolution of the track. Refer to the Simulcast (Adaptive Bitrate) guide for the streams each resolution sends.

danger

setQuality() will not have any effect if multiStream is set to false.

How to use video filters from an external SDK?

• If you want to use video filter from external SDK (e.g., Banuba), implement VideoFrameProcessor and pass it to room.setVideoProcessor(). It changes the camera frames before they are sent, and is removed when the room is left. Learn more in Video Processor.

How to Setup a Custom Video Track ?

You can plug in your custom video track either before going live or dynamically while the session is ongoing.

1. Setup during live stream initialization
2. Setup dynamically using methods

1. Setting up a Custom Track during the initialization of a liveStream

If you're starting the stream with the camera enabled (camEnabled = true) and wish to use a custom track from the beginning, pass it as customCameraTrack to createRoom() as shown below.

caution

The track is published when you join only if camEnabled is true. With camEnabled set to false, publish it later by passing it to enableCam(), or call stop() on it to release the camera.

Example

override fun onCreate(savedInstanceState: Bundle?) {
  //..

  lifecycleScope.launch {
    val videoCustomTrack = try {
      VideoSDK.createCameraVideoTrack(
        encoderConfig = CustomCameraTrackConfig.H720P_W960P,
        facingMode = FacingMode.front,
        multiStream = false,
      )
    } catch (e: VideoSDKException) {
      Log.e("VideoSDK", "Camera track failed: ${e.code} ${e.name}")
      null
    }

    // create a new room instance
    val liveStream = VideoSDK.createRoom(
      roomId = roomId,
      token = token,
      participantName = participantName,
      micEnabled = true,
      // If true, it will use the passed custom track to turn the camera on
      camEnabled = true,
      multiStream = false,
      // Pass the custom track here
      customCameraTrack = videoCustomTrack,
    )
  }
}

2. Setting up a Custom Track with methods

To switch tracks during the live stream, pass the CustomStreamTrack to enableCam() on the Room while the camera is off.

While the camera is on, pass the track to changeCam() instead. It turns the camera off and on again with the new track, so onStreamDisabled and then onStreamEnabled fire.

Both are suspend functions, so call them from a coroutine.

tip

enableCam() throws VideoSDKException with 3054 (ERROR_ALREADY_IN_REQUESTED_STATE) when the camera is already on. Use changeCam() to switch the track of a camera that is on.

Example

lifecycleScope.launch {
  try {
    val customStreamTrack = VideoSDK.createCameraVideoTrack(
      encoderConfig = CustomCameraTrackConfig.H720P_W960P,
      facingMode = FacingMode.back,
      multiStream = false,
    )

    if (liveStream!!.isCamEnabled) {
      liveStream!!.changeCam(customStreamTrack)
    } else {
      liveStream!!.enableCam(customStreamTrack)
    }
  } catch (e: VideoSDKException) {
    Log.e("VideoSDK", "Switching the camera track failed: ${e.code} ${e.name}")
  }
}

Which Configuration is suitable for Device ?

This chart shows the recommended encoder resolution and multiStream setting for each room size.

API Reference

The API references for all the methods and events utilised in this guide are provided below.

• Custom Video Track
• enableCam()
• changeCam()