Optimize Video Tracks - Android

While optimizing for the best viewing experience, it is necessary to fine-tune the video tracks that are being used during the calls.

You can pass a custom video track for the participant's media before and after joining the room.

1. Custom Video Track
2. Custom Screen Share Track

Custom Video Track

You can set a video quality profile (bitrateMode) and control simulcast behavior (maxLayer) to suit different network conditions.

You can also apply background removal and video filters from external SDKs (e.g., Banuba).

Once you publish a custom video track created using createCameraVideoTrack(), VideoSDK remembers its settings and reuses them whenever the webcam is turned back on with enableCam(), for as long as you stay in the room. To change the track configuration, pass a new track created using createCameraVideoTrack() to changeCam(). Filters from external SDKs such as Banuba are applied with a video processor set with room.setVideoProcessor(). It stays applied until you call room.removeVideoProcessor() or leave the room.

How to Create a Custom Video Track ?

• Create a video track with the createCameraVideoTrack() method of VideoSDK. Its parameters set the video resolution, the camera, and the quality controls (bitrateMode and maxLayer).
• All arguments are optional. Pass them by name.
• The camera opens when the track is created. Call stop() on the track to release the camera if you don't publish it.
• It throws VideoSDKException when the track can't be created, for example with 3017 (ERROR_CAMERA_ACCESS_DENIED_OR_DISMISSED) when the CAMERA permission is not granted. The method doesn't ask for the permission, so request it first.
• It is a suspend function. It returns once the camera is open, and throws when the camera can't open: 3023 (ERROR_CAMERA_IN_USE) when another app holds the camera, 3026 (ERROR_VIDEO_SOURCE_INITIATION_FAILED) for any other failure. The work runs on a background thread, so you can call it from the main thread.
• A camera that fails after the track is created ends the track. Set onEnded on the track to find out.
• It throws IllegalStateException if VideoSDK.initialize() has not run.
• You can choose video resolution from the below mentioned list of CustomCameraTrackConfig values for the encoder config:

Config	Resolution	Frame Rate	Optimized (kbps)	Balanced (kbps)	High Quality (kbps)
H90P_W160P	160x90	15 fps	40	90	120
H120P_W160P	160x120	15 fps	40	90	120
H144P_W176P	176x144	15 fps	40	90	120
H180P_W240P	240x180	15 fps	90	120	180
H180P_W320P	320x180	15 fps	60	120	180
H240P_W320P	320x240	15 fps	100	150	200
H360P_W480P	480x360	25 fps	270	360	540
H360P_W640P	640x360	25 fps	300	450	600
H480P_W640P	640x480	25 fps	350	600	900
H480P_W720P	720x480	30 fps	350	600	900
H540P_W960P	960x540	30 fps	600	900	1200
H720P_W960P	960x720	30 fps	900	1200	1600
H720P_W1280P	1280x720	30 fps	1000	1500	2000
H1080P_W1440P	1440x1080	30 fps	1500	2000	2700
H1080P_W1920P	1920x1080	30 fps	1500	2200	2700

The bitrate columns are the maximum bitrate for each bitrateMode.

Example

lifecycleScope.launch {
  try {
    val videoCustomTrack = VideoSDK.createCameraVideoTrack(

      // This will accept the resolution (height x width) of video you want to capture.
      encoderConfig = CustomCameraTrackConfig.H720P_W960P, // CustomCameraTrackConfig.H480P_W640P | ... // Default : CustomCameraTrackConfig.H720P_W1280P

      // It will specify whether to use front or back camera for the video track.
      facingMode = FacingMode.front, // FacingMode.back | Default : FacingMode.front

      // multiStream - we will discuss this parameter in next step.
      multiStream = true, // false | Default : true

      // Optional: The deviceId of a camera from VideoSDK.getCameras().
      cameraId = cameraId, // Default: null, the camera of facingMode

      // Optional: This specifies the maximum number of simulcast layers (maxLayer) to publish.
      maxLayer = EncodingLayer.MAX_LAYER_2, // EncodingLayer.MAX_LAYER_3 | Default: EncodingLayer.MAX_LAYER_3

      // Optional: This controls the video quality and bandwidth usage.
      bitrateMode = BitrateMode.HIGH_QUALITY, // BitrateMode.BANDWIDTH_OPTIMIZED | BitrateMode.BALANCED | Default: BitrateMode.BALANCED

      // Optional: This specifies the video codec used to encode this track.
      codec = VideoCodec.VP8 // VideoCodec.H264 | VideoCodec.VP9 | VideoCodec.AV1 | Default : VideoCodec.VP8
    )
  } catch (e: VideoSDKException) {
    Log.e("VideoSDK", "Camera track failed: ${e.code} ${e.name}")
  }
}

caution

The device decides which custom track configurations it can use. For example, if you set the encoder configuration to 1080p but the camera only supports 720p, the encoder configuration switches to the highest resolution the device can handle, 720p.

What is multiStream?

• It sets whether the track sends multiple resolution layers or a single resolution layer. It is true by default.

Refer to the Simulcast (Adaptive Bitrate) guide for the layers a track publishes and for how participants select the layer they receive.

What is BitrateMode?

BitrateMode is the key setting for video quality. It lets you decide whether to prioritize sharp details, save internet data, or find a good balance between the two.

You can choose from three distinct modes:

• BitrateMode.BANDWIDTH_OPTIMIZED: Prioritizes lower bandwidth usage over video quality. Ideal for users with poor or unstable network conditions.
• BitrateMode.BALANCED: The default setting. It provides a smart compromise between clear video and efficient bandwidth consumption, suitable for most use cases.
• BitrateMode.HIGH_QUALITY: Prioritizes video quality over bandwidth usage. Use this when high-fidelity video is essential and viewers are expected to have strong network connections.

What is MaxLayer?

• This parameter caps how many layers a track publishes: EncodingLayer.MAX_LAYER_2 or EncodingLayer.MAX_LAYER_3. It applies only when multiStream is true, and null means EncodingLayer.MAX_LAYER_3. To publish a single layer, set multiStream to false.

Refer to the Simulcast (Adaptive Bitrate) guide for what each value publishes.

What is codec?

• The codec parameter specifies the video codec used to encode your stream. VideoSDK supports VP8 (default), H264, VP9, and AV1, set through VideoCodec.
• Currently, VideoSDK doesn't support multiStream for H264, VP9, and AV1. Set multiStream to false when you create an H264, VP9, or AV1 track. Otherwise VideoSDK sends a single layer and reports 3046 (ERROR_MULTISTREAM_NOT_SUPPORTED) on onError.

Refer to the Video Codecs guide to compare the four codecs and to handle the cases where a codec is not supported.

How to use video filters from an external SDK?

• If you want to use a video filter from an external SDK (e.g., Banuba), implement VideoFrameProcessor and pass it to room.setVideoProcessor(). It changes the camera frames before they are sent. Learn more in Video Processor.

note

For Banuba integration with VideoSDK, see Banuba Integration with VideoSDK

How to Setup a Custom Video Track ?

The custom track can be set up both before and after the initialization of the room.

1. Setting up a Custom Track during the initialization of a room
2. Setting up a Custom Track with methods

1. Setting up a Custom Track during the initialization of a room

If you are passing camEnabled = true in the createRoom of VideoSDK and want to use custom tracks from the start of the room, you can pass the custom track as customCameraTrack in the createRoom as shown below.

caution

The track is published when you join only if camEnabled is true. With camEnabled set to false, publish it later with enableCam(), or call stop() on it to release the camera.

Example

override fun onCreate(savedInstanceState: Bundle?) {
  //..

  lifecycleScope.launch {
    try {
      val videoCustomTrack = VideoSDK.createCameraVideoTrack(
        encoderConfig = CustomCameraTrackConfig.H720P_W960P,
        facingMode = FacingMode.front,
        multiStream = true,
        maxLayer = EncodingLayer.MAX_LAYER_2,
        bitrateMode = BitrateMode.HIGH_QUALITY,
        codec = VideoCodec.VP8
      )

      // create a new room instance
      room = VideoSDK.createRoom(
        roomId = roomId,
        token = token,
        participantName = participantName,
        micEnabled = true,
        // If true, it will use the passed custom track to turn the camera on
        camEnabled = true,
        //Pass the custom track here
        customCameraTrack = videoCustomTrack,
      )
    } catch (e: VideoSDKException) {
      Log.e("VideoSDK", "Room setup failed: ${e.code} ${e.name}")
    }
  }
}

2. Setting up a Custom Track with methods

To switch tracks during the call, pass the CustomStreamTrack to room.enableCam() while the camera is off.

You can also pass custom track in changeCam() method of Room, while the webcam is on. It turns the webcam off and on again with the new track, so onStreamDisabled and then onStreamEnabled fire.

tip

enableCam() throws VideoSDKException with 3054 (ERROR_ALREADY_IN_REQUESTED_STATE) when the webcam is already on. Use changeCam() to switch the track of a webcam that is on.

Example

lifecycleScope.launch {
  try {
    val videoCustomTrack = VideoSDK.createCameraVideoTrack(
      encoderConfig = CustomCameraTrackConfig.H720P_W960P,
      facingMode = FacingMode.front,
      multiStream = true,
      maxLayer = EncodingLayer.MAX_LAYER_2,
      bitrateMode = BitrateMode.HIGH_QUALITY,
      codec = VideoCodec.VP8
    )

    if (room!!.isCamEnabled) {
      room!!.changeCam(videoCustomTrack)
    } else {
      room!!.enableCam(videoCustomTrack)
    }
  } catch (e: VideoSDKException) {
    Log.e("VideoSDK", "Switching the webcam track failed: ${e.code} ${e.name}")
  }
}

Which Configuration is suitable for Device ?

This section shows the recommended encoderConfig (resolution) and multiStream setting for each participant count.

Custom Screen Share Track

Use this to customise a screen share stream with a predefined encoderConfig (resolution + FPS) for your use case and send it to other participants.

How to Create Custom Screen Share Track ?

• Create a screen share track with the createScreenShareVideoTrack() method of VideoSDK.
• You can choose the encoderConfig from h360p_30fps, h480p_15fps, h480p_30fps, h720p_5fps, h720p_15fps, h720p_30fps, h1080p_15fps and h1080p_30fps.
• createScreenShareVideoTrack() returns the track. It returns null when the user declined, or when encoderConfig is not in the list above. The second case also reports 3071 (ERROR_INVALID_CUSTOM_TRACK_CONFIG) on onError.
• The track holds the user's consent and your settings. Screen capture starts when you pass the track to enableScreenShare().

Example

// data is received from onActivityResult method.
val track = VideoSDK.createScreenShareVideoTrack(

  // This will accept the  height & FPS of video you want to capture.
  encoderConfig = "h720p_15fps", //  `h360p_30fps` | `h1080p_30fps` // Default : `h720p_15fps`

  // It is Intent received from onActivityResult when user provide permission for ScreenShare.
  data = data,

  // Optional: If true, the screen is sent as up to three simulcast layers.
  multiStream = false, // true | Default : false

  // Optional: If true, the audio your device plays is shared too. It needs Android 10 (API 29) or later.
  withAudio = false // true | Default : false
)

// track is null when the user declined
if (track != null) {
  lifecycleScope.launch {
    try {
      room!!.enableScreenShare(track)
    } catch (e: VideoSDKException) {
      Log.e("VideoSDK", "Screen share failed: ${e.code} ${e.name}")
    }
  }
}

note

With withAudio = true, the shared audio is mixed into your mic track, so it is sent only while your mic is on, and muting the mic mutes both. No separate SHARE_AUDIO stream is sent, so other participants hear it in your microphone stream and getShareAudioStats() returns null for you.

How to Setup Custom Screen Share Track ?

To share the screen with a custom track, pass the CustomStreamTrack to room.enableScreenShare().

note

enableScreenShare() throws VideoSDKException with 3054 (ERROR_ALREADY_IN_REQUESTED_STATE) while a screen share is running. Call disableScreenShare() before you share with a new track.

Example

private fun askPermissionForScreenShare() {
  val mediaProjectionManager = application.getSystemService(
      Context.MEDIA_PROJECTION_SERVICE
  ) as MediaProjectionManager
  startActivityForResult(
      mediaProjectionManager.createScreenCaptureIntent(), CAPTURE_PERMISSION_REQUEST_CODE
  )
}

override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
  super.onActivityResult(requestCode, resultCode, data)
  if (requestCode != CAPTURE_PERMISSION_REQUEST_CODE) return
  if (resultCode == RESULT_OK) {
    val track = VideoSDK.createScreenShareVideoTrack(encoderConfig = "h720p_15fps", data = data)
    if (track != null) {
      lifecycleScope.launch {
        try {
          room!!.enableScreenShare(track)
        } catch (e: VideoSDKException) {
          Log.e("VideoSDK", "Screen share failed: ${e.code} ${e.name}")
        }
      }
    }
  }
}

API Reference

The API references for all the methods and events utilised in this guide are provided below.

• Custom Video Track
• Custom Screen Share Track
• enableCam()
• changeCam()
• setVideoProcessor()
• enableScreenShare()