Optimize Video Tracks - Android

While optimizing for the best viewing experience, it is necessary to fine-tune the video tracks that are being used during the calls.

For the best fine-tuning experience, we have introduced the ability to pass a custom video track for the participant's media before and during the meeting.

1. Custom Video Track
2. Custom Screen Share Track

Custom Video Track

This feature gives you powerful tools to deliver the best possible video stream for varying network conditions. You can specify a modern video quality profile (bitrateMode) and control simulcast behavior (maxLayer).

The SDK also supports legacy optimization modes (such as focusing on motion, text, or detail) and allows for the integration of background removal and video filters from external SDKs (e.g., Banuba) to further enhance the video.

From version 2.1.0 onward, once you set a custom video track created using createCameraVideoTrack(), VideoSDK remembers its settings and automatically reuses them whenever the webcam is turned back on. To change the track configuration, call disableWebcam(), then pass a new track created using createCameraVideoTrack() to enableWebcam(). VideoSDK preserves only tracks created using createCameraVideoTrack(). Filters applied through external SDKs such as Banuba are not preserved and must be re-applied each time the webcam is re-enabled.

How to Create a Custom Video Track ?

• You can create a Video Track using createCameraVideoTrack() method of VideoSDK.
• This method can be used to create video track by specifying parameters like video resolution, camera facing mode, and the quality controls (bitrateMode and maxLayer).
• You can choose video resolution from the below mentioned list of values for the encoder config:

Config	Resolution	Frame Rate	Optimized (kbps)	Balanced (kbps)	High Quality (kbps)
h144p_w176p	176x144	15 fps	60	100	150
h240p_w320p	320x240	15 fps	80	150	300
h360p_w640p	360x640	25 fps	200	400	800
h480p_w640p	640x480	25 fps	300	600	1000
h480p_w720p	720x480	30 fps	400	700	1100
h720p_w960p	960x720	30 fps	800	1300	1800
h720p_w1280p	1280x720	30 fps	1000	1600	2400
h1080p_w1440p	1440x1080	30 fps	2000	2500	3500

Example

Kotlin

val videoCustomTrack: CustomStreamTrack = VideoSDK.createCameraVideoTrack(

  // This will accept the resolution (height x width) of video you want to capture.
  "h720p_w960p", // "h480p_w640p" | "h720p_w1280p" ... // Default : "h720p_w1280p"

  // It will specify whether to use front or back camera for the video track.
  "front", // "back" | Default : "front"

  // We will discuss this parameter in next step.
  CustomStreamTrack.VideoMode.MOTION, // CustomStreamTrack.VideoMode.TEXT, CustomStreamTrack.VideoMode.DETAIL ,  Default : CustomStreamTrack.VideoMode.MOTION

  // multiStream - we will discuss this parameter in next step.
  true, // false

  // Pass Context
  this,

  // Optional: Specify a specific camera device.
  videoDeviceInfo, // Default: null

  // Optional: This specifies the maximum number of simulcast layers (maxLayer) to publish.
  2, // 3 |  Default: 3

  // Optional: This controls the video quality and bandwidth usage.
  BitrateMode.HIGH_QUALITY, // .BANDWIDTH_OPTIMIZED | .BALANCED Default: .BALANCED 

  // Optional: This specifies the video codec used to encode this track.
  VideoCodec.VP8 // VideoCodec.H264 | VideoCodec.VP9 | VideoCodec.AV1 , Default : VideoCodec.VP8
  )

Java

CustomStreamTrack customStreamTrack = VideoSDK.createCameraVideoTrack(

  // This will accept the resolution (height x width) of video you want to capture.
  "h720p_w960p", // "h480p_w640p" | "h720p_w1280p" ... // Default : "h720p_w1280p"

  // It will specify whether to use front or back camera for the video track.
  "front", // "back" | Default : "front"

  // We will discuss this parameter in next step.
  CustomStreamTrack.VideoMode.MOTION, // CustomStreamTrack.VideoMode.TEXT, CustomStreamTrack.VideoMode.DETAIL ,  Default : CustomStreamTrack.VideoMode.MOTION

  // multiStream - we will discuss this parameter in next step.
  true, // false

  // Pass Context
  this,

  // Optional: Specify a specific camera device.
  videoDeviceInfo, // Default: null

  // Optional: This specifies the maximum number of simulcast layers (maxLayer) to publish.
  2, // 3 |  Default: 3

  // Optional: This controls the video quality and bandwidth usage.
  BitrateMode.HIGH_QUALITY, // .BANDWIDTH_OPTIMIZED | .BALANCED Default: .BALANCED 

  // Optional: This specifies the video codec used to encode this track.
  VideoCodec.VP8 // VideoCodec.H264 | VideoCodec.VP9 | VideoCodec.AV1 , Default : VideoCodec.VP8
  );

caution

The capabilities of the device have a significant impact on how custom track configurations behave. Assuming a case where you set encoder configuration to 1080p but the camera only supports 720p, then encoder configuration will automatically switch to the highest resolution that the device can handle, which is 720p.

What is multiStream?

• It will specify if the stream should send multiple resolution layers or single resolution layer. It is true by default.

Refer to the Simulcast (Adaptive Bitrate) guide for the layers a track publishes and for how participants select the layer they receive.

What is BitrateMode?

BitrateMode is the key setting for video quality. It lets you decide whether to prioritize sharp details, save internet data, or find a good balance between the two.

You can choose from three distinct modes:

• BitrateMode.BANDWIDTH_OPTIMIZED: Prioritizes lower bandwidth usage over video quality. Ideal for users with poor or unstable network conditions.
• BitrateMode.BALANCED: The default setting. It provides a smart compromise between clear video and efficient bandwidth consumption, suitable for most use cases.
• BitrateMode.HIGH_QUALITY: Prioritizes video quality over bandwidth usage. Use this when high-fidelity video is essential and viewers are expected to have strong network connections.

What is MaxLayer?

• This parameter caps how many layers a track publishes, and accepts 2 or 3. It applies only when multiStream is true, and defaults to 3.

Refer to the Simulcast (Adaptive Bitrate) guide for what each value publishes.

What is optimizationMode?

• It will specify the optimization mode for the video track being generated.

• motion : This type of track should more focus on motion video. For example, webcam video, movies or video games.

  • This type of track will degrade resolution in order to maintain frame rate.

• text : This type of track should more focus on significant sharp edges and areas of consistent color that can change frequently. For example, presentations or web pages with text content.

  • This type of track will degrade frame rate in order to maintain resolution.

• detail : This type of track should more focus on details of the video. For example, presentations, painting or line art.

  • This type of track will degrade frame rate in order to maintain resolution.

What is codec?

• The videoCodec parameter specifies the video codec used to encode your stream. VideoSDK supports VP8 (default), H264, VP9, and AV1, set through VideoCodec.
• Currently, VideoSDK doesn't support multiStream for H264, VP9, and AV1. Set multiStream to false when you create an H264, VP9, or AV1 track. Otherwise VideoSDK sets it to false for you and emits the ERROR_MULTISTREAM_NOT_SUPPORTED error event.

Refer to the Video Codecs guide to compare the four codecs, to check which codec a meeting is actually using, and to handle the cases where a codec is not supported.

What is observer?

• If you want to use video filter from external SDK(e.g., Banuba) then pass instance of CapturerObserver in this parameter.

note

For Banuba integraion with VideoSDK, please visit Banuba Intergation with VideoSDK

How to Setup a Custom Video Track ?

The custom track can be set up both before and after the initialization of the meeting.

1. Setting up a Custom Track during the initialization of a meeting
2. Setting up a Custom Track with methods

1. Setting up a Custom Track during the initialization of a meeting

If you are passing webcamEnabled: true in the initMeeting of VideoSDK and want to use custom tracks from start of the meeting, you can pass custom track in the initMeeting as shown below.

caution

Custom Track will not apply on webcamEnabled: false configuration.

Example

Kotlin

override fun onCreate(savedInstanceState: Bundle?) {
  //..

  val customTracks: MutableMap<String, CustomStreamTrack> = HashMap()
  val videoCustomTrack: CustomStreamTrack = 
    VideoSDK.createCameraVideoTrack(
    "h720p_w960p",
     "front", 
     CustomStreamTrack.VideoMode.MOTION, 
      true, 
      this,
      null, // videoDeviceInfo
      2, 
      BitrateMode.HIGH_QUALITY,
      VideoCodec.VP8)

  customTracks["video"] = videoCustomTrack  //Key must be "video"

  // create a new meeting instance
  val meeting = VideoSDK.initMeeting(
    this@MainActivity, meetingId, participantName,
    //MicEnabled
    true,
    //WebcamEnabled , If true, it will use the passed custom track to turn webcam on
    true,
    // ParticipantId
    null,
    // Mode
    null,
    // MultiStream
    false,
    //Pass the custom tracks here
    customTracks,
    //MetaData
    null
  )
}

Java

@Override
protected void onCreate(Bundle savedInstanceState) {
  //..

  Map<String, CustomStreamTrack> customTracks = new HashMap<>();
  CustomStreamTrack videoCustomTrack = 
  VideoSDK.createCameraVideoTrack(
    "h720p_w960p", 
    "front",
    CustomStreamTrack.VideoMode.MOTION, 
    true, 
    this, 
    null, // videoDeviceInfo
    2, 
    BitrateMode.HIGH_QUALITY,
    VideoCodec.VP8);
  customTracks.put("video", videoCustomTrack);  //Key must be "video"

  // create a new meeting instance
  Meeting meeting = VideoSDK.initMeeting(
    MainActivity.this, meetingId, participantName,
    //MicEnabled
    true,
    //WebcamEnabled , If true, it will use the passed custom track to turn webcam on
    true,
    // ParticipantId
    null,
    // Mode
    null,
    // MultiStream
    false,
    //Pass the custom tracks here
    customTracks,
    //MetaData
    null
    );
}

2. Setting up a Custom Track with methods

In order to switch tracks during the meeting, you have to pass the CustomStreamTrack in the enableWebcam() method of Meeting.

tip

Make sure to call disableWebcam() before you create a new track as it may lead to unexpected behavior.

Example

Kotlin

 val videoCustomTrack: CustomStreamTrack = 
    VideoSDK.createCameraVideoTrack(
    "h720p_w960p",
     "front", 
     CustomStreamTrack.VideoMode.MOTION, 
      true, 
      this,
      null, // videoDeviceInfo
      2, 
      BitrateMode.HIGH_QUALITY,
      VideoCodec.VP8)

meeting!!.enableWebcam(customStreamTrack)

Java

CustomStreamTrack customStreamTrack=
  VideoSDK.createCameraVideoTrack(
    "h720p_w960p", 
    "front",
    CustomStreamTrack.VideoMode.MOTION, 
    true, 
    this, 
    null, // videoDeviceInfo
    2, 
    BitrateMode.HIGH_QUALITY,
    VideoCodec.VP8);
    
meeting.enableWebcam(customStreamTrack);

Which Configuration is suitable for Device ?

In this section, we will understand participant size wise encoder(Resolution) and multiStream configuration.

Custom Screen Share Track

This feature can be used to customise screenshare streams with enhanced optimization mode and predefined encoderConfig(Resolution + FPS) for specific use-case and send it to other participants.

How to Create Custom Screen Share Track ?

• You can create a Screen Share track using createScreenShareVideoTrack() method of VideoSDK.
• This method can be used to create video track using different encoding parameters and optimization mode.

Example

Kotlin

// data is received from onActivityResult method.
VideoSDK.createScreenShareVideoTrack(

  // This will accept the  height & FPS of video you want to capture.
  "h720p_15fps", //  `h360p_30fps` | `h1080p_30fps` // Default : `h720p_15fps`

  // It is Intent received from onActivityResult when user provide permission for ScreenShare.
  data,

  // Pass Conext
  this)

  //Callback to this listener will be made when track is ready with CustomTrack as parameter
  {
    track ->
    meeting!!.enableScreenShare(track)
  }

Java

// data is received from onActivityResult method.
VideoSDK.createScreenShareVideoTrack(
  // This will accept the  height & FPS of video you want to capture.
  "h720p_15fps", //  `h360p_30fps` | `h1080p_30fps` // Default : `h720p_15fps`

  /highlight-next-line
  // It is Intent received from onActivityResult when user provide permission for ScreenShare
  data,

  // Pass Conext
  this,

  //Callback to this listener will be made when track is ready with CustomTrack as parameter
  (track)->{meeting.enableScreenShare(track);}
);

How to Setup Custom Screen Share Track ?

In order to switch tracks during the meeting, you have to pass the CustomStreamTrack in the enableScreenShare() method of Meeting.

note

Make sure to call disableScreenShare() before you create a new track as it may lead to unexpected behavior.

Example

Kotlin

@TargetApi(21)
private fun askPermissionForScreenShare() {
  val mediaProjectionManager = application.getSystemService<Any>(
      Context.MEDIA_PROJECTION_SERVICE
  ) as MediaProjectionManager
  startActivityForResult(
      mediaProjectionManager.createScreenCaptureIntent(), CAPTURE_PERMISSION_REQUEST_CODE
  )
}

@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
  super.onActivityResult(requestCode, resultCode, data)
  if (requestCode != CAPTURE_PERMISSION_REQUEST_CODE) return
  if (resultCode == RESULT_OK) {
    VideoSDK.createScreenShareVideoTrack("h720p_15fps", data, this) { track ->
        meeting!!.enableScreenShare(track)
    }
  }
}

Java

@TargetApi(21)
private void askPermissionForScreenShare() {
  MediaProjectionManager mediaProjectionManager =
      (MediaProjectionManager) getApplication().getSystemService(
          Context.MEDIA_PROJECTION_SERVICE);
  startActivityForResult(
      mediaProjectionManager.createScreenCaptureIntent(), CAPTURE_PERMISSION_REQUEST_CODE);
}

@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
@Override
public void onActivityResult(int requestCode, int resultCode, Intent data) {
  super.onActivityResult(requestCode, resultCode, data);
  if (requestCode != CAPTURE_PERMISSION_REQUEST_CODE)
      return;
  if (resultCode == Activity.RESULT_OK) {
    VideoSDK.createScreenShareVideoTrack("h720p_15fps", data, this, (track)->{
        meeting.enableScreenShare(track);
    });
  }
}

API Reference

The API references for all the methods and events utilised in this guide are provided below.

• Custom Video Track
• Custom Screen Share Track