Optimize Video Track - iOS

While optimizing for the best viewing experience, it is necessary to fine-tune the video tracks that are being used during the calls.

For the best fine-tuning experience, we have introduced the ability to pass a custom video track for the participant's media before and during the meeting.

Custom Video Track

This feature gives you powerful tools to deliver the best possible video stream for varying network conditions. You can specify a modern video quality profile (bitrateMode) and control simulcast behavior (maxLayer).

How to Create a Custom Video Track ?

• You can create a Video Track using createCameraVideoTrack() method of VideoSDK class.

• This method can be used to create video track by specifying parameters like video resolution, camera facing mode, and the quality controls (bitrateMode and maxLayer).

• You can choose video resolution from the below mentioned list of values for the encoder config:

Config	Resolution	Frame Rate	Optimized (kbps)	Balanced (kbps)	High Quality (kbps)
h90p_w160p	160x90	30 fps	40	90	120
h180p_w320p	320x180	30 fps	60	120	180
h360p_w640p	640x360	30 fps	300	450	600
h540p_w960p	960x540	30 fps	600	900	1200
h720p_w1280p	1280x720	30 fps	1000	1500	2000
h1080p_w1920p	1920x1080	30 fps	1500	2200	2700
h120p_w160p	160x120	30 fps	40	90	120
h180p_w240p	240x180	30 fps	90	120	180
h240p_w320p	320x240	30 fps	100	150	200
h360p_w480p	480x360	30 fps	270	360	540
h480p_w640p	640x480	30 fps	350	600	900
h720p_w960p	960x720	30 fps	900	1200	1600
h1080p_w1440p	1440x1080	30 fps	1500	2000	2700

note

Above mentioned encoder configurations are valid for both, landscape as well as portrait mode.

Example

Swift

import WebRTC
guard let videoMediaTrack = try? VideoSDK.createCameraVideoTrack(
  // This will accept the enum value of CustomVideoTrackConfig which contains resolution (height x width) of video you want to capture.
  encoderConfig: .h720p_w960p, // .h360p_w640p | .h480p_w640p ... // Default : .h720p_w1280p

  // It will specify whether to use front or back camera for the video track.
  facingMode: .front, // .back,  Default : .front

  // We will discuss this parameter in next step.
  multiStream: true, // false,  Default : true

  // Optional: This specifies the codec to use for the meeting.
  codec: .H264, // .VP8 | .VP9 | .AV1 , Default : .VP8

  // Optional: This controls the video quality and bandwidth usage.
  bitrateMode: BitrateMode.HIGH_QUALITY, // .BANDWIDTH_OPTIMIZED | .BALANCED Default: .BALANCED 

  // Optional: This specifies the maximum number of simulcast layers (maxLayer) to publish.
  maxLayer: EncodingLayer.MAX_LAYER_2 // 2 |  Default: .MAX_LAYER_3

) else { return}

caution

The capabilities of the device have a significant impact on how custom track configurations behave. Assuming a case where you set encoder configuration to 1080p but the webcam only supports 720p, then encoder configuration will automatically switch to the highest resolution that the device can handle, which is 720p.

What is multiStream?

• It will specify if the stream should send multiple resolution layers or single resolution layer.

multiStream : true By default, VideoSDK sends multiple resolution video streams to the server (whether you are using custom video track or not), For instance, user device capabilty is 720p, so VideoSDK sends 720p along with 360p and 180p streams. This allows VideoSDK to deliver the appropriate stream to each participant based on their network bandwidth.

multiStream : false If you want to restrict the VideoSDK to send only one stream to maintain quality, you can set multiStream to false.

danger

setQuality would not have any effect if multiStream is set to false.

What is BitrateMode?

BitrateMode is the key setting for video quality. It lets you decide whether to prioritize sharp details, save internet data, or find a good balance between the two.

You can choose from three distinct modes:

• .BANDWIDTH_OPTIMIZED: Prioritizes lower bandwidth usage over video quality. Ideal for users with poor or unstable network conditions.
• .BALANCED: The default setting. It provides a smart compromise between clear video and efficient bandwidth consumption, suitable for most use cases.
• .HIGH_QUALITY: Prioritizes video quality over bandwidth usage. Use this when high-fidelity video is essential and viewers are expected to have strong network connections.

What is MaxLayer?

MaxLayer is an advanced parameter that gives you direct control over simulcast, which is the process of sending multiple versions (or "layers") of the same video stream at different resolutions and qualities simultaneously.

• .MAX_LAYER_3 (Default): Publishes all available layers (e.g., high, medium, and low quality). This provides the maximum adaptability for viewers on diverse and unpredictable networks.
• .MAX_LAYER_2: Intelligently publishes only the highest and lowest quality layers, skipping any in the middle. This is useful for providing a high-quality option and a low-bandwidth fallback without the overhead of a third stream.
  • If a device is capable of producing a h720p_w960p stream, setting EncodingLayer.MAX_LAYER_2 will result in two streams being sent: the highest quality (h720p_w960p) and the lowest quality (e.g. h180p_w240p), ensuring a fallback for poor connections.

info

To use the maxLayer parameter, multiStream must be set to true. The multiStream flag enables simulcasting (sending multiple quality streams), while maxLayer specifies how many streams to send.

note

• If the video width or height is 960 or higher, it will create 3 layers.
• If it’s 360 or higher and less than 960, it will create 2 layers.
• If it’s below 360, it will create only 1 layer.
  • For example, if you set MaxLayer to EncodingLayer.MAX_LAYER_3 but your video is 320×180, it will only create 1 layer.

What is codec?

note

SDK supports.AV1 and .VP9 codecs from 2.8.0 and onward SDK versions.

codec specifies the video codec used to encode your stream when you join the meeting. A codec determines how your video is compressed before it's sent to other participants, and each one strikes a different balance between video quality, CPU usage, device support, and multiStream (simulcast) capability. VideoSDK supports four codecs — .VP8 (default), .H264, .VP9, and .AV1 — set via the VideoCodec enum.

Which codec should I use?

Codec	Quality / Compression	CPU	Support	multiStream	Recommended Use
VP8	Good	Low	Very Broad	✅	Default choice for maximum compatibility
H.264	Good	Low	Broad	❌	Mobile and enterprise environments
VP9	Better	Medium-High	Modern Devices	❌	Improved video quality on modern devices
AV1	Best	High*	Growing	❌	Highest video quality where supported

• Hardware acceleration can significantly reduce CPU usage.

caution

Currently, H264, VP9, and AV1 do not support multiStream. If you pass H264, VP9, or AV1 with multiStream: true, VideoSDK will automatically set multiStream to false and emit the ERROR_MULTISTREAM_NOT_SUPPORTED error event.

danger

AV1 has limited device support — older devices may not be able to encode or decode it. Test AV1 on your target devices before using it in production.

Codec fallback behaviour

VideoSDK automatically handles codec compatibility, so an unsupported codec never interrupts a meeting.

• When video cannot be sent using the provided codec

  If your device does not support the provided codec, VideoSDK automatically changes the codec to .VP8 and emits the ERROR_VIDEO_PRODUCE_CODEC_NOT_SUPPORTED onError event callback.

  extension MeetingViewController: MeetingEventListener {
  
     func onError(error: VideoSDKError) {
        if error == .ERROR_VIDEO_PRODUCE_CODEC_NOT_SUPPORTED {
            // VideoSDK switched the codec to VP8 because the provided codec is not supported.
        }
     }
     
  }

• When video cannot be received using the provided codec

  If a participant's device or browser does not support the codec currently being used in the meeting, the participant receives the CODEC_NOT_SUPPORTED error through the meeting's onError event callback.

  extension MeetingViewController: MeetingEventListener {
  
     func onError(error: VideoSDKError) {
        if error == .CODEC_NOT_SUPPORTED {
            // Current codec is not supported on this participant's device/browser
        }
     }
  
  }

  VideoSDK then switches all participants to VP8, and every participant receives the onCodecChanged event callback through the MeetingEventListener.

  extension MeetingViewController: MeetingEventListener {
  
     func onCodecChanged(unsupportedCodec: VideoCodec, currentCodec: VideoCodec, previousCodec: VideoCodec?, kind: String, causedBy: Participant?) {
        print("Current codec:", currentCodec)
        print("Unsupported remote codec:", unsupportedCodec)
        print("Media kind:", kind)
        print("Participant that caused the codec switch:", causedBy?.displayName ?? "N/A")
        print("Previous codec:", previousCodec ?? .VP8)
    }
     
  }

info

This switch is controlled by the codecSwitchEnabled flag in the initMeeting() (default true). If set to false, the codec won't switch to .VP8 and the unsupported participant will see a black screen for the affected video.

How to Setup a Custom Video Track ?

The custom track can be set up both before and after the initialization of the meeting.

1. Setting up a Custom Track during the initialization of a meeting
2. Setting up a Custom Track with methods

1. Setting up a Custom Track during the initialization of a meeting

If you are passing webcamEnabled: true in the createRoom and want to use custom tracks from start of the meeting, you can pass custom track in the customCameraVideoTrack as shown below.

caution

• Custom Track will not apply on webcamEnabled: false configuration.

• Custom video track's facingMode takes precedence over the cameraPosition specified during the join method. Ensure you set facingMode appropriately to achieve the desired camera orientation.

Example

Swift

import VideoSDKRTC

guard let videoMediaTrack = try? VideoSDK.createCameraVideoTrack(
    encoderConfig: .h720p_w960p,
    facingMode: .front,
    multiStream: true,
    codec: .H264 // .VP8 | .VP9 | .AV1 , Default : .VP8
    bitrateMode: .HIGH_QUALITY, // Default: .BALANCED
    maxLayer: .MAX_LAYER_2 // Default: .MAX_LAYER_3
) else {
    return
}

let meeting = VideoSDK.initMeeting(
    meetingId: meetingId,
    participantName: name,
    micEnabled: micEnabled, // optional, default: true
    webcamEnabled: cameraEnabled, // optional, default: true
    // Pass the custom track here which will be used to when webcam is auto started
    customCameraVideoStream: videoMediaTrack
)

2. Setting up a Custom Track with methods

In order to switch tracks during the meeting, you have to pass the CustomTrack in the enableWebCam() method of Room.

tip

Make sure to call disableWebcam() before you create a new track as it may lead to unexpected behavior.

Example

Swift

import VideoSDKRTC

@IBAction func videoButtonTapped(_ sender: Any) {
    if !on {
        guard let videoMediaTrack = try? VideoSDK.createCameraVideoTrack(
            encoderConfig: .h720p_w960p,
            facingMode: .front,
            multiStream: false,
             codec: .H264 // .VP8 | .VP9 | .AV1 , Default : .VP8
            bitrateMode: .HIGH_QUALITY, // Default: .BALANCED
            maxLayer: .MAX_LAYER_2 // Default: .MAX_LAYER_3
        ) else {
            return
        }
        self.meeting?.enableWebcam(customVideoStream: videoMediaTrack)
    } else {
        self.meeting?.disableWebcam()
    }
}

Which Configuration is suitable for Device ?

In this section, we will understand participant size wise encoder(Resolution) and multiStream configuration.

API Reference

The API references for all the methods and events utilised in this guide are provided below.

• createCameraVideoTrack()