Optimize Video Track - iOS

While optimizing for the best viewing experience, it is necessary to fine-tune the video tracks that are being used during the calls.

You can pass a custom video track for the participant's media before joining the room and while in it.

Custom Video Track

A custom video track lets you tune the video for varying network conditions. You can set a video quality profile (bitrateMode) and control simulcast behavior (maxLayer).

Once you set a custom video track created using createCameraVideoTrack(), VideoSDK remembers its settings and automatically reuses them whenever the camera is turned back on. To change the track configuration while in the room, pass a new track created the same way to changeCam(customCameraTrack:). VideoSDK preserves only tracks created with this method. A video processor that you set with setVideoProcessor(processor:), such as a virtual background, also stays applied when the camera is turned off and on again. To remove it, call removeVideoProcessor().

How to Create a Custom Video Track ?

• You can create a Video Track using createCameraVideoTrack() method of VideoSDK class.

• This method can be used to create video track by specifying parameters like video resolution, camera facing mode, the camera to use (cameraId), and the quality controls (bitrateMode and maxLayer).

• The method is async throws, so call it with try await. It returns a CustomStreamTrack.

• The camera opens when the track is created, but the method never asks for camera permission. Ask for permission first, and create the track after access is granted. Until then, the method throws ERROR_CAMERA_ACCESS_DENIED_OR_DISMISSED.

• The method waits up to 2 seconds for the camera to open, and throws the errors that happen in that time, such as ERROR_CAMERA_IN_USE. If the camera fails after that, the track ends and its onEnded handler receives the error.

• If you create a track and don't use it, call stop() on it to release the camera.

• You can choose video resolution from the below mentioned list of values for the encoder config:

Config	Resolution	Frame Rate	Optimized (kbps)	Balanced (kbps)	High Quality (kbps)
h90p_w160p	160x90	30 fps	40	90	120
h180p_w320p	320x180	30 fps	60	120	180
h360p_w640p	640x360	30 fps	300	450	600
h540p_w960p	960x540	30 fps	600	900	1200
h720p_w1280p	1280x720	30 fps	1000	1500	2000
h1080p_w1920p	1920x1080	30 fps	1500	2200	2700
h120p_w160p	160x120	30 fps	40	90	120
h180p_w240p	240x180	30 fps	90	120	180
h240p_w320p	320x240	30 fps	100	150	200
h360p_w480p	480x360	30 fps	270	360	540
h480p_w640p	640x480	30 fps	350	600	900
h720p_w960p	960x720	30 fps	900	1200	1600
h1080p_w1440p	1440x1080	30 fps	1500	2000	2700

note

The encoder configurations above are valid for both landscape and portrait mode.

Example

Swift

import VideoSDKRTC

do {
  let videoMediaTrack = try await VideoSDK.createCameraVideoTrack(
    // This will accept the enum value of CustomCameraTrackConfig which contains resolution (height x width) of video you want to capture.
    encoderConfig: .h720p_w960p, // .h360p_w640p | .h480p_w640p ... // Default : .h720p_w1280p

    // It will specify whether to use front or back camera for the video track.
    facingMode: .front, // .back,  Default : .front

    // We will discuss this parameter in next step.
    multiStream: true, // false,  Default : true

    // Optional: A deviceId from VideoSDK.getCameras(). When set, it decides the camera instead of facingMode.
    cameraId: nil, // Default : nil

    // Optional: This specifies the maximum number of simulcast layers (maxLayer) to publish.
    maxLayer: EncodingLayer.MAX_LAYER_2, // 2 |  Default: .MAX_LAYER_3

    // Optional: This controls the video quality and bandwidth usage.
    bitrateMode: BitrateMode.HIGH_QUALITY, // .BANDWIDTH_OPTIMIZED | .BALANCED Default: .BALANCED

    // Optional: This specifies the video codec used to encode this track.
    codec: .VP8 // .H264 | .VP9 | .AV1 , Default : .VP8
  )
  // Pass videoMediaTrack to createRoom() or changeCam(customCameraTrack:)
} catch {
  print("Failed to create the video track: \(error)")
}

caution

The camera on the device limits how sharp the video can be. If you set an encoder configuration that the camera can't capture, for example 1080p on a camera that supports only 720p, VideoSDK captures at the highest resolution the camera supports and scales it up. The video is still sent at the configured resolution and bitrate, but it is not sharper than what the camera captured.

What is multiStream?

• This parameter specifies whether the stream should send multiple resolution layers or a single resolution layer. It is true by default.

Refer to the Simulcast (Adaptive Bitrate) guide for the layers a track publishes and for how participants select the layer they receive.

What is BitrateMode?

BitrateMode is the key setting for video quality. It lets you decide whether to prioritize sharp details, save internet data, or find a good balance between the two.

You can choose from three distinct modes:

• .BANDWIDTH_OPTIMIZED: Prioritizes lower bandwidth usage over video quality. Ideal for users with poor or unstable network conditions.
• .BALANCED: The default setting. It provides a smart compromise between clear video and efficient bandwidth consumption, suitable for most use cases.
• .HIGH_QUALITY: Prioritizes video quality over bandwidth usage. Use this when high-fidelity video is essential and viewers are expected to have strong network connections.

What is MaxLayer?

• This parameter caps how many layers a track publishes, and accepts EncodingLayer.MAX_LAYER_2 or EncodingLayer.MAX_LAYER_3. It applies only when multiStream is true, and defaults to .MAX_LAYER_3.

Refer to the Simulcast (Adaptive Bitrate) guide for what each value publishes.

What is codec?

• This parameter specifies the video codec used to encode your stream. VideoSDK supports VP8 (default), H264, VP9, and AV1, set through VideoCodec.
• Currently, VideoSDK doesn't support multiStream for H264, VP9, and AV1. multiStream defaults to true, so set multiStream: false when you create an H264, VP9, or AV1 track. Otherwise VideoSDK sets it to false for you and reports the ERROR_MULTISTREAM_NOT_SUPPORTED error through onError(error:).

Refer to the Video Codecs guide to compare the four codecs, to check which codec a room is actually using, and to handle the cases where a codec is not supported.

How to Setup a Custom Video Track ?

The custom track can be set up both when you create the room and after you join it.

1. Setting up a Custom Track when you create the room
2. Setting up a Custom Track with methods

1. Setting up a Custom Track when you create the room

If you are passing camEnabled: true to createRoom and want to use custom tracks from the moment you join the room, you can pass the custom track as customCameraTrack, as shown below.

caution

• With camEnabled: false, the camera stays off when you join, so the custom track is not sent. When you turn the camera on later with enableCam(), VideoSDK uses the settings of this track.

• Custom video track's facingMode (or cameraId, when set) takes precedence over the defaultCamera specified in the createRoom method. Ensure you set facingMode appropriately to achieve the desired camera orientation.

Example

Swift

import VideoSDKRTC

do {
    let videoMediaTrack = try await VideoSDK.createCameraVideoTrack(
        encoderConfig: .h720p_w960p,
        facingMode: .front,
        multiStream: true,
        maxLayer: .MAX_LAYER_2, // Default: .MAX_LAYER_3
        bitrateMode: .HIGH_QUALITY, // Default: .BALANCED
        codec: .VP8 // .H264 | .VP9 | .AV1 , Default : .VP8
    )

    let room = try VideoSDK.createRoom(
        roomId: roomId,
        token: token,
        participantName: name,
        micEnabled: micEnabled, // optional, default: true
        camEnabled: cameraEnabled, // optional, default: true
        // Pass the custom track here which will be used when the camera is auto started
        customCameraTrack: videoMediaTrack
    )
} catch {
    print("Failed to set up the room: \(error)")
}

2. Setting up a Custom Track with methods

In order to switch tracks while in the room, you have to pass the new track to the changeCam(customCameraTrack:) method of Room. If the camera is on, VideoSDK turns it off and on again with the new track, so the new settings apply at once. If the camera is off, it turns on with the new track.

You can also pass a track to enableCam(customCameraTrack:) when you turn the camera on. Both methods are async throws, so call them with try await inside a Task.

Example

Swift

import VideoSDKRTC

@IBAction func changeVideoTrackTapped(_ sender: Any) {
    Task {
        do {
            let videoMediaTrack = try await VideoSDK.createCameraVideoTrack(
                encoderConfig: .h720p_w960p,
                facingMode: .front,
                multiStream: true,
                maxLayer: .MAX_LAYER_2, // Default: .MAX_LAYER_3
                bitrateMode: .HIGH_QUALITY, // Default: .BALANCED
                codec: .VP8 // .H264 | .VP9 | .AV1 , Default : .VP8
            )
            try await self.room?.changeCam(customCameraTrack: videoMediaTrack)
        } catch {
            print("Failed to change the video track: \(error)")
        }
    }
}

Which Configuration is suitable for Device ?

The chart below shows the encoder resolution and multiStream setting to use for each participant count.

API Reference

The API references for all the methods and events utilised in this guide are provided below.

• createCameraVideoTrack()
• changeCam()
• enableCam()
• removeVideoProcessor()