Custom Tracks | Video SDK

Version: 4.x.x

Custom Video Track - iOS

You can create a Video Track using createCameraVideoTrack() method of VideoSDK class.
Use it to create a video track with different encoding parameters, camera facing mode, bitrateMode, maxLayer and camera.
The camera opens when the track is created, so you can show a preview before you join. To publish the track, pass it to createRoom(customCameraTrack:), enableCam(customCameraTrack:) or changeCam(customCameraTrack:).
The method is async throws. Call it with try await.
The method never asks for camera permission. Ask for it first, for example with await AVCaptureDevice.requestAccess(for: .video).

Parameters

encoderConfig:
- type: CustomCameraTrackConfig
- required: false
- default: h720p_w1280p
- You can choose from the below mentioned list of values for the encoder config.

Config	Resolution	Frame Rate	Optimized (kbps)	Balanced (kbps)	High Quality (kbps)
h90p_w160p	160x90	30 fps	40	90	120
h180p_w320p	320x180	30 fps	60	120	180
h360p_w640p	640x360	30 fps	300	450	600
h540p_w960p	960x540	30 fps	600	900	1200
h720p_w1280p	1280x720	30 fps	1000	1500	2000
h1080p_w1920p	1920x1080	30 fps	1500	2200	2700
h120p_w160p	160x120	30 fps	40	90	120
h180p_w240p	240x180	30 fps	90	120	180
h240p_w320p	320x240	30 fps	100	150	200
h360p_w480p	480x360	30 fps	270	360	540
h480p_w640p	640x480	30 fps	350	600	900
h720p_w960p	960x720	30 fps	900	1200	1600
h1080p_w1440p	1440x1080	30 fps	1500	2000	2700

note

These encoder configurations work in both landscape and portrait mode.

facingMode:
- type: FacingMode
- required: false
- default: .front
- Allowed values : .front | .back
- It will specify whether to use front or back camera for the video track. If the device has no camera on that side, the other camera is used.
multiStream
- type: Bool
- required: false
- default: true
- It will specify if the stream should send multiple resolution layers or single resolution layer.
cameraId:
- type: String
- required: false
- default: nil
- The deviceId of the camera to use, from VideoSDK.getCameras(). It is the camera's AVCaptureDevice.uniqueID, not its name. When it is set, it decides the camera instead of facingMode.

info

To learn more about optimizations and best practices for using custom video tracks, follow this guide.

maxLayer:
- type: EncodingLayer
- required: false
- Allowed values : MAX_LAYER_2 | MAX_LAYER_3
- default: MAX_LAYER_3
- Specifies the maximum number of simulcast layers to publish. This parameter only has an effect if multiStream is set to true.
bitrateMode:
- type: BitrateMode
- required: false
- Allowed values : BitrateMode.BANDWIDTH_OPTIMIZED | BitrateMode.BALANCED | BitrateMode.HIGH_QUALITY
- Controls the video quality and bandwidth consumption. You can choose between BitrateMode.HIGH_QUALITY for the best picture, BitrateMode.BANDWIDTH_OPTIMIZED to save data, or BitrateMode.BALANCED for a mix of both. Defaults to BitrateMode.BALANCED.
codec
- type: VideoCodec
- required: false
- default: .VP8
- The video codec to send the camera with, for example .VP8 or .H264. To choose one, see Video Codecs.

Returns

CustomStreamTrack. Its videoTrack shows the camera preview. See CustomStreamTrack.

Throws

VideoSDKError, with one of these codes:

ERROR_CAMERA_ACCESS_DENIED_OR_DISMISSED (3017) when camera permission is not granted.
ERROR_CAMERA_NOT_FOUND (3021) when the device has no camera.
ERROR_INVALID_CUSTOM_TRACK_CONFIG (3071) when no camera matches cameraId.
ERROR_CAMERA_IN_USE (3023) when another app is using the camera, or ERROR_VIDEO_SOURCE_INITIATION_FAILED (3026) when the camera could not be opened. The method waits up to 2 seconds for the camera to open and throws these errors in that time. A later failure ends the track: use onEnded to find out.

Example

Task {
    do {
        let videoMediaTrack = try await VideoSDK.createCameraVideoTrack(
            encoderConfig: .h720p_w960p,
            facingMode: .front,
            multiStream: true,
            maxLayer: .MAX_LAYER_2, // Default: .MAX_LAYER_3
            bitrateMode: .HIGH_QUALITY, // Default: .BALANCED
            codec: .VP8 // Default: .VP8
        )
    } catch {
        print("Failed to create custom video track: \(error)")
    }
}

Custom Audio Track

You can create an Audio Track using createMicrophoneAudioTrack() method of VideoSDK class.
Use it to create an audio track with different noise configs and encoder configs.
The microphone opens when the track is created. To publish the track, pass it to createRoom(customMicrophoneTrack:), unmuteMic(customMicrophoneTrack:) or changeMic(customMicrophoneTrack:).
The method is async throws. Call it with try await.
The method never asks for microphone permission. Ask for it first, for example with await AVCaptureDevice.requestAccess(for: .audio).

Parameters

encoderConfig
- type: CustomMicrophoneTrackConfig
- required: false
- default: .speech_standard
- You can choose from the below mentioned list of values for the encoder config.

Config	Channels	Bitrate (kbps)	Noise Suppression, Echo Cancellation, Auto Gain
speech_low_quality	Mono	24	On
speech_standard	Mono	24	On
music_standard	Mono	40	Off
standard_stereo	Stereo	64	Off
high_quality	Mono	128	Off
high_quality_stereo	Stereo	192	Off

The last column is the default for those flags. noiseConfig can change them.

microphoneId
- type: String
- required: false
- default: nil
- The deviceId of the microphone to use, from VideoSDK.getMics().
noiseConfig
- type: NoiseConfig
- required: false
- Create it with NoiseConfig(noiseSuppression:echoCancellation:autoGainControl:highPassFilter:). Every flag is optional. A flag you leave out defaults to true, except highPassFilter, which defaults to false. Pass nil for a flag to keep the value from encoderConfig. If noiseConfig is nil, every flag comes from encoderConfig.
- noiseSuppression
  - type: Bool?, default true
  - If true, noise suppression would be turned on; otherwise, it would be off.
- echoCancellation
  - type: Bool?, default true
  - If true, echo cancellation would be turned on; otherwise, it would be off.
- autoGainControl
  - type: Bool?, default true
  - If true, autogain control would be turned on; otherwise, it would be off.
- highPassFilter
  - type: Bool?, default false
  - If true, highpass filter would be turned on; otherwise, it would be off. If it comes from encoderConfig, it is off.

Returns

CustomStreamTrack. See CustomStreamTrack.

Throws

VideoSDKError, with one of these codes:

ERROR_MICROPHONE_ACCESS_DENIED_OR_DISMISSED (3018) when microphone permission is not granted.
ERROR_MICROPHONE_NOT_FOUND (3022) when the device has no audio input.
ERROR_INVALID_CUSTOM_TRACK_CONFIG (3071) when no microphone matches microphoneId.
ERROR_MICROPHONE_IN_USE (3024) when another app is using the microphone. The method waits up to 2 seconds for the microphone to open and throws this error in that time. A later failure ends the track: use onEnded to find out.

Example

Task {
    do {
        let audioMediaTrack = try await VideoSDK.createMicrophoneAudioTrack(
            encoderConfig: .speech_standard,
            noiseConfig: NoiseConfig(
                noiseSuppression: true,
                echoCancellation: true,
                autoGainControl: true,
                highPassFilter: true
            )
        )
    } catch {
        print("Failed to create custom audio track: \(error)")
    }
}

You can create a Screen Share track using createScreenShareVideoTrack() method of VideoSDK class.
It sets the encoder profile for the share, separately from the camera. It does not start capture and does not throw.
Pass the track to enableScreenShare(_:), which shows the iOS system picker. The share starts when the user starts the broadcast. Screen sharing also needs a Broadcast Upload Extension, see Screen Share.

Parameters

encoderConfig
- type: CustomScreenShareTrackConfig
- required: false
- default: .h720p_15fps
- You can choose from the below mentioned list of values for the encoder config. The profile sets the frame rate and the bitrate of the share.

Config	Frame Rate	Bitrate (kbps)
h360p_30fps	30 fps	400
h480p_15fps	15 fps	300
h480p_30fps	30 fps	500
h720p_5fps	5 fps	400
h720p_15fps	15 fps	1000
h720p_30fps	30 fps	1500
h1080p_15fps	15 fps	1500
h1080p_30fps	30 fps	2000

multiStream
- type: Bool
- required: false
- default: false
- If true, the share is sent in three quality layers. If false, it is sent as one layer.
withAudio
- type: Bool
- required: false
- default: false
- Accepted and ignored: screen share on iOS carries no audio.

Returns

CustomStreamTrack. It holds only the share settings: its videoTrack and audioTrack are nil.

Example

let shareTrack = VideoSDK.createScreenShareVideoTrack(
    encoderConfig: .h1080p_15fps,
    multiStream: false
)

Task {
    do {
        try await room?.enableScreenShare(shareTrack)
    } catch {
        print("Failed to enable screen share: \(error)")
    }
}

You can also pass a profile directly, for example try await room?.enableScreenShare(.h1080p_15fps).

CustomStreamTrack

createCameraVideoTrack(), createMicrophoneAudioTrack() and createScreenShareVideoTrack() return a CustomStreamTrack. It has these members:

videoTrack: the camera's RTCVideoTrack. Add an RTCMTLVideoView to it to preview the camera before you join. It is nil for a microphone track.
audioTrack: the microphone's RTCAudioTrack. It is nil for a camera track.
codec, facingMode (FacingMode?), encoderConfig, cameraId, noiseConfig, microphoneId: the settings the track was created with.
isEnded: true once the track has ended: you called stop(), its device was closed (for example by disableCam(), muteMic() or leave()), you created a newer track for the same device, or the device failed to open.
onEnded: called once, on the main queue, when the track ends. It receives the VideoSDKError when the camera or microphone failed to open, and nil otherwise. A handler set after the track has ended is called at once.
stop(): closes the camera or microphone that the track opened, for example when the user leaves a preview screen without joining. If a room is sending from the device, the device stays open: use disableCam(), muteMic() or leave() to release it.

Passing an ended track to a room reports ERROR_CUSTOM_VIDEO_TRACK_ENDED (3031) or ERROR_CUSTOM_AUDIO_TRACK_ENDED (3032) to onError(error:), and the room uses its default camera or microphone instead. If enableCam(customCameraTrack:) or unmuteMic(customMicrophoneTrack:) is refused, for example because you have not joined yet, the SDK stops the track you passed. The SDK also stops the track if changeCam(customCameraTrack:) or changeMic(customMicrophoneTrack:) fails. Create a new track before you try again.

let previewView = RTCMTLVideoView(frame: .zero)
var cameraTrack: CustomStreamTrack?

// show a camera preview before joining
func startPreview() {
    Task {
        do {
            cameraTrack = try await VideoSDK.createCameraVideoTrack()
            cameraTrack?.videoTrack?.add(previewView)

            // find out when the track ends, for example when the camera fails to open
            cameraTrack?.onEnded = { error in
                print("Camera track ended: \(error?.name ?? "stopped")")
            }
        } catch {
            print("Failed to create camera track: \(error)")
        }
    }
}

// the user closed the preview screen without joining
func closePreview() {
    cameraTrack?.videoTrack?.remove(previewView)
    cameraTrack?.stop()
}

Got a Question? Ask us on discord

Custom Video Track - iOS​

Parameters​

Returns​

Throws​

Example​

Custom Audio Track​

Parameters​

Returns​

Throws​

Example​

Custom Screen Share Track​

Parameters​

Returns​

Example​

CustomStreamTrack​

Custom Video Track - iOS

Parameters

Returns

Throws

Example

Custom Audio Track

Parameters

Returns

Throws

Example

Custom Screen Share Track

Parameters

Returns

Example

CustomStreamTrack