Custom Video Track - Android

• VideoSDK.createCameraVideoTrack() creates a camera track with the resolution, camera, facing mode, bitrate mode, layers and codec you choose.
• The camera opens when the track is created, so you can preview it before you join. To publish it, pass it to createRoom() as customCameraTrack, or to room.enableCam().
• It is a suspend function, so call it from a coroutine. It returns once the camera is open, and you can call it from the main thread.
• Every parameter is optional. Pass the arguments by name.

createCameraVideoTrack

suspend fun createCameraVideoTrack(
    encoderConfig: CustomCameraTrackConfig? = null,
    facingMode: FacingMode? = null,
    multiStream: Boolean = true,
    cameraId: String? = null,
    maxLayer: EncodingLayer? = null,
    bitrateMode: BitrateMode? = null,
    codec: VideoCodec? = null,
): CustomStreamTrack

Parameters

• encoderConfig:

  • type: CustomCameraTrackConfig?
  • required: false
  • default: CustomCameraTrackConfig.H720P_W1280P
  • You can choose from the below mentioned list of values for the encoder config. Each name reads H<height>P_W<width>P.

Config	Resolution	Frame Rate	Optimized (kbps)	Balanced (kbps)	High Quality (kbps)
H90P_W160P	160x90	15 fps	40	90	120
H120P_W160P	160x120	15 fps	40	90	120
H144P_W176P	176x144	15 fps	40	90	120
H180P_W240P	240x180	15 fps	90	120	180
H180P_W320P	320x180	15 fps	60	120	180
H240P_W320P	320x240	15 fps	100	150	200
H360P_W480P	480x360	25 fps	270	360	540
H360P_W640P	640x360	25 fps	300	450	600
H480P_W640P	640x480	25 fps	350	600	900
H480P_W720P	720x480	30 fps	350	600	900
H540P_W960P	960x540	30 fps	600	900	1200
H720P_W960P	960x720	30 fps	900	1200	1600
H720P_W1280P	1280x720	30 fps	1000	1500	2000
H1080P_W1440P	1440x1080	30 fps	1500	2000	2700
H1080P_W1920P	1920x1080	30 fps	1500	2200	2700

The bitrate columns are the maximum bitrate for each bitrateMode.

note

These encoder configurations are valid for both landscape and portrait mode.

• facingMode:

  • type: FacingMode?
  • required: false
  • default: FacingMode.front
  • Allowed values : FacingMode.front | FacingMode.back
  • Whether to open the front or the back camera.

• multiStream:

  • type: Boolean
  • required: false
  • default: true
  • If true, the camera is sent as several resolution layers, so each viewer can receive the layer its connection can carry. If false, it is sent as a single layer.

• cameraId:

  • type: String?
  • required: false
  • The deviceId of a camera from VideoSDK.getCameras(). It takes priority over facingMode. If it is null, the facingMode camera is used.

• maxLayer:

  • type: EncodingLayer?
  • required: false
  • Allowed values : EncodingLayer.MAX_LAYER_2 | EncodingLayer.MAX_LAYER_3
  • default: EncodingLayer.MAX_LAYER_3
  • Specifies the maximum number of simulcast layers to publish. This parameter only has an effect if multiStream is true. To send a single layer, set multiStream = false.

• bitrateMode:

  • type: BitrateMode?
  • required: false
  • Allowed values : BitrateMode.BANDWIDTH_OPTIMIZED | BitrateMode.BALANCED | BitrateMode.HIGH_QUALITY
  • default: BitrateMode.BALANCED
  • Controls the video quality and bandwidth consumption. You can choose between BitrateMode.HIGH_QUALITY for the best picture, BitrateMode.BANDWIDTH_OPTIMIZED to save data, or BitrateMode.BALANCED for a mix of both.

• codec:

  • type: VideoCodec?
  • required: false
  • Allowed values : VideoCodec.VP8 | VideoCodec.VP9 | VideoCodec.AV1 | VideoCodec.H264
  • default: VideoCodec.VP8
  • The codec to send the camera with. A codec other than VP8 is sent as a single layer: if multiStream is true, the SDK reports 3046 (ERROR_MULTISTREAM_NOT_SUPPORTED) on onError. If the device cannot encode the codec, the SDK reports 3045 (ERROR_VIDEO_PRODUCE_CODEC_NOT_SUPPORTED) and sends VP8. See Video Codecs.

note

To apply your own effects to the camera frames, for example filters from an external SDK, use a video processor. See Video Processor.

info

• To learn more about optimizations and best practices for using custom video tracks, follow this guide.

Returns

• CustomStreamTrack. A failure throws instead.

Throws

• IllegalStateException if VideoSDK.initialize() has not run.

VideoSDKException, with one of these codes:

• 3017 (ERROR_CAMERA_ACCESS_DENIED_OR_DISMISSED) when the CAMERA permission is not granted. The method does not ask for it, so request it first, for example with VideoSDK.checkPermissions().
• 3021 (ERROR_CAMERA_NOT_FOUND) when the device has no camera.
• 3071 (ERROR_INVALID_CUSTOM_TRACK_CONFIG) when cameraId names a camera the device does not have.
• 3053 (ERROR_VIDEO_TRACK_CREATION_FAILED) when the capture profile cannot be read.
• 3023 (ERROR_CAMERA_IN_USE) when another app holds the camera, 3019 (ERROR_CAMERA_PERMISSION_DENIED_BY_OS) when a device policy has disabled the camera, or 3026 (ERROR_VIDEO_SOURCE_INITIATION_FAILED) when the camera fails to open for another reason.

These errors are not reported on onError. A camera that fails after the track was created ends the track: set onEnded to find out.

Example

lifecycleScope.launch {
    try {
        val videoCustomTrack = VideoSDK.createCameraVideoTrack(
            encoderConfig = CustomCameraTrackConfig.H720P_W960P,
            facingMode = FacingMode.front,
            multiStream = true,
            maxLayer = EncodingLayer.MAX_LAYER_2,
            bitrateMode = BitrateMode.HIGH_QUALITY,
        )
    } catch (e: VideoSDKException) {
        Log.e("VideoSDK", "Camera track failed: ${e.code} ${e.name}")
    }
}

Custom Audio Track - Android

• VideoSDK.createMicrophoneAudioTrack() creates a microphone track with the audio profile, device and audio processing you choose.
• The mic opens when the track is created. To publish it, pass it to createRoom() as customMicrophoneTrack, or to room.unmuteMic().
• It is a suspend function, so call it from a coroutine. It returns once the mic is open, and you can call it from the main thread.
• Every parameter is optional. Pass the arguments by name.

createMicrophoneAudioTrack

suspend fun createMicrophoneAudioTrack(
    encoderConfig: CustomMicrophoneTrackConfig? = null,
    microphoneId: String? = null,
    noiseConfig: NoiseConfig? = null,
): CustomStreamTrack

Parameters

• encoderConfig:

  • type: CustomMicrophoneTrackConfig?
  • required: false
  • default: CustomMicrophoneTrackConfig.SPEECH_STANDARD
  • You can choose from the below mentioned list of values for the encoder config.

Encoder Config	Channels	Sample Rate	Bitrate	Auto Gain	Echo Cancellation	Noise Suppression
SPEECH_LOW_QUALITY	Mono	16 kHz	24 kbps	TRUE	TRUE	TRUE
SPEECH_STANDARD	Mono	32 kHz	24 kbps	TRUE	TRUE	TRUE
MUSIC_STANDARD	Mono	48 kHz	40 kbps	FALSE	FALSE	FALSE
STANDARD_STEREO	Stereo	48 kHz	32 kbps	FALSE	FALSE	FALSE
HIGH_QUALITY	Mono	48 kHz	32 kbps	FALSE	FALSE	FALSE
HIGH_QUALITY_STEREO	Stereo	48 kHz	32 kbps	FALSE	FALSE	FALSE

• microphoneId:

  • type: String?
  • required: false
  • The deviceId of an audio device from VideoSDK.getMics(), such as a wired or Bluetooth headset. If it is null, the current mic is used.

• noiseConfig:

  • type: NoiseConfig?
  • required: false
  • Turns audio processing on or off for this track, with the flags noiseSuppression, echoCancellation, autoGainControl and highPassFilter.
  • If you pass a NoiseConfig, it decides every flag. A flag you leave out takes its default: true, true, true and false. Set a flag to null to keep the value from encoderConfig.
  • If it is null, the values from encoderConfig in the table above are used.

Returns

• CustomStreamTrack. A failure throws instead.

Throws

• IllegalStateException if VideoSDK.initialize() has not run.

VideoSDKException, with one of these codes:

• 3018 (ERROR_MICROPHONE_ACCESS_DENIED_OR_DISMISSED) when the RECORD_AUDIO permission is not granted. The method does not ask for it, so request it first.
• 3022 (ERROR_MICROPHONE_NOT_FOUND) when the device has no audio input.
• 3071 (ERROR_INVALID_CUSTOM_TRACK_CONFIG) when microphoneId names a device the phone does not have.
• 3024 (ERROR_MICROPHONE_IN_USE) when the mic will not open or record.

These errors are not reported on onError. A mic that fails after the track was created ends the track: set onEnded to find out.

Example

lifecycleScope.launch {
    try {
        val audioCustomTrack = VideoSDK.createMicrophoneAudioTrack(
            encoderConfig = CustomMicrophoneTrackConfig.SPEECH_STANDARD,
            noiseConfig = NoiseConfig(highPassFilter = true),
        )
    } catch (e: VideoSDKException) {
        Log.e("VideoSDK", "Microphone track failed: ${e.code} ${e.name}")
    }
}

Custom Screen Share Track - Android

• VideoSDK.createScreenShareVideoTrack() creates a screen-share track with the resolution and frame rate you choose.
• The track holds the user's consent and your settings. Screen capture starts when you pass the track to room.enableScreenShare().

createScreenShareVideoTrack

fun createScreenShareVideoTrack(
    encoderConfig: String? = null,
    data: Intent?,
    multiStream: Boolean = false,
    withAudio: Boolean = false,
): CustomStreamTrack?

Parameters

• encoderConfig:

  • type: String?
  • required: false
  • default: h720p_15fps, used when you pass null
  • You can choose from the below mentioned list of values for the encoder config.

Encoder Config	Resolution	Frame Rate	Bitrate
h360p_30fps	640x360	30 fps	400 kbps
h480p_15fps	854x480	15 fps	300 kbps
h480p_30fps	854x480	30 fps	500 kbps
h720p_5fps	1280x720	5 fps	400 kbps
h720p_15fps	1280x720	15 fps	1000 kbps
h720p_30fps	1280x720	30 fps	1500 kbps
h1080p_15fps	1920x1080	15 fps	1500 kbps
h1080p_30fps	1920x1080	30 fps	2000 kbps

note

These encoder configurations are valid for both landscape and portrait mode.

• data

  • type: Intent?
  • required: true
  • The result Intent from the screen-capture consent screen (MediaProjectionManager.createScreenCaptureIntent()). null means the user declined.

• multiStream

  • type: Boolean
  • required: false
  • default: false
  • If true, the screen is sent as up to three simulcast layers instead of a single stream.

• withAudio

  • type: Boolean
  • required: false
  • default: false
  • If true, the audio your device plays is shared too. It needs Android 10 (API 29) or later.
  • The shared audio is mixed into your mic track, so it is sent only while your mic is on, and muting the mic mutes both. No separate SHARE_AUDIO stream is sent, so other participants hear it in your microphone stream and getShareAudioStats() returns null for you.

Returns

• CustomStreamTrack?: the track, or null when the user declined or encoderConfig is not in the table above. The second case also reports 3071 (ERROR_INVALID_CUSTOM_TRACK_CONFIG) on onError.

Example

// data: the result Intent of the screen-capture consent screen
val track = VideoSDK.createScreenShareVideoTrack(encoderConfig = "h720p_15fps", data = data)
if (track != null) {
    lifecycleScope.launch {
        try {
            room.enableScreenShare(track)
        } catch (e: VideoSDKException) {
            Log.e("VideoSDK", "Screen share failed: ${e.code} ${e.name}")
        }
    }
}

CustomStreamTrack

The methods above return a CustomStreamTrack. It holds the settings it was created with, such as encoderConfig, facingMode, cameraId, multiStream, maxLayer, bitrateMode and codec, and these members:

• track: the videosdk.org.webrtc.MediaStreamTrack behind it. For a camera track it is a VideoTrack that you can pass to VideoView.addTrack() to preview the camera before you join. It is null for a screen-share track.
• isEnded: true once the track has ended: it was stopped, its device was closed, or the device failed to open.
• stop(): releases the camera or mic the track opened. A track the room is publishing is left alone: disableCam(), muteMic() or leave() releases it.
• onEnded: a ((VideoSDKError?) -> Unit)? called once, on the main thread, when the track ends. Its argument is the VideoSDKError that made the device fail, such as ERROR_CAMERA_IN_USE, or null when the track was stopped, disabled, replaced or released by leaving the room. Set it to null to remove it. Set after the track has ended, it is called at once.

import videosdk.org.webrtc.VideoTrack

// preview the camera before joining
(videoCustomTrack.track as? VideoTrack)?.let { videoView.addTrack(it) }

// find out when the track ends, for example when the camera fails
videoCustomTrack.onEnded = { error ->
    Log.d("VideoSDK", "Track ended: ${error?.name}")
}

// release the camera if you don't publish the track
videoCustomTrack.stop()