Version: 5.x.x

Optimize Video Tracks - Flutter

While optimizing for the best viewing experience, it is necessary to fine-tune the video tracks that are being used during the calls.

You can pass a custom video track for the participant's media before and during the room.

Custom Video Track

This feature can be used to add custom video encoder configurations, such as the resolution, bitrate mode, layers and codec, and send the video to other participants.

Once you set a custom video track created using createCameraVideoTrack(), VideoSDK remembers its settings and automatically reuses them whenever the webcam is turned back on. To change the track configuration, call disableCam(), then pass a new track created using createCameraVideoTrack() to enableCam().

`How to Create a Custom Video Track ?`

You can create a Video Track using createCameraVideoTrack() method of VideoSDK class.
This method can be used to create video track by specifying parameters like video resolution, camera facing mode, and the quality controls (bitrateMode and maxLayer).
This method does not ask for camera permission, so call VideoSDK.requestPermissions() before you create the track.
If the track cannot be created, the method throws a VideoSDKError, for example 3017 without camera permission, 3021 when there is no camera, 3023 when the camera is in use, 3026 when the camera fails to open, or 3071 for an unknown cameraId.
The camera opens when the track is created. If you do not use the track, call stop() on it to close the camera.
Set the track's onEnded callback to hear when the track ends. It receives a VideoSDKError when the camera was lost or another app took it, and null after stop().
The video resolutions listed below are the encoder configuration options supported on Android and iOS. The bitrate columns show the maximum bitrate of the full-resolution video for each bitrateMode.

Encoder Config	Resolution	Optimized (kbps)	Balanced (kbps)	High Quality (kbps)
h90p_w160p	160x90	40	90	120
h180p_w320p	320x180	60	120	180
h360p_w640p	640x360	300	450	600
h540p_w960p	960x540	600	900	1200
h720p_w1280p	1280x720	1000	1500	2000
h1080p_w1920p	1920x1080	1500	2200	2700
h120p_w160p	160x120	40	90	120
h180p_w240p	240x180	90	120	180
h240p_w320p	320x240	100	150	200
h360p_w480p	480x360	270	360	540
h480p_w640p	640x480	350	600	900
h720p_w960p	960x720	900	1200	1600
h1080p_w1440p	1440x1080	1500	2000	2700

important

The actual video resolution may differ from the requested value due to device and hardware limitations. To check the camera settings in use during the room, call getAppliedVideoConfig() of the Room class.

note

Above mentioned encoder configurations are valid for both, landscape as well as portrait mode.

Example

import 'package:videosdk/videosdk.dart';

// The SDK does not ask for camera permission.
final permissions = await VideoSDK.requestPermissions(Permissions.VIDEO);
if (permissions['video'] != true) return;

try {
  CustomStreamTrack customTrack = await VideoSDK.createCameraVideoTrack(
    // It will be the id of the camera from which the video should be captured.
    cameraId: "camera-id", // OPTIONAL: the deviceId of a camera from VideoSDK.getCameras()

    // This will accept the resolution (height x width) of video you want to capture.
    encoderConfig: CustomCameraTrackConfig.h720p_w1280p, // CustomCameraTrackConfig.h360p_w640p ... // Default : CustomCameraTrackConfig.h720p_w1280p

    // It will specify whether to use front or back camera for the video track.
    facingMode: FacingMode.BACK, // FacingMode.FRONT,  Default : FacingMode.FRONT

    // We will discuss this parameter in next step.
    multiStream: true, // false,  Default : true

    // Optional: This controls the video quality and bandwidth usage.
    bitrateMode: BitrateMode.HIGH_QUALITY, // BitrateMode.BANDWIDTH_OPTIMIZED | BitrateMode.BALANCED , Default : BitrateMode.BALANCED

    // Optional: This specifies the maximum number of simulcast layers (maxLayer) to publish.
    maxLayer: EncodingLayer.MAX_LAYER_2, // EncodingLayer.MAX_LAYER_3, Default: EncodingLayer.MAX_LAYER_3

    // Optional: This specifies the video codec used to encode this track.
    codec: VideoCodec.VP8, // VideoCodec.H264 | VideoCodec.VP9 | VideoCodec.AV1 , Default : VideoCodec.VP8
  );
} on VideoSDKError catch (error) {
  // For example 3017 (permission denied), 3021 (no camera) or 3023 (camera in use)
  print("Could not create the track: ${error.code} ${error.name}");
}

caution

The capabilities of the device have a significant impact on how custom track configurations behave. Assuming a case where you set encoder configuration to 1080p but the webcam only supports 720p, then encoder configuration will automatically switch to the highest resolution that the device can handle, which is 720p.

What is `multiStream`?

This parameter specifies whether the stream should send multiple resolution layers or a single resolution layer. It is true by default.

Refer to the Simulcast (Adaptive Bitrate) guide for the layers a track publishes and for how participants select the layer they receive.

What is `BitrateMode`?

BitrateMode is the key setting for video quality. It lets you decide whether to prioritize sharp details, save internet data, or find a good balance between the two.

You can choose from three distinct modes:

BitrateMode.BANDWIDTH_OPTIMIZED: Prioritizes lower bandwidth usage over video quality. Ideal for users with poor or unstable network conditions.
BitrateMode.BALANCED: The default setting. It provides a smart compromise between clear video and efficient bandwidth consumption, suitable for most use cases.
BitrateMode.HIGH_QUALITY: Prioritizes video quality over bandwidth usage. Use this when high-fidelity video is essential and viewers are expected to have strong network connections.

What is `maxLayer`?

This parameter caps how many layers a track publishes, and accepts EncodingLayer.MAX_LAYER_2 or EncodingLayer.MAX_LAYER_3. It applies only when multiStream is true, and defaults to EncodingLayer.MAX_LAYER_3.

Refer to the Simulcast (Adaptive Bitrate) guide for what each value publishes.

What is `codec`?

This parameter specifies the video codec used to encode your stream. VideoSDK supports VP8 (default), H264, VP9, and AV1, set through VideoCodec.
Currently, VideoSDK doesn't support multiStream for H264, VP9, and AV1. multiStream defaults to true, so set multiStream: false when you create an H264, VP9, or AV1 track. Otherwise VideoSDK sets it to false for you and reports ERROR_MULTISTREAM_NOT_SUPPORTED (3046) through the room's Events.error event.

Refer to the Video Codecs guide to compare the four codecs, to check which codec a room is actually using, and to handle the cases where a codec is not supported.

`How to Setup a Custom Video Track ?`

The custom track can be set up both before and after the initialization of the room.

Setting up a Custom Track during the initialization of a room
Setting up a Custom Track with methods

1. Setting up a Custom Track during the initialization of a room

If you are passing camEnabled: true in the createRoom and want to use custom tracks from start of the room, you can pass custom track in the customCameraTrack as shown below.

caution

Custom Track will not apply on camEnabled: false configuration.

Example

import 'dart:async' show unawaited;
import 'package:flutter/material.dart';
import 'package:videosdk/videosdk.dart';

class RoomScreen extends StatefulWidget {
  final String roomId;
  final String token;

  const RoomScreen(
      {super.key, required this.roomId, required this.token});

  @override
  State<RoomScreen> createState() => _RoomScreenState();
}

class _RoomScreenState extends State<RoomScreen> {
  late Room _room;

  @override
  void initState() {
    super.initState();
    unawaited(initRoom());
  }

  Future<void> initRoom() async {
    // On Android, the SDK does not ask for camera and microphone permission.
    await VideoSDK.requestPermissions();

    //Creating Custom Video Track
    CustomStreamTrack? videoTrack;
    try {
      videoTrack = await VideoSDK.createCameraVideoTrack(
        encoderConfig: CustomCameraTrackConfig.h720p_w960p,
        multiStream: true,
        bitrateMode: BitrateMode.HIGH_QUALITY,
        maxLayer: EncodingLayer.MAX_LAYER_2,
        codec: VideoCodec.VP8,
      );
    } on VideoSDKError catch (error) {
      print("Could not create the track: ${error.code} ${error.name}");
    }

    //Creating a new Room with custom video track
    // create room
    _room = VideoSDK.createRoom(
      roomId: widget.roomId,
      token: widget.token,
      participantName: "John Doe",
      micEnabled: true,
      camEnabled: true,
      customCameraTrack: videoTrack, // custom video track :: optional
    );
  }

  @override
  Widget build(BuildContext context) {
    return YourRoomWidget();
  }
}

2. Setting up a Custom Track with methods

In order to switch tracks during the room, you have to pass the CustomStreamTrack in the enableCam() method of Room. You can also switch to it in one step with changeCam(customCameraTrack: videoTrack), which turns the camera on.

tip

Call disableCam() before you create a new track. While the camera is on, enableCam() with a track throws 3054 and releases the track.

Example

import 'dart:async' show unawaited;
import 'package:flutter/material.dart';
import 'package:videosdk/videosdk.dart';

class RoomScreen extends StatefulWidget {
  final String roomId;
  final String token;

  const RoomScreen(
      {super.key, required this.roomId, required this.token});

  @override
  State<RoomScreen> createState() => _RoomScreenState();
}

class _RoomScreenState extends State<RoomScreen> {
  late Room _room;

  @override
  void initState() {
    super.initState();
    unawaited(initRoom());
  }

  Future<void> initRoom() async {
    ...
  }

  @override
  Widget build(BuildContext context) {
    return Column(
      children: [
        ElevatedButton(
          onPressed: () async {
            try {
              // The camera must be off before you pass a new track
              if (_room.isCamEnabled) await _room.disableCam();

              //Creating Custom Video Track
              CustomStreamTrack videoTrack = await VideoSDK.createCameraVideoTrack(
                  encoderConfig: CustomCameraTrackConfig.h720p_w960p,
                  multiStream: true,
                  bitrateMode: BitrateMode.HIGH_QUALITY,
                  maxLayer: EncodingLayer.MAX_LAYER_2,
                  codec: VideoCodec.VP8,
              );

              await _room.enableCam(videoTrack);
            } on VideoSDKError catch (error) {
              print("Could not enable the camera with the custom track: ${error.code} ${error.name}");
            }
          },
          child: const Text("Enable Cam with Custom Track"),
        ),
      ],
    );
  }
}

`Which Configuration is suitable for Device ?`

In this section, we will understand participant size wise encoder(Resolution) and multiStream configuration.

API Reference

The API references for all the methods and events utilised in this guide are provided below.

Got a Question? Ask us on discord

Custom Video Track​

How to Create a Custom Video Track ?​

Example​

What is multiStream?​

What is BitrateMode?​

What is maxLayer?​

What is codec?​

How to Setup a Custom Video Track ?​

1. Setting up a Custom Track during the initialization of a room​

Example​

2. Setting up a Custom Track with methods​

Example​

Which Configuration is suitable for Device ?​

API Reference​

Custom Video Track

`How to Create a Custom Video Track ?`

Example

What is `multiStream`?

What is `BitrateMode`?

What is `maxLayer`?

What is `codec`?

`How to Setup a Custom Video Track ?`

1. Setting up a Custom Track during the initialization of a room

Example

2. Setting up a Custom Track with methods

Example

`Which Configuration is suitable for Device ?`

API Reference