Simulcast (Adaptive Bitrate) - Flutter

Simulcast is a technique used in real-time video communication where a single camera publishes several versions of the same video at once. Each version is called a layer, and has its own resolution and bitrate.

VideoSDK then delivers the appropriate layer to each participant, based on their requirements and network conditions.

Why Simulcast Matters

Without simulcast, one camera publishes a single version of its video, and everyone receives it. That version has to suit every participant at once, so it is set either high enough for the participant with the best connection, which leaves the participant on a weak connection with frozen video, or low enough for the weakest connection, which leaves everyone else with a soft picture they did not need to settle for.

Simulcast removes that compromise. Because several layers are already being published, VideoSDK can give each participant a different one, and can move a participant to a lighter layer the moment their connection tightens and back up when it recovers. Nobody's experience is limited by anybody else's network.

Configuring Simulcast

Simulcast is configured when creating a video track using createCameraVideoTrack(). The two parameters that control simulcast are multiStream and maxLayer. If you don't pass a custom camera track, the camera takes multiStream from VideoSDK.createRoom() instead, and opens at the size set by its maxResolution parameter: 1280x720 with MaxResolution.HD, the default, or 960x540 with MaxResolution.SD.

multiStream

multiStream determines whether a camera track publishes multiple video layers or a single layer. It is true by default, so simulcast is enabled unless you explicitly disable it.

Setting multiStream to false publishes a single video layer. Every participant then receives the same layer regardless of their screen size or network conditions.

The number of layers depends on the resolution of the track's encoderConfig.

encoderConfig Resolution	Layers
Less than 480 pixels on the longer edge	1
480–959 pixels	2
960 pixels or higher	3

For example, a 1280x720 track can publish:

320x180  → Low
640x360  → Medium
1280x720 → High

On iOS, VideoSDK scales the camera video to the encoderConfig resolution before sending it, so a track publishes the layers listed above. On Android, the layers are scaled down from the video the camera actually captures. If the device cannot capture the requested resolution, the layers are smaller than listed.

maxLayer

maxLayer limits the maximum number of layers published by a track. It accepts EncodingLayer.MAX_LAYER_2 or EncodingLayer.MAX_LAYER_3, defaults to EncodingLayer.MAX_LAYER_3, and applies only when multiStream is true.

For example, a 1280x720 track publishes:

maxLayer: EncodingLayer.MAX_LAYER_3  →  320x180, 640x360, 1280x720
maxLayer: EncodingLayer.MAX_LAYER_2  →  320x180,          1280x720

With maxLayer set to EncodingLayer.MAX_LAYER_2, the middle layer is omitted, so the track publishes only the lowest and highest layers, which preserves both a high-quality option and a low-bandwidth fallback while encoding one stream fewer. This is worth reaching for on devices where encoding three streams costs more CPU or battery than the middle layer is worth.

info

maxLayer never adds layers beyond what the encoderConfig allows, and there is no middle layer to drop on a track that publishes two. On h360p_w640p or h480p_w640p, MAX_LAYER_2 and MAX_LAYER_3 produce the same result.

Refer to the Optimize Video Track guide for the full set of createCameraVideoTrack() parameters and for examples of passing a custom track to a room.

Selecting the Video Quality Layer

Once multiple video layers are published, your application can control which layer each participant receives.

danger

setQuality() chooses between published layers, so it has nothing to act on when a track was created with multiStream set to false. That track publishes one layer, and every participant receives it.

setQuality() sets the layer for a remote participant. It takes a VideoQuality: VideoQuality.LOW, VideoQuality.MEDIUM, or VideoQuality.HIGH. participant.videoQuality returns the quality you last requested, or null if you have not set one. Reach for it when your layout already tells you how much detail a view needs, so you can set the level from your own rules. A speaker in the main stage takes VideoQuality.HIGH, and the same participant in a sidebar takes VideoQuality.LOW.

setQuality() returns a Future that throws a VideoSDKError if the request fails, so await it inside try. It has no effect on the local participant, who is also part of room.participants. AI agents are in room.participants too, and setQuality() throws an UnsupportedError for an agent, so skip them with isAgent().

Example

// Give the main stage speaker the sharpest layer, and everyone else a light one
Future<void> applyGridQuality(String activeSpeakerId) async {
  // A copy, because participants can join or leave while you await
  for (final participant in room.participants.values.toList()) {
    // room.participants is remote only; an AI agent throws UnsupportedError
    if (participant.isAgent()) continue;
    try {
      await participant.setQuality(
          participant.id == activeSpeakerId ? VideoQuality.HIGH : VideoQuality.LOW);
    } on VideoSDKError catch (error) {
      // For example, this participant's camera is off
      print("Couldn't set the quality for ${participant.id}: ${error.code} ${error.name}");
    }
  }
}

note

The requested quality is a preference, not a guarantee. If the participant's network cannot support the requested layer, VideoSDK may deliver a lower layer and move back to the requested quality when the connection improves.

Capping the Quality You Send

You can also limit what your own camera sends, which saves upload bandwidth.

• setWebcamQuality() of the Room class caps the quality of the camera video you send. VideoQuality.HIGH sends every layer, VideoQuality.MEDIUM stops the largest one, and VideoQuality.LOW sends only the smallest, so other participants receive at most that quality. It returns a Future that throws a VideoSDKError if the cap can't be applied, for example ERROR_WEBCAM_NOT_ENABLED (3062) when your camera is off. The cap stays in force when you turn the camera off and on again.
• dynaCast, a parameter of VideoSDK.createRoom(), lets VideoSDK lower the quality of the camera video you send while no participant is watching its larger layers, and raise it again when someone is. It defaults to false, and never goes above the cap set with setWebcamQuality().

Neither has any effect on a camera that publishes a single layer, such as one with multiStream set to false or a codec other than VP8.

Example

final room = VideoSDK.createRoom(
  roomId: roomId,
  token: token,
  participantName: "John Doe",
  dynaCast: true,
);

// Later, while your camera is on
try {
  await room.setWebcamQuality(VideoQuality.MEDIUM);
} on VideoSDKError catch (error) {
  // For example 3062 when your camera is off
  print("Couldn't cap the camera quality: ${error.code} ${error.name}");
}

Checking the Current Video Layer

Use getVideoStats() to check which video quality layer you are receiving from a remote participant. It returns the sample fetched by your previous call and starts fetching the next one, so your first call returns null. It also returns null while the participant has no video stream. To follow layer changes, call it from a timer, for example once a second. The videoQualityChanged event is not delivered in v5.0.0.

Each entry includes:

• currentSpatialLayer — the layer currently being received.
• preferredSpatialLayer — the layer that was requested.
• currentTemporalLayer — the temporal layer currently being received.
• preferredTemporalLayer — the temporal layer that was requested.

Layers are numbered from 0, and a higher number is a higher-quality layer. On a track publishing three layers, 2 is the sharpest and 0 is the lightest. When currentSpatialLayer is below preferredSpatialLayer, a lighter layer than the one requested is being delivered, usually because the connection does not support the requested one yet.

A field is null until its layer is known. For example, preferredSpatialLayer stays null until you request a layer for that participant with setQuality() or setViewPort().

The stats also include metrics such as bitrate, codec, size, rtt, jitter, and packetsLost, plus limitation for your own video, which can help diagnose the current video quality.

Refer to the Understanding Call Quality guide for more information about these metrics.

Example

// Timer is in dart:async
final Timer layerTimer = Timer.periodic(const Duration(seconds: 1), (_) {
  final videoStats = participant.getVideoStats();

  if (videoStats != null && videoStats.isNotEmpty) {
    print("Layer in use: ${videoStats[0]['currentSpatialLayer']}");
    print("Layer requested: ${videoStats[0]['preferredSpatialLayer']}");
  }
});

// Call layerTimer.cancel() when you no longer need it

Simulcast and Codecs

The codec a track is encoded with determines whether it can publish layers at all.

• VP8, the default, supports multiStream, so a track using it publishes layers as described above.
• Refer to the Video Codecs guide for which codecs support multiStream on Flutter, and for how to handle the cases where a codec is not supported.

API Reference

The API references for all the methods and events utilized in this guide are provided below.

• createCameraVideoTrack()
• createRoom()
• setQuality()
• setWebcamQuality()
• getVideoStats()