Video Codecs - Flutter

A video codec compresses your camera video so it can be sent efficiently to other participants. VideoSDK supports four codecs, VP8 (the default), H264, VP9, and AV1, and each one balances picture quality, processing power, and support across browsers and devices in its own way. This guide helps you pick the right one and set it with the codec parameter of VideoSDK.createCameraVideoTrack().

note

The codec parameter is supported from version 3.10.0 onward.

How a Codec Works

Uncompressed camera video is far too large to send in real time, so a codec compresses it. It keeps the detail viewers notice most and stores each frame largely as the changes from the one before, which makes the video much smaller. Your device compresses the video with the codec you choose, and each participant's device uses the same codec to decompress it. VP9 and AV1 compress more efficiently than VP8 and H264, so they can deliver sharper video over the same connection.

Choosing a Codec

VP8 is a reliable starting point for most apps. The comparison below shows how the codecs differ, and each codec's section describes where it works best.

Comparing the four codecs

Codec	Quality / Compression	CPU	Support	Use it when
VP8	Good	Low	Very Broad	You want a dependable default
H264	Good	Low	Broad	Your devices or media pipeline work best with H.264
VP9	Better	Medium-High	Modern Devices	You want better picture quality on modern devices
AV1	Best	High*	Growing	You want the most efficient compression on devices you've tested

* Hardware acceleration can significantly reduce CPU usage.

VP8

• Google released VP8 as an open-source codec in 2010 as part of the WebM project. The WebRTC standard requires browsers to support it, so it runs on the widest range of browsers and devices.
• It is the default codec in VideoSDK and gives good video quality with low CPU use.
• VideoSDK supports multiStream for VP8, so it can publish several quality layers and send a lighter layer to participants viewing small tiles or on slower connections.
• It's also VideoSDK's fallback codec. If your device can't encode the codec you chose, or another participant can't decode it, VideoSDK switches your video to VP8 so the call keeps running.

H264

• H264, also known as AVC, was published in 2003 by the ITU-T and ISO/IEC MPEG. The WebRTC standard requires browsers to support it alongside VP8.
• It gives good video quality with low CPU use, and devices often encode it in hardware, which keeps the processing load light.
• Currently, VideoSDK doesn't support multiStream for H264, so it publishes H264 video as a single quality layer.
• It's a good fit when your devices or media pipeline work best with H.264. VP8 gives the same quality with broader support, so it's usually the simpler default.

VP9

• Google released VP9 in 2013 as the successor to VP8, and it compresses video more efficiently than VP8.
• It may need more processing power than older codecs to encode and decode.
• Currently, VideoSDK doesn't support multiStream for VP9, so it publishes VP9 video as a single quality layer and maxLayer has no effect.
• It works well for one-to-few calls on modern devices where picture quality matters most.

AV1

• The Alliance for Open Media finalized AV1 in 2018 as the successor to VP9. It has the most efficient compression of the four codecs, with excellent video quality.
• As the newest of the four, AV1 is still gaining hardware support and may need more processing power to encode and decode than older codecs. Most of that work happens on the device that sends the video, because decoding is lighter than encoding.
• Currently, VideoSDK doesn't support multiStream for AV1, so it publishes AV1 video as a single quality layer.
• It's a good fit when you want the most efficient compression on browsers and devices you've tested.

caution

AV1 browser support is still growing. In our testing it did not work on some versions of Safari (macOS) or on mobile web (Chrome and Safari on iPhone), so test it on your target browsers and devices before using it.

Setting the Codec

• codec is an optional parameter of the VideoSDK.createCameraVideoTrack() method. When you don't pass it, VideoSDK encodes the track with VideoCodec.VP8.
• Pass the codec as a VideoCodec value: VideoCodec.VP8, VideoCodec.H264, VideoCodec.VP9, or VideoCodec.AV1.
• To use the track, pass it to VideoSDK.createRoom() as customCameraVideoTrack, which applies it when camEnabled is true, or pass it to enableCam() during the meeting. Refer to How to Setup a Custom Video Track for full examples.
• The codec is set when you create the track. From version 3.11.0 onward, VideoSDK reuses that track's settings, codec included, when you turn the camera off and on again, unless it has already switched your video to VideoCodec.VP8. On earlier versions, create a new track with the same codec and pass it to enableCam() each time you turn the camera back on.
• To change the codec during the meeting, call disableCam(), then create a new track with the codec you want and pass it to enableCam().
• For the other parameters of VideoSDK.createCameraVideoTrack(), refer to the Optimize Video Tracks guide.

Example

CustomTrack? videoTrack = await VideoSDK.createCameraVideoTrack(
  encoderConfig: CustomVideoTrackConfig.h720p_w1280p,
  multiStream: false,
  codec: VideoCodec.VP9,
);

await room.enableCam(videoTrack);

Codec and multiStream

• multiStream (simulcast) publishes several quality layers of your video, so VideoSDK can send a lighter layer to participants viewing small tiles or on slower connections.

caution

Currently, VideoSDK doesn't support multiStream for H264, VP9, and AV1. multiStream defaults to true, so set multiStream: false when you create an H264, VP9, or AV1 track. Otherwise VideoSDK sets it to false for you and emits the ERROR_MULTISTREAM_NOT_SUPPORTED error event.

• maxLayer requires multiStream: true, so maxLayer has no effect with H264, VP9, or AV1. Refer to Optimize Video Tracks for what maxLayer controls.

How VideoSDK Handles Codec Compatibility

VideoSDK automatically handles codec compatibility, so an unsupported codec never interrupts a meeting.

When your device cannot send the codec

• If your device doesn't support the codec you passed, VideoSDK automatically switches your camera track to VideoCodec.VP8 so your video keeps flowing, and reports ERROR_VIDEO_PRODUCE_CODEC_NOT_SUPPORTED through the room's Events.error event. You can use this as a signal to pick a different codec for that kind of device.

Example

room.on(Events.error, (error) {
  if (error['name'] == 'ERROR_VIDEO_PRODUCE_CODEC_NOT_SUPPORTED') {
    // VideoSDK switched the codec to VP8 because the provided codec is not supported
  }
});

When a participant cannot receive the codec

• If a participant's device or browser can't decode the codec in use, that participant receives CODEC_NOT_SUPPORTED through the Events.error event.
• VideoSDK then switches to VideoCodec.VP8 every participant whose video that participant can't decode, so everyone can keep seeing each other. Those participants receive the Events.codecChanged event, and their video stays on VideoCodec.VP8 for the rest of the meeting, even after that participant leaves.

Example

room.on(Events.error, (error) {
  if (error['name'] == 'CODEC_NOT_SUPPORTED') {
    // Your device can't decode the codec another participant is sending
  }
});

room.on(Events.codecChanged, (CodecChangeInfo info) {
  // VideoSDK switched your own video to VP8 because another participant can't decode your codec
  print("Current codec: ${info.currentCodec.name}");
  print("Unsupported remote codec: ${info.unsupportedRemoteCodec.name}");
  print("Media kind: ${info.kind}");
  print("Participant that caused the codec switch: ${info.causedBy?.displayName}");
  print("Previous codec: ${info.previousCodec.name}");
});

info

The codecSwitchEnabled parameter of VideoSDK.createRoom() controls this switch and defaults to true. If you set it to false, VideoSDK keeps the current codec and does not emit Events.codecChanged. Participants who can decode that codec keep seeing the video as usual, while a participant who can't may see a black screen.

For each error's code and message, refer to the Error Events guide.

Checking Which Codec Is in Use

• VideoSDK may switch a participant's video to VideoCodec.VP8 during a meeting to keep the call running, so the codec in use isn't always the one that was requested.
• The getVideoStats() method of the Participant class returns a List of statistics entries, and each entry has a codec field. The first entry's codec tells you what that participant's video is encoded with. It returns null when no statistics are available. Refer to the Understanding Call Quality guide for the other statistics it returns.

Example

// Use room.localParticipant for your own video
final participant = room.participants[participantId];
final stats = participant?.getVideoStats();

if (stats != null && stats.isNotEmpty) {
  print("Codec in use: ${stats.first['codec']}");
}

note

These statistics are a snapshot. To follow a codec change as it happens, listen for the Events.codecChanged event instead of polling.

tip

For the smoothest experience, test your codec choice on the lowest-specification device you support, with participants joining from every platform your app runs on. This helps you find any device that can't use your codec, which would make VideoSDK switch video to VP8.

API Reference

The API references for all the methods and events utilized in this guide are provided below.

• VideoCodec
• createRoom()
• enableCam()
• disableCam()
• codecChanged
• CodecChangeInfo
• getVideoStats()