Video Codecs - React Native

A video codec compresses your camera video so it can be sent efficiently to other participants. VideoSDK supports four codecs, VP8 (the default), H264, VP9, and AV1, and each one balances picture quality, processing power, and support across browsers and devices in its own way. This guide helps you pick the right one and set it with the codec parameter of createCameraVideoTrack().

How a Codec Works

Uncompressed camera video is far too large to send in real time, so a codec compresses it. It keeps the detail viewers notice most and stores each frame largely as the changes from the one before, which makes the video much smaller. Your device compresses the video with the codec you choose, and each participant's device uses the same codec to decompress it. VP9 and AV1 compress more efficiently than VP8 and H264, so they can deliver sharper video over the same connection.

Choosing a Codec

VP8 is a reliable starting point for most apps. The comparison below shows how the codecs differ, and each codec's section describes where it works best.

Comparing the four codecs

Codec	Quality / Compression	CPU	Support	Use it when
VP8	Good	Low	Very Broad	You want a dependable default
H264	Good	Low	Broad	Your devices or media pipeline work best with H.264
VP9	Better	Medium-High	Modern Devices	You want better picture quality on modern devices
AV1	Best	High*	Growing	You want the most efficient compression on devices you've tested

* Hardware acceleration can significantly reduce CPU usage.

VP8

• Google released VP8 as an open-source codec in 2010. It runs on the widest range of browsers and devices.
• It is the default codec in VideoSDK and gives good video quality with low CPU use.
• VideoSDK supports multiStream for VP8, so it can publish several quality layers and send a lighter layer to participants viewing small tiles or on slower connections.
• It's also VideoSDK's fallback codec. If your device can't encode the codec you chose, or another participant can't decode it, VideoSDK switches your video to VP8 so the call keeps running.

H264

• H264, also known as AVC, was published in 2003 by the ITU-T and ISO/IEC MPEG. Browsers support it alongside VP8.
• It gives good video quality with low CPU use, and devices often encode it in hardware, which keeps the processing load light.
• VideoSDK doesn't support multiStream for H264, so it publishes H264 video as a single quality layer.
• It's a good fit when your devices or media pipeline work best with H.264. VP8 gives the same quality with broader support, so it's usually the simpler default.

VP9

• Google released VP9 in 2013 as the successor to VP8, and it compresses video more efficiently than VP8.
• It may need more processing power than older codecs to encode and decode.
• VideoSDK doesn't support multiStream for VP9, so it publishes VP9 video as a single quality layer and maxLayer has no effect.
• It works well for one-to-few calls on modern devices where picture quality matters most.

AV1

• The Alliance for Open Media finalized AV1 in 2018 as the successor to VP9. It has the most efficient compression of the four codecs, with excellent video quality.
• As the newest of the four, AV1 is still gaining hardware support and may need more processing power to encode and decode than older codecs. Most of that work happens on the device that sends the video, because decoding is lighter than encoding.
• VideoSDK doesn't support multiStream for AV1, so it publishes AV1 video as a single quality layer.
• It's a good fit when you want the most efficient compression on devices you've tested.

caution

AV1 device support is still growing, and older devices may not be able to encode or decode it. Test it on your target devices before using it in production.

Setting the Codec

• codec is an optional parameter of the createCameraVideoTrack() method. When you don't pass it, VideoSDK encodes the track with VP8.
• Pass the codec as one of the strings "VP8", "H264", "VP9", or "AV1".
• To use the track, pass it to RoomProvider as customCameraTrack, which applies it when camEnabled is true, or pass it to enableCam() during the room. Refer to How to Setup a Custom Video Track for full examples.
• The codec is set when you create the track. When you turn the webcam off and on again, VideoSDK reuses that track's settings, codec included, unless it has already switched your video to VP8.
• To change the codec during the room, call disableCam(), then create a new track with the codec you want and pass it to enableCam(). Creating a new camera track while the webcam is still on may lead to unexpected behavior.
• For the other parameters of createCameraVideoTrack(), refer to the Optimize Video Tracks guide.

Example

import { createCameraVideoTrack, useRoom } from "@videosdk.live/react-native-sdk";
import { TouchableOpacity, Text } from "react-native";

// Use this while your webcam is off. To switch codecs, call disableCam() first.
const StartWebcamWithVP9 = () => {
  const { enableCam } = useRoom();

  const handleEnableWebcam = async () => {
    try {
      const customTrack = await createCameraVideoTrack({
        encoderConfig: "h720p_w1280p",
        multiStream: false,
        codec: "VP9",
      });

      await enableCam(customTrack);
    } catch (err) {
      console.error("Could not start the webcam:", err);
    }
  };

  return (
    <TouchableOpacity onPress={handleEnableWebcam}>
      <Text>Enable webcam</Text>
    </TouchableOpacity>
  );
};

Codec and multiStream

• multiStream (simulcast) publishes several quality layers of your video, so VideoSDK can send a lighter layer to participants viewing small tiles or on slower connections.

caution

VideoSDK doesn't support multiStream for H264, VP9, and AV1. multiStream defaults to true for a track from createCameraVideoTrack(), so set multiStream: false when you create an H264, VP9, or AV1 track. Otherwise VideoSDK sets it to false for you and reports ERROR_MULTISTREAM_NOT_SUPPORTED (3046) through the onError callback.

• maxLayer requires multiStream: true, so maxLayer has no effect with H264, VP9, or AV1. Refer to Optimize Video Tracks for what maxLayer controls.

How VideoSDK Handles Codec Compatibility

VideoSDK automatically handles codec compatibility, so an unsupported codec never interrupts a room.

When your device cannot send the codec

• If your device doesn't support the codec you passed, VideoSDK automatically switches your camera track to VP8 so your video keeps flowing, and reports ERROR_VIDEO_PRODUCE_CODEC_NOT_SUPPORTED (3045) through the onError callback. You can use this as a signal to pick a different codec for that kind of device.
• If the device can encode neither the codec you passed nor VP8, no video is sent, and VideoSDK reports ERROR_NO_SUPPORTED_VIDEO_CODEC (3052). Check for it both in the onError callback and when enableCam() rejects.

Example

import { useRoom, Constants } from "@videosdk.live/react-native-sdk";

function onError(error) {
  if (error.code === Constants.VideoSDKError.ERROR_VIDEO_PRODUCE_CODEC_NOT_SUPPORTED) {
    // VideoSDK switched the codec to VP8 because the provided codec is not supported
  }
}

const mRoom = useRoom({
  onError,
});

When a participant cannot receive the codec

• If a participant's device or browser can't decode the codec in use, that participant receives CODEC_NOT_SUPPORTED (4051) through the onError callback.
• VideoSDK then switches to VP8 every participant whose video that participant can't decode, so everyone can keep seeing each other. Those participants receive the onCodecChanged event through the useRoom hook, and their video stays on VP8 for the rest of the room, even after that participant leaves.

Example

import { useRoom, Constants } from "@videosdk.live/react-native-sdk";

function onError(error) {
  if (error.code === Constants.VideoSDKError.CODEC_NOT_SUPPORTED) {
    // Your device can't decode the codec another participant is sending
  }
}

const mRoom = useRoom({
  onError,
  onCodecChanged: (data) => {
    // VideoSDK switched your own video to VP8 because another participant can't decode your codec
    console.log("Current codec:", data.currentCodec);
    console.log("Unsupported remote codec:", data.unsupportedRemoteCodec);
    console.log("Media kind:", data.kind);
    // causedBy is the Participant whose codec support forced the switch, or null
    console.log("Participant that caused the codec switch:", data.causedBy?.id);
    console.log("Previous codec:", data.previousCodec);
  },
});

Turning off the switch to VP8

• The switch to VP8 is controlled by codecSwitchEnabled in the config of RoomProvider. It is true by default.
• With codecSwitchEnabled: false, VideoSDK does not switch your video to VP8, and onCodecChanged is not called.

Example

<RoomProvider
  config={{
    roomId: "<Id-of-room>",
    codecSwitchEnabled: false, // default: true
  }}
  token={token}
>
  <RoomView />
</RoomProvider>

For each error's code and message, refer to the Error Events guide.

Checking Which Codec Is in Use

• VideoSDK may switch a participant's video to VP8 during a room to keep the call running, so the codec in use isn't always the one that was requested.
• The getVideoStats() method of the useParticipant hook resolves with an array of statistics entries, and each entry has a codec field, such as video/VP8. The first entry's codec tells you what that participant's video is encoded with. When the participant has no active video, or before the first statistics are collected, it resolves null or an empty array, so check for an entry before you read it. Refer to the Understanding Call Quality guide for the other statistics it returns.

Example

import { useParticipant } from "@videosdk.live/react-native-sdk";
import { TouchableOpacity, Text } from "react-native";

const ParticipantCodec = ({ participantId }) => {
  const { getVideoStats } = useParticipant(participantId);

  const checkCodec = async () => {
    try {
      const stats = await getVideoStats();
      console.log("Codec in use:", stats?.[0]?.codec); // for example "video/VP8"
    } catch (err) {
      console.error("getVideoStats failed:", err);
    }
  };

  return (
    <TouchableOpacity onPress={checkCodec}>
      <Text>Check codec</Text>
    </TouchableOpacity>
  );
};

note

These statistics are a snapshot. To follow a change to your own codec as it happens, listen for the onCodecChanged event instead of polling.

tip

For the smoothest experience, test your codec choice on the lowest-specification device you support, with participants joining from every platform your app runs on. This helps you find any device that can't use your codec, which would make VideoSDK switch video to VP8.

API Reference

The API references for all the methods and events utilized in this guide are provided below.

• createCameraVideoTrack()
• customCameraTrack
• codecSwitchEnabled
• enableCam()
• disableCam()
• onError()
• onCodecChanged()
• getVideoStats()