Video Effects - React Native

The SDK can apply effects to your camera video before it is sent. Everyone who receives your video sees the effects, and so does your own video tile.

Effect	Setting	What it does
Auto-framing	framing	Crops and zooms the picture to keep a subject in the center. You provide the subject's position.
Low-light correction	lighting	Brightens the dark parts of the picture and leaves the bright parts alone.
Stylized look	style	Applies POSTERIZE or VIVID to the whole picture.
Color grade	filter	Grades the colors of the picture with WARM, COOL or MONO.

The effects run in this order: framing, lighting, style, then the color grade.

info

v2.0.0 has no background blur or virtual background, so none of these effects changes your background. To process camera frames with your own code or a third-party library, use a video processor instead.

note

This page uses the following asynchronous methods. Refer to their API reference for the errors each Promise may reject with, and handle these rejections appropriately based on your use case.

• setVideoEffects()
• getAppliedVideoEffectsConfig()

How it works

You describe the effects in a configuration object and pass it to setVideoEffects() from @videosdk.live/react-native-sdk.

• Effects run only with pipeline: "GRAPH". The default, "LEGACY", sends your camera video without effects.
• Each effect is off until you turn it on.
• On a device that cannot run the effects, your camera video is sent without effects, and the result of setVideoEffects() has applied set to false. In a room, onError also receives error 3080 (ERROR_VIDEO_EFFECTS_GRAPH_UNAVAILABLE).

Step 1: Turn effects on

Call setVideoEffects(). You can call it before the room joins. The effects start when your camera starts.

import { setVideoEffects } from "@videosdk.live/react-native-sdk";

const turnOnEffects = async () => {
  try {
    const result = await setVideoEffects({
      pipeline: "GRAPH",
      lighting: { enabled: true, strength: 0.8 },
      filter: { lut: "WARM", strength: 0.6 },
    });

    const reason = result.reason ?? "";
    // On Android, a GRAPH request made before the camera starts waits with a "pending:" reason
    if (result.applied !== true && !reason.startsWith("pending:")) {
      // The device cannot run the effects, so the video is sent without them
      console.warn("Video effects are not running:", reason);
    }
  } catch (err) {
    console.error("setVideoEffects failed:", err);
  }
};

On Android, a "GRAPH" request made before any camera has started returns applied: false with a reason that starts with pending:. This is not an error. The SDK keeps the request and runs it when the camera starts.

The settings of each effect:

• lighting: enabled, and strength from 0.0 to 1.0. The default strength is 1.0.
• style: preset is "OFF", "POSTERIZE" or "VIVID", and strength runs from 0.0 to 1.0. levels sets how many color steps "POSTERIZE" uses, from 3 to 17, and an even number is rounded up to the next odd number. The default is 5.
• filter: lut is "OFF", "WARM", "COOL" or "MONO", and strength runs from 0.0 to 1.0.
• framing: enabled, and maxZoom, how far the crop can zoom in, from 1.0 to 4.0. The default is 2.0. See Auto-framing.

A strength of 0.0 is the same as turning that effect off.

Step 2: Change effects

Call setVideoEffects() again with the new configuration.

• Each call replaces the whole configuration. An effect you leave out goes back to its default, so keep your effect settings in your app's state and build the full configuration from them every time.
• A change to an effect, such as a new color grade or strength, applies on the next frame.
• A change of pipeline, from "LEGACY" to "GRAPH" or back, applies on the next frame on iOS. On Android, it applies the next time the camera starts, for example after you turn the camera off and on again or switch cameras.
• To turn every effect off without restarting the camera, keep pipeline: "GRAPH" and turn each effect off.

import { useState } from "react";
import { Text, TouchableOpacity, View } from "react-native";
import { setVideoEffects } from "@videosdk.live/react-native-sdk";

const EffectsControls = () => {
  const [lowLight, setLowLight] = useState(false);
  const [warm, setWarm] = useState(false);

  // Builds the full configuration from all your settings
  const applyEffects = async (nextLowLight, nextWarm) => {
    try {
      await setVideoEffects({
        pipeline: "GRAPH",
        lighting: { enabled: nextLowLight },
        filter: { lut: nextWarm ? "WARM" : "OFF" },
      });
    } catch (err) {
      console.error("setVideoEffects failed:", err);
    }
  };

  const toggleLowLight = () => {
    setLowLight(!lowLight);
    applyEffects(!lowLight, warm);
  };

  const toggleWarm = () => {
    setWarm(!warm);
    applyEffects(lowLight, !warm);
  };

  return (
    <View>
      <TouchableOpacity onPress={toggleLowLight}>
        <Text>{lowLight ? "Low-light correction off" : "Low-light correction on"}</Text>
      </TouchableOpacity>
      <TouchableOpacity onPress={toggleWarm}>
        <Text>{warm ? "Warm colors off" : "Warm colors on"}</Text>
      </TouchableOpacity>
    </View>
  );
};

export default EffectsControls;

Step 3: Check which effects are running

getAppliedVideoEffectsConfig() returns the configuration in force, which can differ from the one you asked for. When applied is false, reason explains why.

import { getAppliedVideoEffectsConfig } from "@videosdk.live/react-native-sdk";

const checkEffects = async () => {
  const current = await getAppliedVideoEffectsConfig();
  // The result can be an empty object, so compare with true
  console.log("Effects running:", current.applied === true, current.reason);
};

Auto-framing

Auto-framing keeps a subject in the center of the picture. The SDK does not detect faces or people itself, so you tell it where the subject is with publishFramingSubject().

• Turn it on with framing: { enabled: true } and pipeline: "GRAPH".
• Pass the subject's box as fractions from 0 to 1 of the video that is sent, not of the raw camera image: the center (cx from left to right, cy from top to bottom), the width and the height. A smaller box zooms in further, up to maxZoom.
• Publish a new box each time your detection finds the subject, not once. The SDK keeps the last box until you publish another.
• Pass 0 for width and height when the subject is lost. After a short wait, the picture eases back to the full frame.
• Until you publish a box, the video is sent without framing.

import {
  publishFramingSubject,
  setVideoEffects,
} from "@videosdk.live/react-native-sdk";

const startFraming = async () => {
  await setVideoEffects({
    pipeline: "GRAPH",
    framing: { enabled: true, maxZoom: 2.0 },
  });
};

// Call this for every face your detector finds in a frame of frameWidth x frameHeight pixels
const onFaceDetected = async (box, frameWidth, frameHeight) => {
  await publishFramingSubject(
    (box.x + box.width / 2) / frameWidth,
    (box.y + box.height / 2) / frameHeight,
    box.width / frameWidth,
    box.height / frameHeight,
  );
};

Video effects and video processors

Camera effects and a video processor set with setVideoProcessor() don't run together. While effects run with pipeline: "GRAPH", the video processor receives no frames. Use one or the other.

API Reference

The API references for all the methods utilized in this guide are provided below.

• setVideoEffects()
• getAppliedVideoEffectsConfig()
• publishFramingSubject()
• VideoEffectsConfig