Virtual Background - React

Virtual backgrounds enhance your live stream experience by allowing you to replace your physical background with a digital image or apply a blur effect. This is ideal for creators and streamers who want to maintain privacy, reduce distractions, or add a personal touch to their broadcast. You can use a preloaded background image or supply your own via a CDN.

important

• Currently, this feature is only available on Google Chrome, Firefox and Brave browser.

Install VideoSDK Media Processor package

NPM

npm install --save "@videosdk.live/videosdk-media-processor-web"

YARN

yarn add "@videosdk.live/videosdk-media-processor-web"

Instantiate Virtual Background Processor

After installing the library, initialize an instance of the VirtualBackgroundProcessor.

// Import package
import { VirtualBackgroundProcessor } from "@videosdk.live/videosdk-media-processor-web";

const MeetingView = () => {
  // Instantiate VirtualBackgroundProcessor Class
  const videoProcessor = new VirtualBackgroundProcessor();

  const handleStartVirtualBackground = () => {};

  const handleStopVirtualBackground = () => {};

  const handleChangeConfig = () => {};

  return (
    <>
      <button onClick={handleStartVirtualBackground}>
        Start Virtual Background
      </button>
      <button onClick={handleChangeConfig}>Change Virtual Background</button>
      <button onClick={handleStopVirtualBackground}>
        Stop Virtual Background
      </button>
    </>
  );
};

Using Virtual Background

Utilize the processor by first initializing it, then start processing a stream, and finally, pass the processed video stream to VideoSDK.

Initializing

The first step is to initialize the virtual background processor if it's not already prepared.

const MeetingView = () =>{
  // Instantiate VirtualBackgroundProcessor Class
  const videoProcessor = new VirtualBackgroundProcessor();

  const handleStartVirtualBackground = () => {
    // Initialize processor if not ready
    if (!videoProcessor.ready) {
      await videoProcessor.init();
    }
  }
  const handleStopVirtualBackground = () => {
    ...
  }
  return <>...</>;
}

Starting the Processor

To initiate the processing of a video stream, you need to provide a MediaStream that you want to process and a config to be used for processing. Once these parameters are passed to the processor, it will return a MediaStream.

1. MediaStream : MediaStream is essentially a video stream captured from the camera. You can use the createCameraVideoTrack method to generate a MediaStream.

2. config : This object accepts type and imageUrl property. type property accepts filter type image or blur. imageUrl is the path of the image and it will be ignored on blur filter type.

   NOTE : If you intend to display a specific background image, make sure it is uploaded on a CDN.

import { useMeeting, createCameraVideoTrack } from "@videosdk.live/react-sdk";
import { VirtualBackgroundProcessor } from "@videosdk.live/videosdk-media-processor-web";

const MeetingView = () => {
  // Instantiate VirtualBackgroundProcessor Class
  const videoProcessor = new VirtualBackgroundProcessor();

  const { changeWebcam } = useMeeting({});

  const handleStartVirtualBackground = async () => {
    // Initialize processor if not ready
    if (!videoProcessor.ready) {
      await videoProcessor.init();
    }

    // Configuration for starting processor
    const config = {
      type: "image", // "blur"
      imageUrl: "https://cdn.videosdk.live/virtual-background/cloud.jpeg",
      // Here is a list of background images you can use for your project.
      // imageUrl: "https://cdn.videosdk.live/virtual-background/beach.jpeg",
      // imageUrl: "https://cdn.videosdk.live/virtual-background/san-fran.jpeg",
      // imageUrl: "https://cdn.videosdk.live/virtual-background/paper-wall.jpeg",
    };

    // Getting stream from webcam
    const stream = await createCameraVideoTrack({});
    const processedStream = await videoProcessor.start(stream, config);
  };
  return <>...</>;
};

Passing Processed Stream to VideoSDK

Once you have the processed stream, you can pass it to functions like enableWebcam(), changeWebcam() or toggleWebcam() to apply the virtual background effect during your live stream.

const MeetingView = () => {
  // Instantiate VirtualBackgroundProcessor Class
  const videoProcessor = new VirtualBackgroundProcessor();

  const { changeWebcam } = useMeeting({});

  const handleStartVirtualBackground = async () => {
    // Initialize processor if not ready
    if (!videoProcessor.ready) {
      await videoProcessor.init();
    }

    // Configuration for starting processor
    const config = {
      type: "image", // "blur"
      imageUrl: "https://cdn.videosdk.live/virtual-background/cloud.jpeg",
      // Here is a list of background images you can use for your project.
      // imageUrl: "https://cdn.videosdk.live/virtual-background/beach.jpeg",
      // imageUrl: "https://cdn.videosdk.live/virtual-background/san-fran.jpeg",
      // imageUrl: "https://cdn.videosdk.live/virtual-background/paper-wall.jpeg",
    };

    // Getting stream from webcam
    const stream = await createCameraVideoTrack({});
    const processedStream = await videoProcessor.start(stream, config);

    changeWebcam(processedStream);
  };
  return <>...</>;
};

Updating Video Processor Configuration

If you want to change the background while the Video Processor is running, you can call the updateProcessorConfig method to modify the filters or processing type.

const MeetingView = () => {
  // Instantiate VirtualBackgroundProcessor Class
  const videoProcessor = new VirtualBackgroundProcessor();

  const { changeWebcam } = useMeeting({});

  const handleChangeConfig = async () => {
    const config = {
      type: "image", // "blur"
      imageUrl: "https://cdn.videosdk.live/virtual-background/cloud.jpeg",
    };

    videoProcessor.updateProcessorConfig(config);
  };
  return <>...</>;
};

Stopping Virtual Background Processor

Additionally, you can use the stop() method on the processor to halt the background processing and replace the video stream with a new plain video stream.

const MeetingView = () => {
  // Instantiate VirtualBackgroundProcessor Class
  const videoProcessor = new VirtualBackgroundProcessor();

  const { changeWebcam } = useMeeting({});

  const handleStopVirtualBackground = async () => {
    videoProcessor.stop();

    // Pass webcam MediaStream in VideoSDK `changeWebcam` method
    const stream = await createCameraVideoTrack({});
    changeWebcam(stream);
  };
  return <>...</>;
};

Extras

Want your live stream to start with virtual background already applied? Here’s how to pass the processed stream right at the beginning.

import { useState, useEffect } from "react";
import {
  MeetingProvider,
  createCameraVideoTrack,
} from "@videosdk.live/react-sdk";
import { VirtualBackgroundProcessor } from "@videosdk.live/videosdk-media-processor-web";

const [mediastream, setMediaStream] = useState(null);

// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

useEffect(async () => {
  const stream = await createCameraVideoTrack({});

  if (!videoProcessor.ready) {
    await videoProcessor.init();
  }
  const processedStream = await videoProcessor.start(stream, {
    type: "image", // "blur"
    imageUrl: `https://cdn.videosdk.live/virtual-background/cloud.jpeg`,
  });
  setMediaStream(processedStream);
}, []);

return (
  mediastream && (
    <MeetingProvider
      config={{
        meetingId,
        micEnabled: true,
        webcamEnabled: true,
        name: "TestUser",
        customCameraVideoTrack: mediastream, // Pass processed MediaStream in VideoSDK
      }}
      //...
    >
      <></>
    </MeetingProvider>
  )
);

API Reference

The API references for all the methods utilized in this guide are provided below.

• createCameraVideoTrack
• enableWebcam()
• disableWebcam()
• toggleWebcam()