Skip to main content
Version: 0.1.x

Virtual Background (BETA)

Virtual backgrounds allow participants in a video call or meeting to replace their actual background with a digital image or video, which can be either preloaded images or user-uploaded images. This feature can be useful for a variety of reasons, such as maintaining privacy by hiding potentially distracting or private elements in the background, reducing visual clutter to help focus on the speaker or content, and adding an element of fun or creativity to a meeting.

info
  • Currently, this feature is only available on Google Chrome, Firefox and Brave browser.
  • This feature is in Beta release, so feel free to reach out to us on Discord. We'd love to hear feedback from you.

virtual-background

Install VideoSDK Media Processor package

npm install --save "@videosdk.live/videosdk-media-processor-web"

Instantiate Virtual Background Processor

With the library installed lets create an instance of the VirtualBackgrouProcessor.

// Import package
import { VirtualBackgroundProcessor } from "@videosdk.live/videosdk-media-processor-web";

const MeetingView = () => {
// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

const handleStartVirtualBackground = () => {};

const handleStopVirtualBackground = () => {};

const handleChangeConfig = () => {};

return (
<>
<button onClick={handleStartVirtualBackground}>
Start Virtual Background
</button>
<button onClick={handleChangeConfig}>Change Virtual Background</button>
<button onClick={handleStopVirtualBackground}>
Stop Virtual Background
</button>
</>
);
};

Using Virtual Background

To use the virtual background processor, you will be first initializing it, then you will start processing a stream and finally pass the processed video stream to the VideoSDK.

Initializing

First thing to do is initialize the virtual background processor if its not already ready.

const MeetingView = () =>{
// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

const handleStartVirtualBackground = () => {
// Initialize processor if not ready
if (!videoProcessor.ready) {
await videoProcessor.init();
}
}
const handleStopVirtualBackground = () => {
...
}
return <>...</>;
}

Starting the Processor

To start processsing a video stream you will need MediaStram which you want to process and config which should be used from processing. When the required parameters are passed to the processor, it will return MediaStream.

  1. MediaStream : MediaStream is just a video stream which is being capture from the camera. You can use the createCameraVideoTrack to create a MediaStream.

  2. config : This object accepts type and imageUrl property. type property accepts filter type image or blur. imageUrl is the path of the image and it will be ignored on blur filter type.

    NOTE : If you want to display specific background image, do make sure it is uploaded on CDN.

import { useMeeting, createCameraVideoTrack } from "@videosdk.live/react-sdk";
import { VirtualBackgroundProcessor } from "@videosdk.live/videosdk-media-processor-web";

const MeetingView = () => {
// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

const { changeWebcam } = useMeeting({});

const handleStartVirtualBackground = async () => {
// Initialize processor if not ready
if (!videoProcessor.ready) {
await videoProcessor.init();
}

// Configuration for starting processor
const config = {
type: "image", // "blur"
imageUrl: "https://cdn.videosdk.live/virtual-background/cloud.jpeg",
// Here is a list of background images you can use for your project.
// imageUrl: "https://cdn.videosdk.live/virtual-background/beach.jpeg",
// imageUrl: "https://cdn.videosdk.live/virtual-background/san-fran.jpeg",
// imageUrl: "https://cdn.videosdk.live/virtual-background/paper-wall.jpeg",
};

// Getting stream from webcam
const stream = await createCameraVideoTrack({});
const processedStream = await videoProcessor.start(stream, config);
};
return <>...</>;
};

Passing Processed Stream to VideoSDK

Now that we have the processed stream you can pass it in the enableWebcam(), changeWebcam() or toggleWebcam().

const MeetingView = () => {
// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

const { changeWebcam } = useMeeting({});

const handleStartVirtualBackground = async () => {
// Initialize processor if not ready
if (!videoProcessor.ready) {
await videoProcessor.init();
}

// Configuration for starting processor
const config = {
type: "image", // "blur"
imageUrl: "https://cdn.videosdk.live/virtual-background/cloud.jpeg",
// Here is a list of background images you can use for your project.
// imageUrl: "https://cdn.videosdk.live/virtual-background/beach.jpeg",
// imageUrl: "https://cdn.videosdk.live/virtual-background/san-fran.jpeg",
// imageUrl: "https://cdn.videosdk.live/virtual-background/paper-wall.jpeg",
};

// Getting stream from webcam
const stream = await createCameraVideoTrack({});
const processedStream = await videoProcessor.start(stream, config);

changeWebcam(processedStream);
};
return <>...</>;
};

Updating Video Processor Configuration

If you wish to change the background while the Video Processor is running, you can call the updateProcessorConfig method and change the filters or processing type.

const MeetingView = () => {
// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

const { changeWebcam } = useMeeting({});

const handleChangeConfig = async () => {
const config = {
type: "image", // "blur"
imageUrl: "https://cdn.videosdk.live/virtual-background/cloud.jpeg",
};

videoProcessor.updateProcessorConfig(config);
};
return <>...</>;
};

Stopping Virtual Background Processor

You can use the stop() on the processor to stop the background processing and then replace the video stream with a new plain video stream.

const MeetingView = () => {
// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

const { changeWebcam } = useMeeting({});

const handleStopVirtualBackground = async () => {
videoProcessor.stop();

// Pass webcam MediaStream in VideoSDK `changeWebcam` method
const stream = await createCameraVideoTrack({});
changeWebcam(stream);
};
return <>...</>;
};

Extras

You can also pass this processed stream during initialization of the meeting as well

import { useState, useEffect } from "react";
import {
MeetingProvider,
createCameraVideoTrack,
} from "@videosdk.live/react-sdk";
import { VirtualBackgroundProcessor } from "@videosdk.live/videosdk-media-processor-web";

const [mediastream, setMediaStream] = useState(null);

// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

useEffect(async () => {
const stream = await createCameraVideoTrack({});

if (!videoProcessor.ready) {
await videoProcessor.init();
}
const processedStream = await videoProcessor.start(stream, {
type: "image", // "blur"
imageUrl: `https://cdn.videosdk.live/virtual-background/cloud.jpeg`,
});
setMediaStream(processedStream);
}, []);

return (
mediastream && (
<MeetingProvider
config={{
meetingId,
micEnabled: true,
webcamEnabled: true,
name: "TestUser",
customCameraVideoTrack: mediastream, // Pass processed MediaStream in VideoSDK
}}
//...
>
<></>
</MeetingProvider>
)
);

API Reference

The API references for all the methods utilised in this guide are provided below.