Version: 0.x.x

Virtual Background - Javascript

Virtual backgrounds enhance the Live Stream experience by enabling participants to replace their physical background with a digital image or video. This feature offers several benefits, including maintaining privacy, reducing visual distractions, and adding an element of creativity or fun to the liveStream. Users can choose from preloaded images or upload their own backgrounds.

important

Currently, this feature is supported only on desktop browsers.

Install VideoSDK Virtual Background package

NPM
YARN

npm install --save "@videosdk.live/videosdk-virtual-background-web"

yarn add "@videosdk.live/videosdk-virtual-background-web"

warning

Note: We will no longer be publishing new versions of the @videosdk.live/videosdk-media-processor-web package. Going forward, we will exclusively maintain a dedicated package for this feature. Please use @videosdk.live/videosdk-virtual-background-web instead.

Instantiate Virtual Background Processor

After installing the library, initialize an instance of the VirtualBackgroundProcessor.

// Import package
import { VirtualBackgroundProcessor } from "@videosdk.live/videosdk-virtual-background-web";

let liveStream;

// Initialize liveStream
liveStream = VideoSDK.initMeeting({
  // ...
});

// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

Using Virtual Background

Utilize the processor by first initializing it, then start processing a stream, and finally, pass the processed video stream to VideoSDK.

Initializing

The processor supports model selection during initialization.

VBModel.landscape → Uses Selfie Segmenter Landscape
VBModel.multiClass → Uses Multi-Class Selfie Segmentation
VBModel.meet → Uses MediaPipe Meet Segmentation.

note

The VBModel.multiClass model is currently in beta.

info

Model selection is available from version 0.1.0 and above. If no value is provided, VBModel.landscape will be used by default.

// Import package
import { VirtualBackgroundProcessor, VBModel, BlurIntensity } from "@videosdk.live/videosdk-virtual-background-web";

let liveStream;

// Initialize liveStream
liveStream = VideoSDK.initMeeting({
  // ...
});

// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

const StartVirtualBackgroundBtn = document.getElementById(
  "StartVirtualBackgroundBtn",
);
StartVirtualBackgroundBtn.addEventListener("click", async () => {
  // Initialize processor with model selection
  if (!videoProcessor.ready) {
    await videoProcessor.init(VBModel.landscape); // or "VBModel.multiClass" | "VBModel.meet"
  }
});

Starting the Processor

To initiate the processing of a video stream, you need to provide a MediaStream that you want to process and a config to be used for processing. Once these parameters are passed to the processor, it will return a MediaStream.

MediaStream : MediaStream is essentially a video stream captured from the camera. You can use the createCameraVideoTrack method to generate a MediaStream.
config : This object accepts type, imageUrl, and blurIntensity properties.
- type : Accepts filter type image or blur.
- imageUrl : The path of the image, which will be ignored when the type is blur.
- blurIntensity : The intensity of the blur effect. Accepts a value of BlurIntensity enum (low, mid, or high). The default value is mid. This property will be ignored when the type is image.
NOTE : If you intend to display a specific background image, make sure it is uploaded on a CDN.

info

Blur intensity support is available from version 0.1.1 and above.

// Import package
import { VirtualBackgroundProcessor, VBModel, BlurIntensity } from "@videosdk.live/videosdk-virtual-background-web";
let liveStream;

// Initialize liveStream
liveStream = VideoSDK.initMeeting({
  // ...
});

// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

const StartVirtualBackgroundBtn = document.getElementById(
  "StartVirtualBackgroundBtn",
);
StartVirtualBackgroundBtn.addEventListener("click", async () => {
  // Initialize processor if not ready
  if (!videoProcessor.ready) {
    await videoProcessor.init(VBModel.landscape); // or "VBModel.multiClass" | "VBModel.meet"
  }

  // Configuration for starting processor
  const config = {
    type: "image", // "blur"
    imageUrl: "https://cdn.videosdk.live/virtual-background/cloud.jpeg",
    // blurIntensity: BlurIntensity.mid, // "low" | "mid" | "high" (only applicable if type is "blur")
    // Here is a list of background images you can use for your project.
    // imageUrl: "https://cdn.videosdk.live/virtual-background/beach.jpeg",
    // imageUrl: "https://cdn.videosdk.live/virtual-background/san-fran.jpeg",
    // imageUrl: "https://cdn.videosdk.live/virtual-background/paper-wall.jpeg",
  };

  // Getting stream from webcam
  const stream = await VideoSDK.createCameraVideoTrack({});
  const processedStream = await videoProcessor.start(stream, config);
});

Passing Processed Stream to VideoSDK

Once you have the processed stream, you can pass it to functions like enableWebcam(), changeWebcam() or toggleWebcam() to apply the virtual background effect during your liveStream.

// Import package
import { VirtualBackgroundProcessor, VBModel, BlurIntensity } from "@videosdk.live/videosdk-virtual-background-web";

let liveStream;

// Initialize liveStream
liveStream = VideoSDK.initMeeting({
  // ...
});

// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

const StartVirtualBackgroundBtn = document.getElementById(
  "StartVirtualBackgroundBtn",
);
StartVirtualBackgroundBtn.addEventListener("click", async () => {
  // Initialize processor if not ready
  if (!videoProcessor.ready) {
    await videoProcessor.init(VBModel.landscape); // or "VBModel.multiClass" | "VBModel.meet"
  }

  // Configuration for starting processor
  const config = {
    type: "image", // "blur"
    imageUrl: "https://cdn.videosdk.live/virtual-background/cloud.jpeg",
    // blurIntensity: BlurIntensity.mid, // "low" | "mid" | "high" (only applicable if type is "blur")
    // Here is a list of background images you can use for your project.
    // imageUrl: "https://cdn.videosdk.live/virtual-background/beach.jpeg",
    // imageUrl: "https://cdn.videosdk.live/virtual-background/san-fran.jpeg",
    // imageUrl: "https://cdn.videosdk.live/virtual-background/paper-wall.jpeg",
  };

  // Getting stream from webcam
  const stream = await VideoSDK.createCameraVideoTrack({});
  const processedStream = await videoProcessor.start(stream, config);

  liveStream?.changeWebcam(processedStream);
});

Updating Video Processor Configuration

If you want to change the background while the Video Processor is running, you can call the updateProcessorConfig method to modify the filters or processing type.

// Import package
import { VirtualBackgroundProcessor } from "@videosdk.live/videosdk-virtual-background-web";
let liveStream;

// Initialize liveStream
liveStream = VideoSDK.initMeeting({
  // ...
});

// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

const updateVirtualBackgroundBtn = document.getElementById(
  "updateVirtualBackgroundBtn",
);
updateVirtualBackgroundBtn.addEventListener("click", async () => {
  const config = {
    type: "image", // "blur"
    imageUrl: "https://cdn.videosdk.live/virtual-background/cloud.jpeg",
    // blurIntensity: BlurIntensity.mid, // "low" | "mid" | "high" (only applicable if type is "blur")
  };

  videoProcessor.updateProcessorConfig(config);
});

Stopping Virtual Background Processor

Additionally, you can use the stop() method on the processor to halt the background processing and replace the video stream with a new plain video stream.

// Import package
import { VirtualBackgroundProcessor } from "@videosdk.live/videosdk-virtual-background-web";
let liveStream;

// Initialize liveStream
liveStream = VideoSDK.initMeeting({
  // ...
});

// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

const stopVirtualBackgroundBtn = document.getElementById(
  "stopVirtualBackgroundBtn",
);
stopVirtualBackgroundBtn.addEventListener("click", async () => {
  videoProcessor.stop();

  // Pass webcam MediaStream in VideoSDK `changeWebcam` method
  const stream = await VideoSDK.createCameraVideoTrack({});
  liveStream?.changeWebcam(stream);
});

Disposing Virtual Background Processor

Use dispose() when you are completely done with the processor. This performs a full cleanup and releases all underlying GPU/WASM resources. After calling dispose(), the processor instance cannot be reused.

import { VirtualBackgroundProcessor } from "@videosdk.live/videosdk-virtual-background-web";
let liveStream;

// Initialize liveStream
liveStream = VideoSDK.initMeeting({
  // ...
});

// Instantiate VirtualBackgroundProcessor Class
const videoProcessor = new VirtualBackgroundProcessor();

const disposeVirtualBackgroundBtn = document.getElementById(
  "disposeVirtualBackgroundBtn",
);

disposeVirtualBackgroundBtn.addEventListener("click", () => {
  videoProcessor.dispose();
});

Extras

You can also pass the processed stream during initialization of the liveStream.

// Import package
import { VirtualBackgroundProcessor, VBModel, BlurIntensity } from "@videosdk.live/videosdk-virtual-background-web";
const videoProcessor = new VirtualBackgroundProcessor();

const stream = await VideoSDK.createCameraVideoTrack({});
if (!videoProcessor.ready) {
  await videoProcessor.init(VBModel.landscape); // or "VBModel.multiClass" | "VBModel.meet"
}
const processedStream = await videoProcessor.start(stream, {
  type: "image", // "blur"
  imageUrl: `https://cdn.videosdk.live/virtual-background/cloud.jpeg`,
  // blurIntensity: BlurIntensity.mid, // "low" | "mid" | "high" (only applicable if type is "blur")
});

let liveStream;
liveStream = VideoSDK.initMeeting({
  meetingId: "liveStreamId",
  micEnabled: true,
  webcamEnabled: true,
  name: "TestUser",
  customCameraVideoTrack: processedStream, // Pass processed MediaStream in VideoSDK
});

API Reference

The API references for all the methods utilized in this guide are provided below.

Got a Question? Ask us on discord

Install VideoSDK Virtual Background package​

Instantiate Virtual Background Processor​

Using Virtual Background​

Initializing​

Starting the Processor​

Passing Processed Stream to VideoSDK​

Updating Video Processor Configuration​

Stopping Virtual Background Processor​

Disposing Virtual Background Processor​

Extras​

API Reference​