Raw Tracks - JavaScript

Each participant can send a webcam, a mic, a screen share and screen-share audio. Raw tracks let a participant send more audio or video tracks at the same time, for example a second camera or a PDF drawn on a canvas.

Each raw track has a name that every participant sees, and each viewer can consume, pause, render and set the quality of it independently.

note

This page uses the following asynchronous methods. Refer to their API reference for the errors each Promise may reject with, and handle these rejections appropriately based on your use case.

• publishTrack()
• unpublishTrack()
• startConsumingRawTrack()
• stopConsumingRawTrack()
• setQuality()

Send a raw track

Call publishTrack() with a name and any live MediaStreamTrack. It resolves with the local Stream.

Parameters

• name
  • type: string
  • required: true
  • 1 to 64 characters, unique per participant across audio and video. Viewers use it to tell one raw track from another.
• track
  • type: MediaStreamTrack
  • required: true
  • A live audio or video track. Your app owns it: the SDK never stops it.
• simulcast
  • type: boolean
  • required: false
  • default: true
  • Video only. Sends several quality layers so each viewer can receive the one that suits its network and tile size.
• maxLayer
  • type: number
  • required: false
  • Video only. The highest number of simulcast layers to send, from 1 to 3. 1 sends a single layer.

Example: a canvas

const canvas = document.getElementById("pdfCanvas");
const pdfTrack = canvas.captureStream(15).getVideoTracks()[0];

try {
  const stream = await meeting.publishTrack({
    name: "pdf",
    track: pdfTrack,
  });
  console.log(stream.kind, stream.name); // "rawVideo", "pdf"
} catch (err) {
  console.error("publishTrack failed:", err);
}

Example: a second camera

const cameras = await VideoSDK.getCameras();

// Any MediaStream works; createCameraVideoTrack sets the resolution for you.
const cameraStream = await VideoSDK.createCameraVideoTrack({
  cameraId: cameras[1].deviceId,
  encoderConfig: "h720p_w1280p",
});

await meeting.publishTrack({
  name: "cam2",
  track: cameraStream.getVideoTracks()[0],
});

info

• The track is sent with the size and frame rate it already has. Set them when you create the track.
• If the connection drops, the SDK sends the raw track again after the meeting reconnects, as long as the track is still live.
• Using a name that is already in use throws ERROR_RAW_TRACK_NAME_EXISTS instead of replacing the track. Unpublish the old one first.

Stop a raw track

unpublishTrack(name) stops sending the track. The track itself keeps running, so stop it yourself when you no longer need it.

await meeting.unpublishTrack("pdf");

// The SDK does not stop the track; release it when you are done with it.
pdfTrack.stop();

If the track ends on its own (for example the camera is unplugged), the SDK stops sending it.

Receive raw tracks

Raw tracks arrive on the participant's existing stream-enabled and stream-disabled events, with kind set to rawVideo or rawAudio and name set to the sender's name. Apps that treat every video stream as the webcam keep working, because raw tracks never use the video or audio kinds.

participant.on("stream-enabled", (stream) => {
  if (stream.kind === "rawVideo") {
    const tile = participant.renderVideo({
      type: "rawVideo",
      rawTrackName: stream.name,
    });
    document.getElementById("rawTracks").appendChild(tile);
  }

  if (stream.kind === "rawAudio") {
    const audio = participant.renderAudio({
      type: "rawAudio",
      rawTrackName: stream.name,
    });
    document.body.appendChild(audio);
  }
});

participant.on("stream-disabled", (stream) => {
  if (stream.kind === "rawVideo") {
    // remove the tile you added for stream.name
  }
});

renderVideo() keeps one element per raw track name. While its tile is off screen, the raw video is paused for this viewer, and its quality follows the tile size, the same as the webcam.

List a participant's raw tracks

participant.rawTracks is a Map of every raw track the participant is sending, keyed by name, including ones this viewer has not consumed yet. It updates with the media-status-changed event, which carries name for raw tracks.

participant.on("media-status-changed", ({ kind, newStatus, name }) => {
  if (kind === "rawVideo" || kind === "rawAudio") {
    console.log(`${name} is now ${newStatus ? "on" : "off"}`);
    console.log([...participant.rawTracks.keys()]);
  }
});

Start consuming raw tracks manually

With autoConsume on (the default), raw tracks arrive automatically. If you join with autoConsume: false, consume each one by name:

await participant.startConsumingRawTrack("pdf");

// later
await participant.stopConsumingRawTrack("pdf");

You can call startConsumingRawTrack() before the sender has started the track; it arrives as soon as it starts.

Pause, resume and quality

Each viewer controls its own copy of a raw track. Nothing here affects other viewers or the participant's other tracks.

const stream = participant.getRawTrackStream("pdf");

await stream.pause();
await stream.resume();

// remote video only
await stream.setQuality("low"); // "low" | "med" | "high"

// pause or resume every raw video track you receive
await meeting.pauseAllStreams("rawVideo");
await meeting.resumeAllStreams("rawVideo");

To render at a fixed quality instead of following the tile size, pass maxQuality to renderVideo(). It applies even if the stream arrives after the tile is rendered.

participant.renderVideo({
  type: "rawVideo",
  rawTrackName: "pdf",
  maxQuality: "high",
});

Errors

Code	Name	When
3080	ERROR_INVALID_RAW_TRACK	The name is missing or invalid, or the track is not a live audio or video track.
3081	ERROR_RAW_TRACK_NAME_EXISTS	A raw track with this name is already published.
3082	ERROR_RAW_TRACK_NOT_FOUND	unpublishTrack() was called with an unknown name.
3083	ERROR_RAW_TRACK_PUBLISH_FAILED	The track could not be sent.
4090	START_CONSUMING_RAW_TRACK_FAILED	The server could not consume the raw track.
4091	STOP_CONSUMING_RAW_TRACK_FAILED	The server could not stop consuming the raw track.

API Reference

The API references for all the methods and events used in this guide are provided below.

• publishTrack()
• unpublishTrack()
• rawTracks
• startConsumingRawTrack()
• stopConsumingRawTrack()
• renderVideo()
• stream-enabled
• Stream