Skip to main content
Version: 1.x.x

Raw Tracks - React

Each participant can send a webcam, a mic, a screen share and screen-share audio. Raw tracks let a participant send more audio or video tracks at the same time, for example a second camera or a PDF drawn on a canvas.

Each raw track has a name that every participant sees, and each viewer can consume, pause, render and set the quality of it independently.

note

This page uses the following asynchronous methods. Refer to their API reference for the errors each Promise may reject with, and handle these rejections appropriately based on your use case.

Send a raw track​

Call publishTrack() from useMeeting with a name and any live MediaStreamTrack. It resolves with the local Stream.

Parameters​

  • name
    • type: string
    • required: true
    • 1 to 64 characters, unique per participant across audio and video. Viewers use it to tell one raw track from another.
  • track
    • type: MediaStreamTrack
    • required: true
    • A live audio or video track. Your app owns it: the SDK never stops it.
  • simulcast
    • type: boolean
    • required: false
    • default: true
    • Video only. Sends several quality layers so each viewer can receive the one that suits its network and tile size.
  • maxLayer
    • type: number
    • required: false
    • Video only. The highest number of simulcast layers to send, from 1 to 3. 1 sends a single layer.

Example: a canvas​

import { useRef, useState } from "react";
import { useMeeting } from "@videosdk.live/react-sdk";

function ShareCanvasButton({ canvasRef }) {
const { publishTrack, unpublishTrack } = useMeeting();
const trackRef = useRef(null);
const [sharing, setSharing] = useState(false);

const toggle = async () => {
try {
if (sharing) {
await unpublishTrack("pdf");
// The SDK does not stop the track; release it when you are done.
trackRef.current.stop();
setSharing(false);
return;
}

trackRef.current = canvasRef.current
.captureStream(15)
.getVideoTracks()[0];
await publishTrack({ name: "pdf", track: trackRef.current });
setSharing(true);
} catch (err) {
console.error("raw track failed:", err);
}
};

return <button onClick={toggle}>{sharing ? "Stop PDF" : "Share PDF"}</button>;
}

Example: a second camera​

import {
createCameraVideoTrack,
useMediaDevice,
useMeeting,
} from "@videosdk.live/react-sdk";

function ShareSecondCameraButton() {
const { getCameras } = useMediaDevice();
const { publishTrack } = useMeeting();

const share = async () => {
const cameras = await getCameras();

// Any MediaStream works; createCameraVideoTrack sets the resolution for you.
const cameraStream = await createCameraVideoTrack({
cameraId: cameras[1].deviceId,
encoderConfig: "h720p_w1280p",
});

await publishTrack({
name: "cam2",
track: cameraStream.getVideoTracks()[0],
});
};

return <button onClick={share}>Share second camera</button>;
}
info
  • The track is sent with the size and frame rate it already has. Set them when you create the track.
  • If the connection drops, the SDK sends the raw track again after the meeting reconnects, as long as the track is still live.
  • Using a name that is already in use throws ERROR_RAW_TRACK_NAME_EXISTS instead of replacing the track. Unpublish the old one first.
  • unpublishTrack(name) stops sending the track but never stops the track itself. If the track ends on its own (for example the camera is unplugged), the SDK stops sending it.

Render raw tracks​

useParticipant returns rawTracks, an object keyed by name with one entry for every raw track the participant is sending, including ones this viewer has not consumed yet.

{
pdf: { name: "pdf", kind: "rawVideo", stream: Stream | null },
cam2: { name: "cam2", kind: "rawVideo", stream: Stream | null },
}

stream is null until the track is consumed. Render each entry with VideoPlayer or AudioPlayer, passing the track's name:

import { useParticipant, VideoPlayer, AudioPlayer } from "@videosdk.live/react-sdk";

function RawTracks({ participantId }) {
const { rawTracks } = useParticipant(participantId);

return Object.values(rawTracks).map(({ name, kind }) =>
kind === "rawVideo" ? (
<VideoPlayer
key={name}
participantId={participantId}
type="rawVideo"
rawTrackName={name}
/>
) : (
<AudioPlayer
key={name}
participantId={participantId}
type="rawAudio"
rawTrackName={name}
/>
)
);
}

VideoPlayer pauses a raw video while its tile is off screen and sets its quality from the tile size, the same as the webcam. Only that raw track is affected.

Apps that treat every video stream as the webcam keep working: raw tracks use the kinds rawVideo and rawAudio, never video or audio, so webcamStream and micStream never change.

Events​

Raw tracks also arrive on the existing onStreamEnabled / onStreamDisabled callbacks, with stream.kind set to rawVideo or rawAudio and stream.name set to the sender's name. onMediaStatusChanged carries name for raw tracks.

useParticipant(participantId, {
onStreamEnabled: (stream) => {
if (stream.kind === "rawVideo") {
console.log("raw video started:", stream.name);
}
},
onMediaStatusChanged: ({ kind, newStatus, name }) => {
if (kind === "rawVideo" || kind === "rawAudio") {
console.log(`${name} is now ${newStatus ? "on" : "off"}`);
}
},
});

Start consuming raw tracks manually​

With autoConsume on (the default), raw tracks arrive automatically. If you set autoConsume: false in the MeetingProvider config, consume each one by name:

const { rawTracks, startConsumingRawTrack, stopConsumingRawTrack } =
useParticipant(participantId);

// subscribe to the "pdf" track listed in rawTracks
await startConsumingRawTrack("pdf");

// later
await stopConsumingRawTrack("pdf");

You can call startConsumingRawTrack() before the sender has started the track; it arrives as soon as it starts.

Pause, resume and quality​

Each viewer controls its own copy of a raw track. Nothing here affects other viewers or the participant's other tracks.

const { rawTracks } = useParticipant(participantId);
const pdfStream = rawTracks.pdf?.stream;

await pdfStream?.pause();
await pdfStream?.resume();

// remote video only
await pdfStream?.setQuality("low"); // "low" | "med" | "high"

useStream also returns name and setQuality for any stream. To pause or resume every raw video you receive, use pauseAllStreams from useMeeting:

const { pauseAllStreams, resumeAllStreams } = useMeeting();

await pauseAllStreams("rawVideo");
await resumeAllStreams("rawVideo");

Errors​

CodeNameWhen
3080ERROR_INVALID_RAW_TRACKThe name is missing or invalid, or the track is not a live audio or video track.
3081ERROR_RAW_TRACK_NAME_EXISTSA raw track with this name is already published.
3082ERROR_RAW_TRACK_NOT_FOUNDunpublishTrack() was called with an unknown name.
3083ERROR_RAW_TRACK_PUBLISH_FAILEDThe track could not be sent.
4090START_CONSUMING_RAW_TRACK_FAILEDThe server could not consume the raw track.
4091STOP_CONSUMING_RAW_TRACK_FAILEDThe server could not stop consuming the raw track.

API Reference​

The API references for all the methods and events used in this guide are provided below.

Got a Question? Ask us on discord