Version: 0.x.x

Realtime Translation - React

Realtime translation allows you to convert spoken audio into translated text instantly during a live meeting session. This guide explains how to use the startTranslation(), stopTranslation(), and changeTranslationLanguage() methods of the useTranslation hook to manage translation in real time in your application.

You can define a participant's translation language in the MeetingProvider config, or update it at any point during the session with the changeTranslationLanguage() method.

note

The translation language is always a language code, both in the translationLanguage config of MeetingProvider and in changeTranslationLanguage(). Only the codes listed in Supported Languages return translated text. If you pass an unsupported code, no translation output is received.

note

Realtime translation requires @videosdk.live/react-sdk 0.4.6 or later. The speakingLanguage config requires 0.4.7 or later.

Integrating Realtime Translation Feature

Start Translation: You start realtime translation with the startTranslation() method. The onTranslationStateChanged callback then reports TRANSLATION_STARTING, followed by TRANSLATION_STARTED once translation is running.
Translation Data: As translation progresses, the onTranslationText callback receives the translated text, the participant who spoke it, the timestamp, and the language.
Change Translation Language: A participant can switch to another language while translation is running with changeTranslationLanguage(). The room then broadcasts a translation-language-changed event carrying the participant whose language changed and the language they selected.
Stop Translation: When you stop translation with stopTranslation(), the onTranslationStateChanged callback reports TRANSLATION_STOPPING, followed by TRANSLATION_STOPPED.

Step 1: Configure the translation languages

You can set the participant's translation language and speaking language in the MeetingProvider config.

import { MeetingProvider } from "@videosdk.live/react-sdk";

<MeetingProvider
  config={{
    meetingId: "abcd-efgh-ijkl",
    name: "John Doe",
    micEnabled: true,
    webcamEnabled: true,
    translationLanguage: "en",
    speakingLanguage: "en",
  }}
  token={token}
>
  <MeetingView />
</MeetingProvider>;

translationLanguage: The language the participant receives translated text in. The onTranslationText callback fires only when the text is in this language, so a participant who does not set translationLanguage receives no translated text until they call changeTranslationLanguage().
speakingLanguage: Fixes the language the participant speaks in for the whole session. When it is not set, the participant's translationLanguage doubles as their spoken language. Refer to Speaking Language and Translation Language for how the two interact.

Step 2: Start realtime translation

Initiate the realtime translation using the startTranslation() method of the useTranslation hook.

import { useTranslation } from "@videosdk.live/react-sdk";

const { startTranslation } = useTranslation();

// Starts realtime translation
try {
  await startTranslation();
} catch (err) {
  console.error("startTranslation failed:", err);
}

Step 3: Change the translation language

Switch to another language while translation is running using the changeTranslationLanguage() method. The method takes the language code to switch to.

const { changeTranslationLanguage } = useTranslation();

// Changes the translation language to Spanish
try {
  await changeTranslationLanguage("es");
} catch (err) {
  console.error("changeTranslationLanguage failed:", err);
}

Step 4: Stop realtime translation

Terminate the realtime translation using the stopTranslation() method.

const { stopTranslation } = useTranslation();

// Stops realtime translation
try {
  await stopTranslation();
} catch (err) {
  console.error("stopTranslation failed:", err);
}

note

Each of these three methods returns a Promise that rejects when the request fails. The same error also reaches the onError callback of useMeeting().

Step 5: Listen for the translation events

Here, you configure the callback methods that useTranslation accepts for the translation events.

import { Constants, useTranslation } from "@videosdk.live/react-sdk";

function onTranslationStateChanged(data) {
  const { status } = data;

  if (status === Constants.translationEvents.TRANSLATION_STARTING) {
    console.log("Realtime Translation is starting");
  } else if (status === Constants.translationEvents.TRANSLATION_STARTED) {
    console.log("Realtime Translation is started");
  } else if (status === Constants.translationEvents.TRANSLATION_STOPPING) {
    console.log("Realtime Translation is stopping");
  } else if (status === Constants.translationEvents.TRANSLATION_STOPPED) {
    console.log("Realtime Translation is stopped");
  }
}

function onTranslationText(data) {
  const { participantId, participantName, text, timestamp, type, language } =
    data;
  console.log(`${participantName} (${language}): ${text} ${timestamp}`);
}

function onTranslationLanguageChanged(data) {
  const { peerId, language } = data;
  console.log(`${peerId} switched translation language to ${language}`);
}

const { startTranslation, changeTranslationLanguage, stopTranslation } =
  useTranslation({
    onTranslationStateChanged,
    onTranslationText,
    onTranslationLanguageChanged,
  });

onTranslationStateChanged receives the current status of translation, which is one of the translationEvents constants shown above. The same value is available at any time on the translationState property returned by the useMeeting hook.
onTranslationText receives the translated text, the participantId and participantName of the speaker, the timestamp at which the text was spoken, the language the text was translated into, and a type of either full or partial. It fires only for text in the receiving participant's own translation language.
onTranslationLanguageChanged receives the peerId of the participant who changed their language and the language they switched to. It fires for every participant in the meeting, not only the one who made the change.

Example

The following React code snippet allows you to start, stop, and change the language of realtime translation with just a click.

import { Constants, useTranslation } from "@videosdk.live/react-sdk";
import { useState } from "react";

const TranslationView = () => {
  const [isTranslationOn, setIsTranslationOn] = useState(false);
  const [messages, setMessages] = useState([]);

  // Callback for translation state changed
  function onTranslationStateChanged(data) {
    const { status } = data;

    // Check for starting status
    if (status === Constants.translationEvents.TRANSLATION_STARTING) {
      console.log("Realtime Translation is starting");
    }
    // Check for started status
    else if (status === Constants.translationEvents.TRANSLATION_STARTED) {
      console.log("Realtime Translation is started");
      setIsTranslationOn(true);
    }
    // Check for stopping status
    else if (status === Constants.translationEvents.TRANSLATION_STOPPING) {
      console.log("Realtime Translation is stopping");
    }
    // Check for stopped status
    else if (status === Constants.translationEvents.TRANSLATION_STOPPED) {
      console.log("Realtime Translation is stopped");
      setIsTranslationOn(false);
    }
  }

  // Callback for translation text
  function onTranslationText(data) {
    const { participantName, text, language } = data;
    setMessages((prev) => [...prev, `${participantName} (${language}): ${text}`]);
  }

  // Callback for translation language changed
  function onTranslationLanguageChanged(data) {
    const { peerId, language } = data;
    console.log(`${peerId} switched translation language to ${language}`);
  }

  // Getting the methods from the useTranslation hook
  const { startTranslation, changeTranslationLanguage, stopTranslation } =
    useTranslation({
      onTranslationStateChanged,
      onTranslationText,
      onTranslationLanguageChanged,
    });

  const handleStartTranslation = async () => {
    try {
      await startTranslation();
    } catch (err) {
      console.error("startTranslation failed:", err);
    }
  };

  const handleStopTranslation = async () => {
    try {
      await stopTranslation();
    } catch (err) {
      console.error("stopTranslation failed:", err);
    }
  };

  const handleChangeLanguage = async (event) => {
    try {
      await changeTranslationLanguage(event.target.value);
    } catch (err) {
      console.error("changeTranslationLanguage failed:", err);
    }
  };

  return (
    <div>
      <button onClick={handleStartTranslation} disabled={isTranslationOn}>
        Start Translation
      </button>
      <button onClick={handleStopTranslation} disabled={!isTranslationOn}>
        Stop Translation
      </button>

      <select onChange={handleChangeLanguage} disabled={!isTranslationOn}>
        <option value="en">English</option>
        <option value="es">Spanish</option>
        <option value="de">German</option>
        <option value="fr">French</option>
        <option value="hi">Hindi</option>
      </select>

      {messages.map((message, index) => (
        <p key={index}>{message}</p>
      ))}
    </div>
  );
};

Speaking Language and Translation Language

The speakingLanguage config defines the language a participant speaks in, while translationLanguage defines the language they want to receive translated text in. The two examples below show how translation behaves with and without a speaking language.

Example 1: Without speakingLanguage

Both participants set only their translation language, so each participant's translation language doubles as their spoken language.

// Participant 1 (Spanish)
<MeetingProvider
  config={{
    // ...
    translationLanguage: "es", // Spanish
  }}
  token={token}
>
  <MeetingView />
</MeetingProvider>;

// Participant 2 (German)
<MeetingProvider
  config={{
    // ...
    translationLanguage: "de", // German
  }}
  token={token}
>
  <MeetingView />
</MeetingProvider>;

Participant 1 selected Spanish (es) as their translation language, so Spanish is also their spoken language. Participant 2 selected German (de), so German is also their spoken language.

Speaker	Speaking Language (Input)	Receiver	Translated Output
Participant 1	Spanish (`es`)	Participant 2	German (`de`)
Participant 2	German (`de`)	Participant 1	Spanish (`es`)

In this case each participant must speak in their own translation language, because no speaking language is defined.

Example 2: With speakingLanguage on one participant

Here only Participant 1 sets a speakingLanguage. That fixes the input language for their audio, while Participant 2's translation language still doubles as their spoken language.

// Participant 1 (speaks English, receives Spanish)
<MeetingProvider
  config={{
    // ...
    translationLanguage: "es", // Spanish (output)
    speakingLanguage: "en", // English (input)
  }}
  token={token}
>
  <MeetingView />
</MeetingProvider>;

// Participant 2 (no speakingLanguage, receives German)
<MeetingProvider
  config={{
    // ...
    translationLanguage: "de", // German (output)
  }}
  token={token}
>
  <MeetingView />
</MeetingProvider>;

Participant 1 speaks in English (en) and receives translations in Spanish (es). Participant 2 has no speakingLanguage, so German (de) is both the language they speak and the language they receive translations in.

Speaker	Speaking Language (Input)	Receiver	Translated Output
Participant 1	English (`en`)	Participant 2	German (`de`)
Participant 2	German (`de`)	Participant 1	Spanish (`es`)

Participant 1 speaks English and Participant 2 receives it translated into German, while Participant 1 receives Participant 2's German speech translated into Spanish.

Key takeaways

When speakingLanguage is not set, the participant's translation language doubles as their speaking language.
When only one participant sets a speakingLanguage, that participant's input language is fixed, while the others keep using their translation language as their input language.
Setting speakingLanguage gives you better control and consistency, especially in multilingual meetings.

Supported Languages

These are the supported language codes:

Multilingual (Spanish + English): multi
Bulgarian: bg
Catalan: ca
Chinese (Mandarin, Simplified): zh, zh-CN, zh-Hans
Chinese (Mandarin, Traditional): zh-TW, zh-Hant
Chinese (Cantonese, Traditional): zh-HK
Czech: cs
Danish: da, da-DK
Dutch: nl
English: en, en-US, en-AU, en-GB, en-NZ, en-IN
Estonian: et
Finnish: fi
Flemish: nl-BE
French: fr, fr-CA
German: de
German (Switzerland): de-CH
Greek: el
Hindi: hi
Hungarian: hu
Indonesian: id
Italian: it
Japanese: ja
Korean: ko, ko-KR
Latvian: lv
Lithuanian: lt
Malay: ms
Norwegian: no
Polish: pl
Portuguese: pt, pt-BR, pt-PT
Romanian: ro
Russian: ru
Slovak: sk
Spanish: es, es-419
Swedish: sv, sv-SE
Thai: th, th-TH
Turkish: tr
Ukrainian: uk
Vietnamese: vi

Got a Question? Ask us on discord

Integrating Realtime Translation Feature​

Step 1: Configure the translation languages​

Step 2: Start realtime translation​

Step 3: Change the translation language​

Step 4: Stop realtime translation​

Step 5: Listen for the translation events​

Example​

Speaking Language and Translation Language​

Example 1: Without speakingLanguage​

Example 2: With speakingLanguage on one participant​

Key takeaways​

Supported Languages​

Integrating Realtime Translation Feature

Step 1: Configure the translation languages

Step 2: Start realtime translation

Step 3: Change the translation language

Step 4: Stop realtime translation

Step 5: Listen for the translation events

Example

Speaking Language and Translation Language

Example 1: Without speakingLanguage

Example 2: With speakingLanguage on one participant

Key takeaways

Supported Languages