Realtime Translation - Javascript

Realtime translation allows you to convert spoken audio into translated text instantly during a live meeting session. This guide explains how to use the startTranslation(), stopTranslation(), and changeTranslationLanguage() methods to manage translation in real time in your application.

You can define a participant's translation language when initializing the meeting with initMeeting(), or update it at any point during the session with the changeTranslationLanguage() method.

note

The translation language is always a language code, both in the translationLanguage parameter of initMeeting() and in changeTranslationLanguage(). Only the codes listed in Supported Languages return translated text. If you pass an unsupported code, no translation output is received.

note

Realtime translation requires @videosdk.live/js-sdk 0.3.9 or later. The speakingLanguage parameter requires 0.3.10 or later.

Integrating Realtime Translation Feature

1. Start Translation: You start realtime translation with the startTranslation() method. The translation-state-changed event then reports TRANSLATION_STARTING, followed by TRANSLATION_STARTED once translation is running.

2. Translation Data: As translation progresses, you receive the translation-text event with the translated text, the participant who spoke it, the timestamp, and the language.

3. Change Translation Language: A participant can switch to another language while translation is running with changeTranslationLanguage(). The room then broadcasts a translation-language-changed event carrying the participant whose language changed and the language they selected.

4. Stop Translation: When you stop translation with stopTranslation(), the translation-state-changed event reports TRANSLATION_STOPPING, followed by TRANSLATION_STOPPED.

Step 1: Configure the translation languages

• You can set the participant's translation language and speaking language while initializing the meeting with initMeeting().

// Initialize Meeting
const meeting = VideoSDK.initMeeting({
  meetingId: "abcd-efgh-ijkl",
  name: "John Doe",
  micEnabled: true,
  webcamEnabled: true,
  translationLanguage: "en",
  speakingLanguage: "en",
});

• translationLanguage: The language the participant receives translated text in. A translation-text event arrives only when the text is in this language, so a participant who does not set translationLanguage receives no translated text until they call changeTranslationLanguage().

• speakingLanguage: Fixes the language the participant speaks in for the whole session. When it is not set, the participant's translationLanguage doubles as their spoken language. Refer to Speaking Language and Translation Language for how the two interact.

Step 2: Start realtime translation

• Initiate the realtime translation using the startTranslation() method.

// Starts realtime translation
try {
  await meeting.startTranslation();
} catch (err) {
  console.error("startTranslation failed:", err);
}

Step 3: Change the translation language

• Switch to another language while translation is running using the changeTranslationLanguage() method. The method takes the language code to switch to.

// Changes the translation language to Spanish
try {
  await meeting.changeTranslationLanguage("es");
} catch (err) {
  console.error("changeTranslationLanguage failed:", err);
}

Step 4: Stop realtime translation

• Terminate the realtime translation using the stopTranslation() method.

// Stops realtime translation
try {
  await meeting.stopTranslation();
} catch (err) {
  console.error("stopTranslation failed:", err);
}

note

Each of these three methods returns a Promise that rejects when the request fails.

Step 5: Listen for the translation events

• Here, you configure the callback methods for the translation events.

import { VideoSDK } from "@videosdk.live/js-sdk";

const Constants = VideoSDK.Constants;

// Listen for translation state changed event
meeting.on("translation-state-changed", (data) => {
  let { status } = data;

  if (status === Constants.translationEvents.TRANSLATION_STARTING) {
    console.log("Realtime Translation is starting");
  } else if (status === Constants.translationEvents.TRANSLATION_STARTED) {
    console.log("Realtime Translation is started");
  } else if (status === Constants.translationEvents.TRANSLATION_STOPPING) {
    console.log("Realtime Translation is stopping");
  } else if (status === Constants.translationEvents.TRANSLATION_STOPPED) {
    console.log("Realtime Translation is stopped");
  }
});

// Listen for translation text event
meeting.on("translation-text", (data) => {
  let { participantId, participantName, text, timestamp, type, language } = data;
  console.log(`${participantName} (${language}): ${text} ${timestamp}`);
});

// Listen for translation language changed event
meeting.on("translation-language-changed", (data) => {
  let { peerId, language } = data;
  console.log(`${peerId} switched translation language to ${language}`);
});

• translation-state-changed carries the current status of translation, which is one of the translationEvents constants shown above. The same value is available at any time on the meeting.translationState property.

• translation-text carries the translated text, the participantId and participantName of the speaker, the timestamp at which the text was spoken, the language the text was translated into, and a type of either full or partial. It fires only for text in the receiving participant's own translation language.

• translation-language-changed carries the peerId of the participant who changed their language and the language they switched to. It reaches every participant in the meeting, not only the one who made the change.

Example

• The following JavaScript code snippet allows you to start, stop, and change the language of realtime translation with just a click.

// Meeting object
let meeting;

// Initialize Meeting
meeting = VideoSDK.initMeeting({
  // ...
  translationLanguage: "en",
  speakingLanguage: "en",
});

const Constants = VideoSDK.Constants;

// Get start button element
const startTranslationBtn = document.getElementById("startTranslationBtn");

// Get stop button element
const stopTranslationBtn = document.getElementById("stopTranslationBtn");

// Get language selector element
const translationLanguageSelect = document.getElementById(
  "translationLanguageSelect"
);

// Listen for translation state changed event
meeting?.on("translation-state-changed", (data) => {
  const { status } = data;

  // Check for starting status
  if (status === Constants.translationEvents.TRANSLATION_STARTING) {
    console.log("Realtime Translation is starting");
  }
  // Check for started status
  else if (status === Constants.translationEvents.TRANSLATION_STARTED) {
    console.log("Realtime Translation is started");
  }
  // Check for stopping status
  else if (status === Constants.translationEvents.TRANSLATION_STOPPING) {
    console.log("Realtime Translation is stopping");
  }
  // Check for stopped status
  else if (status === Constants.translationEvents.TRANSLATION_STOPPED) {
    console.log("Realtime Translation is stopped");
  }
});

// Listen for translation text event
meeting?.on("translation-text", (data) => {
  // Destructuring data
  const { participantId, participantName, text, timestamp, type, language } =
    data;
  console.log(`${participantName} (${language}): ${text} ${timestamp}`);
});

// Listen for translation language changed event
meeting?.on("translation-language-changed", (data) => {
  // Destructuring data
  const { peerId, language } = data;
  console.log(`${peerId} switched translation language to ${language}`);
});

// Listen for click event
startTranslationBtn.addEventListener("click", async () => {
  // Start realtime translation
  try {
    await meeting?.startTranslation();
  } catch (err) {
    console.error("startTranslation failed:", err);
  }
});

// Listen for click event
stopTranslationBtn.addEventListener("click", async () => {
  // Stop realtime translation
  try {
    await meeting?.stopTranslation();
  } catch (err) {
    console.error("stopTranslation failed:", err);
  }
});

// Listen for change event
translationLanguageSelect.addEventListener("change", async (event) => {
  // Change the translation language
  try {
    await meeting?.changeTranslationLanguage(event.target.value);
  } catch (err) {
    console.error("changeTranslationLanguage failed:", err);
  }
});

Speaking Language and Translation Language

The speakingLanguage parameter defines the language a participant speaks in, while translationLanguage defines the language they want to receive translated text in. The two examples below show how translation behaves with and without a speaking language.

Example 1: Without speakingLanguage

Both participants set only their translation language, so each participant's translation language doubles as their spoken language.

// Participant 1 (Spanish)
const meeting = VideoSDK.initMeeting({
  // ...
  translationLanguage: "es", // Spanish
});

// Participant 2 (German)
const meeting = VideoSDK.initMeeting({
  // ...
  translationLanguage: "de", // German
});

Participant 1 selected Spanish (es) as their translation language, so Spanish is also their spoken language. Participant 2 selected German (de), so German is also their spoken language.

Speaker	Speaking Language (Input)	Receiver	Translated Output
Participant 1	Spanish (es)	Participant 2	German (de)
Participant 2	German (de)	Participant 1	Spanish (es)

In this case each participant must speak in their own translation language, because no speaking language is defined.

Example 2: With speakingLanguage on one participant

Here only Participant 1 sets a speakingLanguage. That fixes the input language for their audio, while Participant 2's translation language still doubles as their spoken language.

// Participant 1 (speaks English, receives Spanish)
const meeting = VideoSDK.initMeeting({
  // ...
  translationLanguage: "es", // Spanish (output)
  speakingLanguage: "en", // English (input)
});

// Participant 2 (no speakingLanguage, receives German)
const meeting = VideoSDK.initMeeting({
  // ...
  translationLanguage: "de", // German (output)
});

Participant 1 speaks in English (en) and receives translations in Spanish (es). Participant 2 has no speakingLanguage, so German (de) is both the language they speak and the language they receive translations in.

Speaker	Speaking Language (Input)	Receiver	Translated Output
Participant 1	English (en)	Participant 2	German (de)
Participant 2	German (de)	Participant 1	Spanish (es)

Participant 1 speaks English and Participant 2 receives it translated into German, while Participant 1 receives Participant 2's German speech translated into Spanish.

Key takeaways

• When speakingLanguage is not set, the participant's translation language doubles as their speaking language.
• When only one participant sets a speakingLanguage, that participant's input language is fixed, while the others keep using their translation language as their input language.
• Setting speakingLanguage gives you better control and consistency, especially in multilingual meetings.

Supported Languages

These are the supported language codes:

• Multilingual (Spanish + English): multi
• Bulgarian: bg
• Catalan: ca
• Chinese (Mandarin, Simplified): zh, zh-CN, zh-Hans
• Chinese (Mandarin, Traditional): zh-TW, zh-Hant
• Chinese (Cantonese, Traditional): zh-HK
• Czech: cs
• Danish: da, da-DK
• Dutch: nl
• English: en, en-US, en-AU, en-GB, en-NZ, en-IN
• Estonian: et
• Finnish: fi
• Flemish: nl-BE
• French: fr, fr-CA
• German: de
• German (Switzerland): de-CH
• Greek: el
• Hindi: hi
• Hungarian: hu
• Indonesian: id
• Italian: it
• Japanese: ja
• Korean: ko, ko-KR
• Latvian: lv
• Lithuanian: lt
• Malay: ms
• Norwegian: no
• Polish: pl
• Portuguese: pt, pt-BR, pt-PT
• Romanian: ro
• Russian: ru
• Slovak: sk
• Spanish: es, es-419
• Swedish: sv, sv-SE
• Thai: th, th-TH
• Turkish: tr
• Ukrainian: uk
• Vietnamese: vi