Live Translation for Livestreams - React Native

Realtime translation turns what participants say into text in other languages, while the room runs. Each participant receives the text in the language they choose. This guide uses the startTranslation(), changeTranslationLanguage() and stopTranslation() methods of the useTranslation hook.

note

Languages are set as language codes, both in the config of RoomProvider and in changeTranslationLanguage(). Only the codes listed in Supported Languages return translated text. The SDK does not check the code: with a code that is not supported, no translated text arrives.

note

This page uses the following asynchronous methods. Refer to their API reference for the errors each Promise may reject with, and handle these rejections appropriately based on your use case.

• startTranslation()
• changeTranslationLanguage()
• stopTranslation()

How it works

1. Start translation: startTranslation() starts translation for the whole room. onTranslationStateChanged reports TRANSLATION_STARTING, then TRANSLATION_STARTED.
2. Receive the text: onTranslationText delivers each translated line, in the language this participant receives.
3. Change the language: changeTranslationLanguage() changes the language this participant receives. Everyone else keeps their own language.
4. Stop translation: stopTranslation() stops translation for the whole room. onTranslationStateChanged reports TRANSLATION_STOPPING, then TRANSLATION_STOPPED.

Step 1: Set the languages

Set the participant's languages in the config of RoomProvider.

import { RoomProvider } from "@videosdk.live/react-native-sdk";

const App = () => {
  return (
    <RoomProvider
      config={{
        roomId: "abcd-efgh-ijkl",
        participantName: "John Doe",
        micEnabled: true,
        camEnabled: true,
        speakingLanguage: "en",
        translationLanguage: "es",
      }}
      token="YOUR_TOKEN"
    >
      <RoomView />
    </RoomProvider>
  );
};

• translationLanguage: The language this participant receives translated text in. Without it, the participant receives no translated text, even while translation runs, until they call changeTranslationLanguage().
• speakingLanguage: The language this participant speaks. You can set it only here. When it is not set, the participant's translationLanguage is also used as the language they speak. See Speaking Language and Translation Language.

Step 2: Listen for the translation events

Pass your callback functions to the useTranslation hook.

import { Constants, useTranslation } from "@videosdk.live/react-native-sdk";

function onTranslationStateChanged(data) {
  const { state } = data;

  if (state === Constants.translationEvents.TRANSLATION_STARTING) {
    console.log("Realtime Translation is starting");
  } else if (state === Constants.translationEvents.TRANSLATION_STARTED) {
    console.log("Realtime Translation is started");
  } else if (state === Constants.translationEvents.TRANSLATION_STOPPING) {
    console.log("Realtime Translation is stopping");
  } else if (state === Constants.translationEvents.TRANSLATION_STOPPED) {
    console.log("Realtime Translation is stopped");
  }
}

function onTranslationText(data) {
  const { participantName, text, language, type } = data;
  console.log(`${participantName} (${language}, ${type}): ${text}`);
}

function onTranslationLanguageChanged(data) {
  const { participantId, language } = data;
  console.log(`${participantId} now receives ${language}`);
}

const { startTranslation, changeTranslationLanguage, stopTranslation } =
  useTranslation({
    onTranslationStateChanged,
    onTranslationText,
    onTranslationLanguageChanged,
  });

• onTranslationStateChanged receives the state of translation, one of the Constants.translationEvents values shown above. The same value is available at any time as translationState from the useRoom hook.
• onTranslationText receives the translated text, the participantId and participantName of the speaker, the participantLanguage they spoke, the language of the text, a timestamp in milliseconds and a type of "PARTIAL" or "FULL" (Constants.TranslationType). Consecutive PARTIAL lines revise the same sentence, so replace the last line you show instead of adding a new one. A FULL line is final.
• onTranslationLanguageChanged receives the participantId of the participant who changed their language and the language they now receive. It fires for every participant in the room, not only for the one who made the change.

Step 3: Start realtime translation

Start translation with the startTranslation() method.

const { startTranslation } = useTranslation();

// Starts realtime translation
try {
  await startTranslation();
} catch (err) {
  console.error("startTranslation failed:", err);
}

Step 4: Change the translation language

Change the language this participant receives with the changeTranslationLanguage() method. The change takes effect when onTranslationLanguageChanged reports it.

const { changeTranslationLanguage } = useTranslation();

// Receives the translation in French from now on
try {
  await changeTranslationLanguage("fr");
} catch (err) {
  console.error("changeTranslationLanguage failed:", err);
}

Step 5: Stop realtime translation

Stop translation with the stopTranslation() method. It stops translation for every participant.

const { stopTranslation } = useTranslation();

// Stops realtime translation
try {
  await stopTranslation();
} catch (err) {
  console.error("stopTranslation failed:", err);
}

Example

The following component starts and stops translation, changes the language with a tap, and shows the final lines.

import { useState } from "react";
import { ScrollView, Text, TouchableOpacity, View } from "react-native";
import { Constants, useTranslation } from "@videosdk.live/react-native-sdk";

const LANGUAGES = ["en", "es", "de", "fr", "hi"];

const TranslationView = () => {
  const [isTranslationOn, setIsTranslationOn] = useState(false);
  const [lines, setLines] = useState([]);

  // Callback for translation state changed
  function onTranslationStateChanged(data) {
    const { state } = data;

    if (state === Constants.translationEvents.TRANSLATION_STARTED) {
      setIsTranslationOn(true);
    } else if (state === Constants.translationEvents.TRANSLATION_STOPPED) {
      setIsTranslationOn(false);
    }
  }

  // Callback for translation text: keep only the final lines
  function onTranslationText(data) {
    const { participantName, text, type } = data;

    if (type === Constants.TranslationType.FULL) {
      setLines((previousLines) => [...previousLines, `${participantName}: ${text}`]);
    }
  }

  const { startTranslation, changeTranslationLanguage, stopTranslation } =
    useTranslation({
      onTranslationStateChanged,
      onTranslationText,
    });

  const handleStartTranslation = async () => {
    try {
      await startTranslation();
    } catch (err) {
      console.error("startTranslation failed:", err);
    }
  };

  const handleStopTranslation = async () => {
    try {
      await stopTranslation();
    } catch (err) {
      console.error("stopTranslation failed:", err);
    }
  };

  const handleChangeLanguage = async (language) => {
    try {
      await changeTranslationLanguage(language);
    } catch (err) {
      console.error("changeTranslationLanguage failed:", err);
    }
  };

  return (
    <View style={{ flex: 1 }}>
      <TouchableOpacity onPress={handleStartTranslation} disabled={isTranslationOn}>
        <Text>Start Translation</Text>
      </TouchableOpacity>
      <TouchableOpacity onPress={handleStopTranslation} disabled={!isTranslationOn}>
        <Text>Stop Translation</Text>
      </TouchableOpacity>

      <View style={{ flexDirection: "row" }}>
        {LANGUAGES.map((language) => (
          <TouchableOpacity
            key={language}
            onPress={() => handleChangeLanguage(language)}
          >
            <Text>{language}</Text>
          </TouchableOpacity>
        ))}
      </View>

      <ScrollView>
        {lines.map((line, index) => (
          <Text key={index}>{line}</Text>
        ))}
      </ScrollView>
    </View>
  );
};

export default TranslationView;

Speaking Language and Translation Language

speakingLanguage is the language a participant speaks, and translationLanguage is the language they receive translated text in. The two examples below show how translation behaves with and without a speaking language.

Example 1: Without speakingLanguage

Both participants set only a translation language, so each participant's translation language is also the language they speak.

// Participant 1, in the config of RoomProvider
const participant1Languages = {
  translationLanguage: "es", // Spanish
};

// Participant 2, in the config of RoomProvider
const participant2Languages = {
  translationLanguage: "de", // German
};

Speaker	Speaking Language (Input)	Receiver	Translated Output
Participant 1	Spanish (es)	Participant 2	German (de)
Participant 2	German (de)	Participant 1	Spanish (es)

Each participant must speak in their own translation language, because no speaking language is set.

Example 2: With speakingLanguage on one participant

Only Participant 1 sets a speakingLanguage. It fixes the language of their speech, while Participant 2's translation language is still the language they speak.

// Participant 1 speaks English and receives Spanish
const participant1Languages = {
  translationLanguage: "es", // Spanish (output)
  speakingLanguage: "en", // English (input)
};

// Participant 2 has no speakingLanguage and receives German
const participant2Languages = {
  translationLanguage: "de", // German (output)
};

Speaker	Speaking Language (Input)	Receiver	Translated Output
Participant 1	English (en)	Participant 2	German (de)
Participant 2	German (de)	Participant 1	Spanish (es)

Key takeaways

• When speakingLanguage is not set, the participant's translation language is also the language they speak.
• When only one participant sets a speakingLanguage, only that participant's spoken language is fixed. The others keep speaking their translation language.
• Setting speakingLanguage gives you more control, especially in rooms with many languages.

Supported Languages

These are the supported language codes:

• Multilingual (Spanish + English): multi
• Bulgarian: bg
• Catalan: ca
• Chinese (Mandarin, Simplified): zh, zh-CN, zh-Hans
• Chinese (Mandarin, Traditional): zh-TW, zh-Hant
• Chinese (Cantonese, Traditional): zh-HK
• Czech: cs
• Danish: da, da-DK
• Dutch: nl
• English: en, en-US, en-AU, en-GB, en-NZ, en-IN
• Estonian: et
• Finnish: fi
• Flemish: nl-BE
• French: fr, fr-CA
• German: de
• German (Switzerland): de-CH
• Greek: el
• Hindi: hi
• Hungarian: hu
• Indonesian: id
• Italian: it
• Japanese: ja
• Korean: ko, ko-KR
• Latvian: lv
• Lithuanian: lt
• Malay: ms
• Norwegian: no
• Polish: pl
• Portuguese: pt, pt-BR, pt-PT
• Romanian: ro
• Russian: ru
• Slovak: sk
• Spanish: es, es-419
• Swedish: sv, sv-SE
• Thai: th, th-TH
• Turkish: tr
• Ukrainian: uk
• Vietnamese: vi

API Reference

The API references for all the methods and events utilized in this guide are provided below.

• startTranslation()
• changeTranslationLanguage()
• stopTranslation()
• onTranslationStateChanged
• onTranslationText
• onTranslationLanguageChanged
• speakingLanguage and translationLanguage