Realtime Translation - iOS

Realtime translation turns the speech in a room into text, translated into each participant's own language, while the room runs. This guide explains how to use the startTranslation(config:), stopTranslation() and changeTranslationLanguage(_:) methods of the Room class.

You set a participant's translation language in createRoom(), and can change it at any point in the room with changeTranslationLanguage(_:).

note

The translation language is always a language code, both in the translationLanguage parameter of createRoom() and in changeTranslationLanguage(_:). Only the codes listed in Supported Languages return translated text. If you pass an unsupported code, no translated text arrives.

Integrating Realtime Translation Feature

1. Start Translation: You start realtime translation with the startTranslation(config:) method. The onTranslationStateChanged(state:) event then reports TRANSLATION_STARTING, followed by TRANSLATION_STARTED once translation is running.

2. Translation Data: As translation progresses, the onTranslationText(_:) event delivers a TranslationText with the translated text, the participant who spoke it, the timestamp and the language.

3. Change Translation Language: A participant can switch to another language while translation is running with changeTranslationLanguage(_:). The onTranslationLanguageChanged(participantId:language:) event then reports the new language.

4. Stop Translation: When you stop translation with stopTranslation(), the onTranslationStateChanged(state:) event reports TRANSLATION_STOPPING, followed by TRANSLATION_STOPPED.

Step 1: Configure the translation languages

• Set the participant's translation language and speaking language when you create the room with createRoom().

do {
    room = try VideoSDK.createRoom(
        roomId: "abcd-efgh-xyzw",
        token: "<Your-Token>",
        participantName: "John Doe",
        translationLanguage: "hi",
        speakingLanguage: "en"
    )
} catch {
    print("Failed to create the room: \(error)")
}

• translationLanguage: The language the participant receives translated text in. onTranslationText(_:) delivers only text in this language, so a participant without a translation language receives no translated text until they call changeTranslationLanguage(_:).

• speakingLanguage: The language the participant speaks for the whole session. When it is not set, the participant's translationLanguage is also used as their spoken language. Refer to Speaking Language and Translation Language for how the two work together.

Step 2: Start realtime translation

• Start realtime translation for the room with the startTranslation(config:) method, once the room is joined. config is optional. Leave it out to use the default settings.

@IBAction func startTranslationTapped(_ sender: Any) {
    Task {
        do {
            try await room?.startTranslation()
        } catch let error as VideoSDKError {
            print("Failed to start translation: \(error.name) (\(error.code))")
        } catch {
            print("Failed to start translation: \(error)")
        }
    }
}

• One translation runs for the whole room. A participant who joins while it is already running may not receive TRANSLATION_STARTED, so don't wait for that event before you show translated text.

Step 3: Change the translation language

• Switch to another language with the changeTranslationLanguage(_:) method. It changes the language for this participant only.

@IBAction func frenchTapped(_ sender: Any) {
    Task {
        do {
            try await room?.changeTranslationLanguage("fr")
        } catch {
            print("Failed to change the translation language: \(error)")
        }
    }
}

• The new language applies once onTranslationLanguageChanged(participantId:language:) reports it for this participant. Until then, onTranslationText(_:) keeps delivering the old language.

Step 4: Stop realtime translation

• Stop realtime translation with the stopTranslation() method. It stops translation for every participant in the room.

@IBAction func stopTranslationTapped(_ sender: Any) {
    Task {
        do {
            try await room?.stopTranslation()
        } catch {
            print("Failed to stop translation: \(error)")
        }
    }
}

note

• startTranslation(config:), stopTranslation() and changeTranslationLanguage(_:) are async throws. Call them with try await inside a Task, and catch the VideoSDKError they throw.
• Before you join the room, or after you leave it, they throw ERROR_ACTION_PERFORMED_BEFORE_ROOM_JOINED (3035). While the room is reconnecting, they throw ERROR_ROOM_RECONNECTING (3065).
• When the server refuses the request, they throw START_TRANSLATION_FAILED (4042), STOP_TRANSLATION_FAILED (4043) or CHANGE_TRANSLATION_LANGUAGE_FAILED (4044).

Step 5: Listen for the translation events

• Add these callbacks to your RoomEventListener.

import VideoSDKRTC

extension RoomViewController: RoomEventListener {

    func onTranslationStateChanged(state: TranslationState) {
        switch state {
        case .TRANSLATION_STARTING:
            print("Realtime Translation is starting")
        case .TRANSLATION_STARTED:
            print("Realtime Translation is started")
        case .TRANSLATION_STOPPING:
            print("Realtime Translation is stopping")
        case .TRANSLATION_STOPPED:
            print("Realtime Translation is stopped")
        @unknown default:
            break
        }
    }

    func onTranslationText(_ data: TranslationText) {
        print("\(data.participantName) (\(data.language)): \(data.text)")
    }

    func onTranslationLanguageChanged(participantId: String, language: String) {
        print("\(participantId) now receives translations in \(language)")
    }
}

• onTranslationStateChanged(state:) receives a TranslationState: TRANSLATION_STARTING, TRANSLATION_STARTED, TRANSLATION_STOPPING or TRANSLATION_STOPPED. The translationState property of the Room class holds the last state you received.

• onTranslationText(_:) receives a TranslationText with text, participantId, participantName, participantLanguage (the language the speaker spoke), language (the language the text was translated into), timestamp and type. It fires only for text in this participant's translation language.

• type is .PARTIAL while the service is still revising a line, and .FULL once the line is final. A new .PARTIAL line replaces the previous one, so add a line to a transcript only when it is .FULL.

• onTranslationLanguageChanged(participantId:language:) fires when any participant changes their translation language, including you. Compare participantId with room?.localParticipant.id to react only to your own change.

Example

• The following code starts and stops realtime translation, and switches the language, with a tap. Partial lines replace each other, and full lines are kept.

import UIKit
import VideoSDKRTC

class TranslationViewController: UIViewController {

    var room: Room?
    @IBOutlet weak var captionLabel: UILabel!
    private var finishedLines: [String] = []

    // create the room with a translationLanguage, add this view controller
    // as a listener with room?.addEventListener(self) and join the room
    // ...

    @IBAction func startTapped(_ sender: Any) {
        runTranslationAction { try await $0.startTranslation() }
    }

    @IBAction func stopTapped(_ sender: Any) {
        runTranslationAction { try await $0.stopTranslation() }
    }

    @IBAction func frenchTapped(_ sender: Any) {
        runTranslationAction { try await $0.changeTranslationLanguage("fr") }
    }

    private func runTranslationAction(_ action: @escaping (Room) async throws -> Void) {
        guard let room else { return }
        Task {
            do {
                try await action(room)
            } catch let error as VideoSDKError {
                print("Translation request failed: \(error.name) (\(error.code))")
            } catch {
                print("Translation request failed: \(error)")
            }
        }
    }
}

extension TranslationViewController: RoomEventListener {

    func onTranslationStateChanged(state: TranslationState) {
        print("Translation state: \(state.rawValue)")
    }

    func onTranslationText(_ data: TranslationText) {
        let line = "\(data.participantName): \(data.text)"
        if data.type == .FULL {
            finishedLines.append(line)
            captionLabel.text = finishedLines.suffix(3).joined(separator: "\n")
        } else {
            // a partial line replaces the previous partial line
            captionLabel.text = (finishedLines.suffix(2) + [line]).joined(separator: "\n")
        }
    }
}

Speaking Language and Translation Language

speakingLanguage sets the language a participant speaks, while translationLanguage sets the language they receive translated text in. The two examples below show how translation works with and without a speaking language.

Example 1: Without speakingLanguage

Both participants set only their translation language, so each participant's translation language is also their spoken language.

// On Participant 1's device (Spanish)
let room1 = try VideoSDK.createRoom(
    roomId: roomId,
    token: token,
    participantName: "Participant 1",
    translationLanguage: "es"
)

// On Participant 2's device (German)
let room2 = try VideoSDK.createRoom(
    roomId: roomId,
    token: token,
    participantName: "Participant 2",
    translationLanguage: "de"
)

Speaker	Speaking Language (Input)	Receiver	Translated Output
Participant 1	Spanish (es)	Participant 2	German (de)
Participant 2	German (de)	Participant 1	Spanish (es)

Here each participant must speak their own translation language, because no speaking language is set.

Example 2: With speakingLanguage on one participant

Here only Participant 1 sets a speakingLanguage. That fixes the language of their speech, while Participant 2's translation language is still also their spoken language.

// On Participant 1's device (speaks English, receives Spanish)
let room1 = try VideoSDK.createRoom(
    roomId: roomId,
    token: token,
    participantName: "Participant 1",
    translationLanguage: "es",
    speakingLanguage: "en"
)

// On Participant 2's device (no speakingLanguage, receives German)
let room2 = try VideoSDK.createRoom(
    roomId: roomId,
    token: token,
    participantName: "Participant 2",
    translationLanguage: "de"
)

Speaker	Speaking Language (Input)	Receiver	Translated Output
Participant 1	English (en)	Participant 2	German (de)
Participant 2	German (de)	Participant 1	Spanish (es)

Key takeaways

• When speakingLanguage is not set, the participant's translation language is also their speaking language.
• When only one participant sets a speakingLanguage, that participant's input language is fixed, while the others keep using their translation language as their input language.
• Setting speakingLanguage gives you more control, especially in rooms with many languages.

Supported Languages

These are the supported language codes:

• Multilingual (Spanish + English): multi
• Bulgarian: bg
• Catalan: ca
• Chinese (Mandarin, Simplified): zh, zh-CN, zh-Hans
• Chinese (Mandarin, Traditional): zh-TW, zh-Hant
• Chinese (Cantonese, Traditional): zh-HK
• Czech: cs
• Danish: da, da-DK
• Dutch: nl
• English: en, en-US, en-AU, en-GB, en-NZ, en-IN
• Estonian: et
• Finnish: fi
• Flemish: nl-BE
• French: fr, fr-CA
• German: de
• German (Switzerland): de-CH
• Greek: el
• Hindi: hi
• Hungarian: hu
• Indonesian: id
• Italian: it
• Japanese: ja
• Korean: ko, ko-KR
• Latvian: lv
• Lithuanian: lt
• Malay: ms
• Norwegian: no
• Polish: pl
• Portuguese: pt, pt-BR, pt-PT
• Romanian: ro
• Russian: ru
• Slovak: sk
• Spanish: es, es-419
• Swedish: sv, sv-SE
• Thai: th, th-TH
• Turkish: tr
• Ukrainian: uk
• Vietnamese: vi

API Reference

The API references for all the methods and events utilized in this guide are provided below.

• createRoom()
• translationLanguage
• speakingLanguage
• startTranslation()
• stopTranslation()
• changeTranslationLanguage()
• onTranslationStateChanged()
• onTranslationText()
• onTranslationLanguageChanged()