Live Captioning for Livestreams - iOS

Live captioning enhances your livestreams by converting hosts' speech into text in real-time, boosting accessibility and engagement. Using the startTranscription() and stopTranscription() functions you can enable or disable captions on the fly, and display captions dynamically in your UI for all viewers.

startTranscription()

The startTranscription() method, accessible from the Room object, is used to initiate live captions in a live stream. This method accepts an optional TranscriptionConfig with the following properties:

• 1. webhookURL (optional): This is the webhook URL that receives an event whenever the state of the live captions changes, such as when they start or stop. You can learn more about webhooks here

• 2. summary (optional):

  • enabled: Indicates whether realtime transcription summary generation is enabled. Summary will be available after realtime transcription has stopped. Default: false.
  • prompt (optional): provides guidelines or instructions for generating a custom summary based on the realtime transcription content.

• 3. modelConfig (optional): Extra settings for the transcription model, sent as they are.

stopTranscription()

The stopTranscription() method, accessible from the Room object, is used to stop the live captions in a live stream.

Both methods are async throws. Call them with try await inside a Task after you join the live stream. If a call fails, for example because you have not joined yet, it throws a VideoSDKError.

Example

@IBAction func startLiveCaptionTapped(_ sender: Any) {

    let config =  TranscriptionConfig(
            webhookURL: "YOUR_WEBHOOK_URL",
            summary: SummaryConfig(
            enabled: true,
            prompt: "Write summary in sections like Title, Agenda, Speakers, Action Items, Outlines, Notes and Summary"
            )
    )

    Task {
        do {
            try await room?.startTranscription(config: config)
        } catch {
            print("Failed to start live captions: \(error)")
        }
    }
}

@IBAction func stopLiveCaptionTapped(_ sender: Any) {
    Task {
        do {
            try await room?.stopTranscription()
        } catch {
            print("Failed to stop live captions: \(error)")
        }
    }
}

Events associated with live captioning

• All participants-including all the hosts and audience members will receive a callback on the onTranscriptionStateChanged event with the current state of the captioning.
• All participants-including all the hosts and audience members will receive a callback on the onTranscriptionText event with the latest captions whenever a host speaks.

import VideoSDKRTC

extension LiveStreamViewController: RoomEventListener {

// Listen for transcription state changed event
    func onTranscriptionStateChanged(state: TranscriptionState) {

        switch state {
        case .TRANSCRIPTION_STARTING:
            print("transcription starting")

        case .TRANSCRIPTION_STARTED:
            print("transcription started")

        case .TRANSCRIPTION_STOPPING:
            print("transcription stopping")

        case .TRANSCRIPTION_STOPPED:
            print("transcription stopped")

        @unknown default:
            break
        }

    }

// Listen for transcription text event
    func onTranscriptionText(_ data: TranscriptionText) {
        print("transcription text: ", data.text)
        print("participant name", data.participantName)
    }
}

tip

To show captions in each viewer's own language, use real-time translation. Each viewer sets a language with the translationLanguage parameter of createRoom() or with changeTranslationLanguage(_:). After you call startTranslation(), each viewer receives the translated text through onTranslationText(_:). See Realtime Translation for a step-by-step guide.

API Reference

The API references for all the methods utilized in this guide are provided below.

• startTranscription()
• stopTranscription()
• onTranscriptionStateChanged()
• onTranscriptionText()