Live Captioning for Livestreams - React Native

Live captioning shows the hosts' speech as text to every viewer in real time. Using the useTranscription hook, you can start and stop captions during the stream and show the caption text in your UI.

Caption state and caption text arrive as events, so you can place captions anywhere in your broadcast layout.

tip

To show each viewer the hosts' speech in their own language, use Live Translation.

note

This page uses the following asynchronous methods. Refer to their API reference for the errors each Promise may reject with, and handle these rejections appropriately based on your use case.

• startTranscription()
• stopTranscription()

startTranscription()

The startTranscription() method, accessible from the useTranscription hook, is used to initiate live captions in a live stream. It takes an optional config object with these fields:

• 1. webhookUrl (optional): This is the webhook URL that VideoSDK calls when the state of the live captions changes, such as when they start or stop. You can learn more about webhooks here

• 2. modelConfig (optional): Model configuration used for the captions.

• 3. summary (optional): Leave it out if you do not want a summary.

  • enabled: Indicates whether realtime transcription summary generation is enabled. Summary will be available after realtime transcription has stopped.
  • prompt: provides guidelines or instructions for generating a custom summary based on the realtime transcription content.

stopTranscription()

The stopTranscription() method, accessible from the useTranscription hook, is used to stop the live captions in a live stream.

Example

import { View, TouchableOpacity, Text, StyleSheet } from "react-native";
import { useTranscription } from "@videosdk.live/react-native-sdk";

const LiveStreamView = () => {
  const { startTranscription, stopTranscription } = useTranscription();

  const handleStartCaptions = async () => {
    // Configuration for captions
    const config = {
      webhookUrl: "https://www.example.com",
      summary: {
        enabled: true,
        prompt:
          "Write summary in sections like Title, Agenda, Speakers, Action Items, Outlines, Notes and Summary",
      },
    };

    try {
      await startTranscription(config);
    } catch (err) {
      console.error("startTranscription failed:", err);
    }
  };

  const handleStopCaptions = async () => {
    try {
      await stopTranscription();
    } catch (err) {
      console.error("stopTranscription failed:", err);
    }
  };

  return (
    <View style={styles.container}>
      <TouchableOpacity 
        style={styles.button} 
        onPress={handleStartCaptions}
      >
        <Text style={styles.buttonText}>Start Live Captions</Text>
      </TouchableOpacity>
      <TouchableOpacity 
        style={styles.button} 
        onPress={handleStopCaptions}
      >
        <Text style={styles.buttonText}>Stop Live Captions</Text>
      </TouchableOpacity>
    </View>
  );
};

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    alignItems: 'center',
    gap: 10,
  },
  button: {
    backgroundColor: '#007AFF',
    padding: 12,
    borderRadius: 8,
    minWidth: 200,
    alignItems: 'center',
  },
  buttonText: {
    color: 'white',
    fontSize: 16,
    fontWeight: '600',
  },
});

export default LiveStreamView;

Events associated with live captioning

• All participants-including all the hosts and audience members will receive a callback on the onTranscriptionStateChanged event with the current state of the captioning.
• All participants-including all the hosts and audience members will receive a callback on the onTranscriptionText event with the latest captions whenever a host speaks.

import { View, Text, StyleSheet } from "react-native";
import { Constants, useTranscription } from "@videosdk.live/react-native-sdk";

const LiveStreamView = () => {
  // Callback function for transcription state change event
  function onTranscriptionStateChanged(data) {
    const { state } = data;

    if (state === Constants.transcriptionEvents.TRANSCRIPTION_STARTING) {
      console.log("Live Captions starting");
    } else if (state === Constants.transcriptionEvents.TRANSCRIPTION_STARTED) {
      console.log("Live Captions started");
    } else if (
      state === Constants.transcriptionEvents.TRANSCRIPTION_STOPPING
    ) {
      console.log("Live Captions stopping");
    } else if (state === Constants.transcriptionEvents.TRANSCRIPTION_STOPPED) {
      console.log("Live Captions stopped");
    }
  }

  // Callback function for transcription text event
  function onTranscriptionText(data) {
    let { participantName, text, timestamp } = data;
    console.log(`${participantName}: ${text} ${timestamp}`);
  }

  // Passing callback functions to useTranscription hook
  const { startTranscription, stopTranscription } = useTranscription({
    onTranscriptionStateChanged,
    onTranscriptionText,
  });

  return (
    <View style={styles.container}>
      {/* Your room UI components */}
    </View>
  );
};

const styles = StyleSheet.create({
  container: {
    flex: 1,
  },
});

export default LiveStreamView;

API Reference

The API references for all the methods utilized in this guide are provided below.

• startTranscription()
• stopTranscription()
• onTranscriptionStateChanged
• onTranscriptionText