Live Captioning for Livestreams - React Native

Live captioning enhances your livestreams by converting hosts' speech into text in real-time, boosting accessibility and engagement. Using the useTranscription hook, you can enable or disable captions on the fly, and display captions dynamically in your UI for all viewers.

Video SDK offers flexible configuration and event-driven updates to help you integrate captions seamlessly into your broadcast layout.

startTranscription()

The startTranscription() method, accesible from the useTranscription hook, is used to initiate live captions in a live stream. This method accepts the following two parameters:

• 1. webhookUrl (optional): This is the webhook URL where you can listen to events related to the recording, such as the start and stop of recording. It triggers when the recording is completed and stored on the server. You can learn more about webhooks here

• 2. summary (optional):

  • enabled: Indicates whether realtime transcription summary generation is enabled. Summary will be available after realtime transcription stopped. Default: false.
  • prompt: provides guidelines or instructions for generating a custom summary based on the realtime transcription content.

stopTranscription()

The stopTranscription() method, accesible from the useTranscription hook, is used to stop the live captions in a live stream.

Example

import { View, TouchableOpacity, Text, StyleSheet } from "react-native";
import { useMeeting,useTranscription } from "@videosdk.live/react-native-sdk";

const LiveStreamView = () => {
  const { startTranscription, stopTranscription } = useTranscription();

  const handleStartCaptions = () => {
    // Configuration for captions
    const config = {
      webhookUrl: "https://www.example.com",
      summary: {
        enabled: true,
        prompt:
          "Write summary in sections like Title, Agenda, Speakers, Action Items, Outlines, Notes and Summary",
      },
    };

    startTranscription(config);
  };

  const handleStopCaptions = () => {
    stopTranscription();
  };

  return (
    <View style={styles.container}>
      <TouchableOpacity 
        style={styles.button} 
        onPress={handleStartCaptions}
      >
        <Text style={styles.buttonText}>Start Live Captions</Text>
      </TouchableOpacity>
      <TouchableOpacity 
        style={styles.button} 
        onPress={handleStopCaptions}
      >
        <Text style={styles.buttonText}>Stop Live Captions</Text>
      </TouchableOpacity>
    </View>
  );
};

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    alignItems: 'center',
    gap: 10,
  },
  button: {
    backgroundColor: '#007AFF',
    padding: 12,
    borderRadius: 8,
    minWidth: 200,
    alignItems: 'center',
  },
  buttonText: {
    color: 'white',
    fontSize: 16,
    fontWeight: '600',
  },
});

export default LiveStreamView;

Events associated with live captioning

• All participants-including all the hosts and audience members will receive a callback on the onTranscriptionStateChanged event with the current state of the captioning.
• All participants-including all the hosts and audience members will receive a callback on the onTranscriptionText event with the latest captions whenever a host speaks.

import { View, Text, StyleSheet } from "react-native";
import { Constants, useMeeting, useTranscription} from "@videosdk.live/react-native-sdk";

const LiveStreamView = () => {
  // Callback function for transcription state change event
  function onTranscriptionStateChanged(data) {
    const { status, id } = data;

    if (status === Constants.transcriptionEvents.TRANSCRIPTION_STARTING) {
      console.log("Live Captions starting", id);
    } else if (status === Constants.transcriptionEvents.TRANSCRIPTION_STARTED) {
      console.log("Live Captions started", id);
    } else if (
      status === Constants.transcriptionEvents.TRANSCRIPTION_STOPPING
    ) {
      console.log("Live Captions stopping", id);
    } else if (status === Constants.transcriptionEvents.TRANSCRIPTION_STOPPED) {
      console.log("Live Captions stopped", id);
    }
  }

  // Callback function for transcription text event
  function onTranscriptionText(data) {
    let { participantId, participantName, text, timestamp, type } = data;
    console.log(`${participantName}: ${text} ${timestamp}`);
  }

  // Passing callback functions to useTranscription hook
  const { startTranscription, stopTranscription } = useTranscription({
    onTranscriptionStateChanged,
    onTranscriptionText,
  });

  return (
    <View style={styles.container}>
      {/* Your meeting UI components */}
    </View>
  );
};

const styles = StyleSheet.create({
  container: {
    flex: 1,
  },
});

export default LiveStreamView;

API Reference

The API references for all the methods utilized in this guide are provided below.

• startTranscription()
• stopTranscription()
• onTranscriptionStateChanged
• onTranscriptionText