Live Captioning for Livestreams - Flutter

Live captioning converts hosts' speech into text in real time. Using the Room class, you can enable or disable captions on the fly, and display captions dynamically in your UI for all viewers.

startTranscription()

The startTranscription() method, accessible from the Room class, is used to initiate live captions in a live stream. This method accepts an optional config (TranscriptionConfig) with the following properties:

• 1. webhookURL (optional): This is the webhook URL that is called when the state of the live captions changes, such as when they start or stop. You can learn more about webhooks here

• 2. summary (optional): A SummaryConfig with these two properties:

  • enabled (required): Indicates whether realtime transcription summary generation is enabled. Summary will be available after realtime transcription stopped.
  • prompt (optional): provides guidelines or instructions for generating a custom summary based on the realtime transcription content.

• 3. modelConfig (optional): A Map<String, dynamic> with settings for the transcription model, sent to the server as-is.

stopTranscription()

The stopTranscription() method, accessible from the Room class, is used to stop the live captions in a live stream.

Both methods return a Future that throws a VideoSDKError when the request fails, so await them inside try / catch.

Events associated with live captioning

• All participants-including all the hosts and audience members will receive a callback on the transcriptionStateChanged event with the current state of the captioning, as a TranscriptionState.
• All participants-including all the hosts and audience members will receive a callback on the transcriptionText event with the latest captions, as a TranscriptionText, whenever a host speaks. Its type is TranscriptionType.REALTIME while the caption can still change and TranscriptionType.FULL_SENTENCE once it is settled, or null.

Example

import 'dart:developer';

import 'package:flutter/material.dart';
import 'package:videosdk/videosdk.dart';
class LiveStreamView extends StatefulWidget {
  ...
}

class _LiveStreamViewState extends State<LiveStreamView> {
  // Room object
  late Room _room;

  @override
  void initState() {
    ...
    setupLivestreamEventListener();
  }

  @override
  Widget build(BuildContext context) {
    return Column(children: [
      ElevatedButton(
        onPressed: () async {
          // Configuration for realtime transcription
          final summaryConfig = SummaryConfig(
              enabled: true, prompt: 'Initial prompt for summary');
          final transcriptionConfig = TranscriptionConfig(
            webhookURL: "https://your-webhook-url.com",
            summary: summaryConfig,
          );
          // Start realtime transcription
          try {
            await _room.startTranscription(config: transcriptionConfig);
          } on VideoSDKError catch (e) {
            log("Could not start live captions: ${e.code} ${e.name}");
          }
        },
        child: const Text("Start Realtime Transcription"),
      ),
      ElevatedButton(
        onPressed: () async {
          // Stop realtime transcription
          try {
            await _room.stopTranscription();
          } on VideoSDKError catch (e) {
            log("Could not stop live captions: ${e.code} ${e.name}");
          }
        },
        child: const Text("Stop Realtime Transcription"),
      ),
    ]);
  }

  void setupLivestreamEventListener() {
    // Listen for transcription state changed event
    _room.on(Events.transcriptionStateChanged, (TranscriptionState state) {
      //State can be :: TranscriptionState.TRANSCRIPTION_STARTING
      //State can be :: TranscriptionState.TRANSCRIPTION_STARTED
      //State can be :: TranscriptionState.TRANSCRIPTION_STOPPING
      //State can be :: TranscriptionState.TRANSCRIPTION_STOPPED
      log("Livestream transcription status : ${state.name}");
    });

    // Listen for transcription text event
    _room.on(Events.transcriptionText, (TranscriptionText data) {
      log("${data.participantName}: ${data.text} ${data.timestamp}");
    });
  }
}

API Reference

The API references for all the methods utilized in this guide are provided below.

• startTranscription()
• stopTranscription()
• Events.transcriptionStateChanged
• Events.transcriptionText