Realtime Transcription - Flutter

Realtime transcription allows you to transcribe audio content into text in real-time during a session. This guide will walk you through using the startTranscription() and stopTranscription() functions to manage realtime transcription in your application.

Moreover, VideoSDK offers flexibility in configuring real-time transcription, allowing you to set up webhooks for this purpose.

Integrating Realtime Transcription Feature

The above image represents,

1. Start Transcription: The SDK Client initiates real-time transcription using the startTranscription method.

2. Resource Acquisition: VideoSDK server requests necessary resources from transcription service.

   • If the request is denied, startTranscription() throws a VideoSDKError.
   • If the request is successful, the client receives the transcriptionStateChanged event with TRANSCRIPTION_STARTED, indicating that transcription has begun.

3. Transcription Data: As transcription progresses, the client receives transcriptionText event with a TranscriptionText: the text, the speaker's participantId and participantName, the timestamp, and a type that is TranscriptionType.REALTIME for a partial result that can still change, TranscriptionType.FULL_SENTENCE for the settled sentence, or null.

4. Stop Transcription: When the client decides to stop transcription, it informs the VideoSDK server to release resources.

   • The client then receives the transcriptionStateChanged event with TRANSCRIPTION_STOPPED to confirm that transcription has ended and resources are released.

Step 1: Configure Realtime Transcription

• In this step, we set up the configuration for realtime transcription. We define the webhook URL where the webhooks will be received. TranscriptionConfig takes webhookURL, summary and modelConfig, all optional.

// Realtime Transcription Configuration
TranscriptionConfig transcriptionConfig = TranscriptionConfig(
    webhookURL: "https://webhook.your-api-server.com",
    summary: SummaryConfig(
        enabled: true,
        prompt:
            "Write summary in sections like Title, Agenda, Speakers, Action Items, Outlines, Notes and Summary"));

Step 2: Listen for the transcription events

• Here, we configure the callback methods for transcription events.

// Listen for transcription state changed event
room.on(Events.transcriptionStateChanged, (TranscriptionState state) {
    switch (state) {
      case TranscriptionState.TRANSCRIPTION_STARTING:
        print("onTranscriptionStateChanged, Realtime Transcription is starting");
      case TranscriptionState.TRANSCRIPTION_STARTED:
        print("onTranscriptionStateChanged, Realtime Transcription has started");
      case TranscriptionState.TRANSCRIPTION_STOPPING:
        print("onTranscriptionStateChanged, Realtime Transcription is stopping");
      case TranscriptionState.TRANSCRIPTION_STOPPED:
        print("onTranscriptionStateChanged, Realtime Transcription has stopped");
    }
});

// Listen for transcription text event
room.on(Events.transcriptionText, (TranscriptionText transcriptionText) {
  print("TranscriptionText : ${transcriptionText.participantName} :  ${transcriptionText.text} ${transcriptionText.timestamp}");

});

• transcriptionStateChanged delivers a TranscriptionState, such as TranscriptionState.TRANSCRIPTION_STARTED.

Step 3: Start realtime transcription

• Initiate the realtime transcription using the startTranscription() method.

// Starts realtime transcription
try {
  await room.startTranscription(config: transcriptionConfig);
} on VideoSDKError catch (error) {
  print("startTranscription failed: ${error.code} ${error.name}");
}

Step 4: Stop realtime transcription

• Terminate the realtime transcription using the stopTranscription() method.

// Stops realtime transcription
try {
  await room.stopTranscription();
} on VideoSDKError catch (error) {
  print("stopTranscription failed: ${error.code} ${error.name}");
}

startTranscription() and stopTranscription() return a Future that throws a VideoSDKError when the SDK refuses the call, so await them inside try/catch. On Android, a call made while the room reconnects reports 3065 on Events.error instead of throwing, so keep an Events.error handler too.

important

You can access a summary of your realtime transcription using the Fetch Realtime Transcription API.

Example

• The following code snippet allows you to start and stop realtime transcription with just a click. When you click the "Start Realtime Transcription" button, it begins realtime transcription. Clicking the "Stop Realtime Transcription" button ends the realtime transcription.

import 'dart:developer';

import 'package:flutter/material.dart';
import 'package:videosdk/videosdk.dart';

class RoomScreen extends StatefulWidget {
  ...
}

class _RoomScreenState extends State<RoomScreen> {
  // Room object
  late Room _room;

  @override
  void initState() {
    super.initState();
    ...
    setupRoomEventListener();
  }

  @override
  Widget build(BuildContext context) {
    return Column(children: [
      ElevatedButton(
        onPressed: () async {
          // Configuration for realtime transcription
          TranscriptionConfig transcriptionConfig = TranscriptionConfig(
              webhookURL: "https://webhook.your-api-server.com",
              summary: SummaryConfig(
                  enabled: true,
                  prompt:
                      "Write summary in sections like Title, Agenda, Speakers, Action Items, Outlines, Notes and Summary"));
          // Start realtime transcription
          try {
            await _room.startTranscription(config: transcriptionConfig);
          } on VideoSDKError catch (error) {
            log("startTranscription failed: ${error.code} ${error.name}");
          }
        },
        child: const Text("Start Realtime Transcription"),
      ),
      ElevatedButton(
        onPressed: () async {
          // Stop realtime transcription
          try {
            await _room.stopTranscription();
          } on VideoSDKError catch (error) {
            log("stopTranscription failed: ${error.code} ${error.name}");
          }
        },
        child: const Text("Stop Realtime Transcription"),
      ),
    ]);
  }

  void setupRoomEventListener() {
    // Listen for transcription state changed event
    _room.on(Events.transcriptionStateChanged, (TranscriptionState state) {
      //State can be :: TRANSCRIPTION_STARTING
      //State can be :: TRANSCRIPTION_STARTED
      //State can be :: TRANSCRIPTION_STOPPING
      //State can be :: TRANSCRIPTION_STOPPED
      log("Room transcription state : ${state.name}");
    });

    // Listen for transcription text event
    _room.on(Events.transcriptionText, (TranscriptionText transcriptionText) {
      print(
          "TranscriptionText : ${transcriptionText.participantName} :  ${transcriptionText.text} ${transcriptionText.timestamp}");
    });
  }
}

API Reference

The API references for all the methods utilized in this guide are provided below.

• startTranscription()
• stopTranscription()
• Events.transcriptionStateChanged
• Events.transcriptionText