Post Transcription & Summary - Android

This guide will walk you through the necessary steps to enable post-transcription functionality in your application, allowing users to access transcriptions of recorded rooms.

You can also set up webhooks to follow post transcription as it runs.

Integrating Post Transcription Feature

In the image above, the recording starts with transcription and summary turned on. A webhook is sent when the recording starts, when it stops, when transcription starts, and again once the transcription is finished and its summary is generated.

Step 1: Configure Post Transcription & Summary

• In this step, we set up the configuration for post transcription and summary generation. We define the webhook URL where the webhooks will be received.

// Webhook URL where, webhooks are received
val webhookUrl = "https://www.example.com"

// Post Transcription Configuration
val prompt = "Write summary in sections like Title, Agenda, Speakers, Action Items, Outlines, Notes and Summary"
val summaryConfig = SummaryConfig(true, prompt)
val modelId = "raman_v1"
val transcription = PostTranscriptionConfig(true, summaryConfig, modelId)

Step 2: Start Recording with Transcription Configuration

• Once the configuration is set, we initiate the recording process for the room, incorporating the transcription configuration we defined earlier.

// Start Recording
lifecycleScope.launch {
    try {
        room!!.startRecording(webhookUrl = webhookUrl, awsDirPath = "AWS Directory Path", transcription = transcription)
    } catch (e: VideoSDKException) {
        Log.e("VideoSDK", "Start recording failed: ${e.code} ${e.name}")
    }
}

note

• startRecording() and stopRecording() are suspend functions. Call them from a coroutine, such as lifecycleScope.launch (from androidx.lifecycle:lifecycle-runtime-ktx), and catch VideoSDKException, which they throw when the request fails.
• They return before the recording has started or stopped. onRecordingStarted() tells you that the recording is running, and onRecordingStopped() that it has stopped.
• Called before the room is joined, or while it is reconnecting, they do nothing and report 3035 (ERROR_ACTION_PERFORMED_BEFORE_ROOM_JOINED) or 3065 (ERROR_ROOM_RECONNECTING) on onError instead of throwing.

Step 3: Stop Recording

• When the room ends or recording is no longer needed, we stop the recording process.

// Stop Recording
lifecycleScope.launch {
    try {
        room!!.stopRecording()
    } catch (e: VideoSDKException) {
        Log.e("VideoSDK", "Stop recording failed: ${e.code} ${e.name}")
    }
}

Step 4: Fetch Post Transcriptions

• After the room recording is stopped, post transcription and summary data can be obtained using the Post Transcription API.

• The API provides information such as the status of transcription, file paths for transcription data in various formats (JSON, SRT, TXT, TSV, VTT), and a summarized text file.

• The transcription document is structured as follows:

{
  "id": "40b0a4ed-9842-40c9-a288-e4b1bf98a90a",
  "status": "completed",
  "roomId": "abc-xyzw-lmno",
  "sessionId": "621497578bea0d0404c35c4c",
  "recordingId": "65d303d6d2c373dfd71b38a2",
  "filePath": "https://cdn.videosdk.live/encoded/videos/dummy.mp4",
  "transcriptionFilePaths": {
    "json": "https://cdn.videosdk.live/transcriptions/dummy/dummy.json",
    "srt": "https://cdn.videosdk.live/transcriptions/dummy/dummy.srt",
    "txt": "https://cdn.videosdk.live/transcriptions/dummy/dummy.txt",
    "tsv": "https://cdn.videosdk.live/transcriptions/dummy/dummy.tsv",
    "vtt": "https://cdn.videosdk.live/transcriptions/dummy/dummy.vtt"
  },
  "summarizedFilePaths": {
    "txt": "https://cdn.videosdk.live/transcriptions/dummy/dummy-summary.txt"
  },
  "userStorage": null,
  "start": "2024-02-27T16:00:36.828Z",
  "end": "2024-02-27T16:01:46.939Z"
}

Example

• The following code snippet allows you to start and stop recording rooms with just a click. When you click the "Start Recording" button, it begins recording the room and enables post-transcription, which generates a summary of the room's content. Clicking the "Stop Recording" button ends the recording.

import android.os.Bundle
import android.widget.Button
import android.widget.Toast
import androidx.appcompat.app.AppCompatActivity
import androidx.lifecycle.lifecycleScope
import kotlinx.coroutines.launch
import live.videosdk.rtc.android.Room
import live.videosdk.rtc.android.lib.VideoSDKException
import live.videosdk.rtc.android.lib.transcription.PostTranscriptionConfig
import live.videosdk.rtc.android.lib.transcription.SummaryConfig

class RoomActivity : AppCompatActivity() {
    private lateinit var room: Room

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_room)

        // create the room
        //...

        val startButton = findViewById<Button>(R.id.startRecording)
        val stopButton = findViewById<Button>(R.id.stopRecording)

        startButton.setOnClickListener {
            // Configuration for post transcription
            val prompt = "Write summary in sections like Title, Agenda, Speakers, Action Items, Outlines, Notes and Summary"
            val summaryConfig = SummaryConfig(true, prompt)
            val modelId = "raman_v1"
            val transcription = PostTranscriptionConfig(true, summaryConfig, modelId)

            // Leave out `webhookUrl` or `awsDirPath` if you don't have them.
            // Start recording with post transcription
            lifecycleScope.launch {
                try {
                    room.startRecording("YOUR WEB HOOK URL", "AWS Directory Path", transcription = transcription)
                } catch (e: VideoSDKException) {
                    Toast.makeText(this@RoomActivity, "Start recording failed: ${e.message}", Toast.LENGTH_SHORT).show()
                }
            }
        }

        stopButton.setOnClickListener {
            // Stop recording
            lifecycleScope.launch {
                try {
                    room.stopRecording()
                } catch (e: VideoSDKException) {
                    Toast.makeText(this@RoomActivity, "Stop recording failed: ${e.message}", Toast.LENGTH_SHORT).show()
                }
            }
        }
    }
}

important

Please be aware that there may be a delay in processing the transcription data after the recording has been stopped. This delay could vary depending on factors such as server load and the duration of the session.

API Reference

The API references for all the methods utilized in this guide are provided below.

• startRecording()
• stopRecording()
• fetchPostTranscriptions