Video Processor Android Setup - Flutter

• VideoSDK allows you to add unique effects to your video stream before transmission during video calls with custom native video processing code. This guide offers step-by-step instructions for creating and implementing a custom video processor for frame manipulation on Android.

• To enable native processor development on Android, add the following dependency to your build.gradle file located at <project root>/android/build.gradle:

build.gradle

dependencies {
  implementation 'io.github.webrtc-sdk:android:125.6422.02'
  // other app dependencies
  }

Step-by-Step Guide: Integrating Native Video Processor in Flutter

Step 1: Create Your Own Processor Code

• First, create a class that implements the VideoProcessor interface. In this class, define the onFrameReceived function, which will contain the logic for manipulating video frames according to your needs. After applying the desired effect, make sure to return the processed frame.

Kotlin

YourOwnBackgroundProcessor.kt

package com.yourOwnPackage

import live.videosdk.webrtc.VideoProcessor
import org.webrtc.VideoFrame

class YourOwnBackgroundProcessor : VideoProcessor {
    // Override the onFrameReceived function to access VideoFrames
    override fun onFrameReceived(videoFrame: VideoFrame): VideoFrame {
        // Function to manipulate frame as per requirement
        return applyEffect(videoFrame)
    }

    private fun applyEffect(videoFrame: VideoFrame): VideoFrame {
        // Apply effect here and return the processed VideoFrame
        return processedVideoFrame 
    }
}

Java

YourOwnBackgroundProcessor.java

package com.yourOwnPackage

import live.videosdk.webrtc.VideoProcessor;
import org.webrtc.VideoFrame;

public class YourOwnBackgroundProcessor implements VideoProcessor {
    //Override the onFrameReceived function to get access to the VideoFrames
    @Override
    public VideoFrame onFrameReceived(VideoFrame videoFrame) {
        //Function to manupulate frame as per requirement.
        VideoFrame processedVideoFrame = applyEffect(videoFrame)
        return processedVideoFrame
    }
    private VideoFrame applyEffect(VideoFrame videoFrame) {
        //Apply Effect here and return the processed Video Frame
        return processedVideoFrame
    }
}

Step 2: Register your Processor

• In your MainActivity, register your custom video processor using the registerVideoProcessor() method from the VideoSDK class during app initialization. Provide a string representing the unique processor name along with an instance of your processor to complete the registration. This processor name will be used later to apply the effects offered by your processor.

Kotlin

MainActivity.kt

package com.example.example

import io.flutter.embedding.android.FlutterActivity
import android.os.Bundle
import com.yourOwnPackage.YourOwnBackgroundProcessor
import live.videosdk.videosdk.VideoSDK

class MainActivity : FlutterActivity() {

    private lateinit var bgProcessor: YourOwnBackgroundProcessor

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        //Create an instance of your Processor 
        bgProcessor = YourOwnBackgroundProcessor()
        //Create an instance of the VideoSDK class
        val videoSDK = VideoSDK.getInstance()
        //Register your processor with a unique name and its instance
        videoSDK.registerVideoProcessor("ProcessorName", bgProcessor)
    }
}

Java

MainActivity.java

package com.example.example;

import io.flutter.embedding.android.FlutterActivity;
import android.os.Bundle;
import com.yourOwnPackage.YourOwnBackgroundProcessor;
import live.videosdk.videosdk.VideoSDK;

public class MainActivity extends FlutterActivity {

    private YourOwnBackgroundProcessor bgProcessor;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        //Create an instance of your Processor 
        bgProcessor = new YourOwnBackgroundProcessor();
        //Create an instance of the VideoSDK class
        VideoSDK videoSDK = VideoSDK.getInstance();
        //Register your processor with a unique name and its instance
        videoSDK.registerVideoProcessor("ProcessorName", bgProcessor);
    }
}

Step 3: Apply the Processor

• Once you have registered the processor during initialization, you can use it throughout the entire app lifecycle. To apply the effect provided by the processor, use the applyVideoProcessor() method from the VideoSDK class. This method requires the name of the processor that was used during registration.
• If you have registered multiple processors and wish to switch between them, simply call the applyVideoProcessor() method with the name of the desired processor.

import 'package:flutter/material.dart';
import 'package:videosdk/videosdk.dart';

class MeetingScreen extends StatefulWidget {
  //Existing configuration
}

class _MeetingScreenState extends State<MeetingScreen> {
  late Room _room;

  @override
  void initState() {
    //Existing configuration
  }

  @override
  Widget build(BuildContext context) {
    return Column(
      children:[
        // To apply the effect from the processor, provide the name used during registration.
        ElevatedButton(
          onPressed:(){
            VideoSDK.applyVideoProcessor(videoProcessorName: "ProcessorName");
          },
          child: const Text("Apply Video Processor"),
        ),
      ]
    );
  }
}

Step 4: Remove the Processor

• You can remove the processor when you no longer need the effect. To do this, use the removeVideoProcessor() method from the VideoSDK class to eliminate the effect provided by the processor. This method will remove the effect of the currently active video processor.

import 'package:flutter/material.dart';
import 'package:videosdk/videosdk.dart';

class MeetingScreen extends StatefulWidget {
  //Existing configuration
}

class _MeetingScreenState extends State<MeetingScreen> {
  late Room _room;

  @override
  void initState() {
    //Existing configuration
  }

  @override
  Widget build(BuildContext context) {
    return Column(
      children:[
        // To apply the effect from the processor, provide the name used during registration.
        ElevatedButton(
          onPressed:(){
            VideoSDK.removeVideoProcessor();
          },
          child: const Text("Remove Video Processor"),
        ),
      ]
    );
  }
}

API Reference

The API references for all the methods utilized in this guide are provided below.

• registerVideoProcessor()
• applyVideoProcessor
• removeVideoProcessor()