Video Processor iOS Setup - Flutter

• VideoSDK allows you to add unique effects to your video stream before transmission during video calls with custom native video processing code. This guide offers step-by-step instructions for creating and implementing a custom video processor for frame manipulation on iOS.

• To enable native processor development on iOS, comment the use_frameworks! line in your podfile:

build.gradle

target 'Runner' do
  # use_frameworks!
  use_modular_headers!

  flutter_install_all_ios_pods File.dirname(File.realpath(__FILE__))
end

Step-by-Step Guide: Integrating Native Video Processor in Flutter

Step 1: Create Your Own Processor Code

• First, create a class that implements the VideoProcessor interface. In this class, define the onFrameReceived function, which will contain the logic for manipulating video frames according to your needs. After applying the desired effect, make sure to return the processed frame.

Swift

YourOwnBackgroundProcessor.swift

import Foundation

 public class YourOwnBackgroundProcessor: VideoProcessor {
    
     // Override the onFrameReceived function to access VideoFrames
     public override func onFrameReceived(_ frame: RTCVideoFrame) -> RTCVideoFrame? {
         // Function to manipulate frame as per requirement
         let processedframe = applyEffect(<#T##frame: RTCVideoFrame##RTCVideoFrame#>)
         return processedframe
         
     }

     private func applyEffect(_ frame:RTCVideoFrame) -> RTCVideoFrame?{
         // Apply effect here and return the processed VideoFrame
         return processedFrame
     }
}

Objective-C

YourOwnBackgroundProcessor.m

#import <Foundation/Foundation.h>
#import <WebRTC/RTCVideoFrame.h>
#import "videosdk_webrtc/VideoProcessor.h"

@interface YourOwnBackgroundProcessor : VideoProcessor

@end

@implementation YourOwnBackgroundProcessor

// Override the onFrameReceived function to access VideoFrames
- (RTCVideoFrame *)onFrameReceived:(RTCVideoFrame *)frame {
    // Function to manipulate frame as per requirement
    RTCVideoFrame *processedFrame = [self applyEffect:frame];
    return processedFrame;
}

- (RTCVideoFrame *)applyEffect:(RTCVideoFrame *)frame {
    // Apply effect here and return the processed VideoFrame
    return processedFrame; // Define processedFrame appropriately
}

@end

Step 2: Register your Processor

• In your AppDelegate, register your custom video processor using the registerVideoProcessor() method from the VideoSDK class during app initialization. Provide a string representing the unique processor name along with an instance of your processor to complete the registration. This processor name will be used later to apply the effects offered by your processor.

Swift

AppDelegate.swift

import UIKit
import Flutter
import videosdk
import Foundation 

@UIApplicationMain
@objc class AppDelegate: FlutterAppDelegate {
  override func application(
    _ application: UIApplication,
    didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
  ) -> Bool {
    {
        //Create an instance of your Processor 
        let bgProcessor = YourOwnBackgroundProcessor()
        //Create an instance of the VideoSDK class
        let videoSDK = VideoSDK.getInstance
        //Register your processor with a unique name and its instance
        videoSDK.registerVideoProcessor(videoProcessorName: "ProcessorName", videoProcessor: bgProcessor)
    }

    return super.application(application, didFinishLaunchingWithOptions: launchOptions)
  }
}

Step 3: Apply the Processor

• Once you have registered the processor during initialization, you can use it throughout the entire app lifecycle. To apply the effect provided by the processor, use the applyVideoProcessor() method from the VideoSDK class. This method requires the name of the processor that was used during registration.
• If you have registered multiple processors and wish to switch between them, simply call the applyVideoProcessor() method with the name of the desired processor.

import 'package:flutter/material.dart';
import 'package:videosdk/videosdk.dart';

class MeetingScreen extends StatefulWidget {
  //Existing configuration
}

class _MeetingScreenState extends State<MeetingScreen> {
  late Room _room;

  @override
  void initState() {
    //Existing configuration
  }

  @override
  Widget build(BuildContext context) {
    return Column(
      children:[
        // To apply the effect from the processor, provide the name used during registration.
        ElevatedButton(
          onPressed:(){
            VideoSDK.applyVideoProcessor(videoProcessorName: "ProcessorName");
          },
          child: const Text("Apply Video Processor"),
        ),
      ]
    );
  }
}

Step 4: Remove the Processor

• You can remove the processor when you no longer need the effect. To do this, use the removeVideoProcessor() method from the VideoSDK class to eliminate the effect provided by the processor. This method will remove the effect of the currently active video processor.

import 'package:flutter/material.dart';
import 'package:videosdk/videosdk.dart';

class MeetingScreen extends StatefulWidget {
  //Existing configuration
}

class _MeetingScreenState extends State<MeetingScreen> {
  late Room _room;

  @override
  void initState() {
    //Existing configuration
  }

  @override
  Widget build(BuildContext context) {
    return Column(
      children:[
        // To apply the effect from the processor, provide the name used during registration.
        ElevatedButton(
          onPressed:(){
            VideoSDK.removeVideoProcessor();
          },
          child: const Text("Remove Video Processor"),
        ),
      ]
    );
  }
}

API Reference

The API references for all the methods utilized in this guide are provided below.

• registerVideoProcessor()
• applyVideoProcessor()
• removeVideoProcessor()