Video Processor - iOS

VideoSDK allows you to add custom effects to your video stream before it's transmitted during video calls. This lets you enhance your video in creative ways or achieve specific functionalities.

Understanding Video Processing

Video processing involves manipulating the raw video data (frames) captured by your camera. VideoSDK provides a tool called VideoSDKVideoProcessor to achieve this.

The VideoSDKVideoProcessor Protocol

VideoSDKVideoProcessor protocol defines a way for your code to interact with the video stream. It essentially lets you intercept each frame before it's sent and apply your processing logic.

Key Function: onFrameReceived

This is the heart of video processing with VideoSDK. Whenever a new video frame is captured by your camera, the onFrameReceived function is called. This function provides you with the raw frame data, allowing you to modify it as needed.

How it Works:

1. Implement the Protocol: You create a class that implements the VideoSDKVideoProcessor protocol. Within this class, you define the onFrameReceived function.

2. Process the Frame: Inside the onFrameReceived function, you access the raw video frame data. You can then apply your desired effects or manipulations to this data.

3. Return the Processed Frame: After processing the frame, return the modified version back to VideoSDK. This processed frame will then be transmitted during the video call.

Example: Grayscale Video

This example demonstrates a simple CustomVideoProcessor class that converts the video to grayscale:

class CustomVideoProcessor: VideoSDKVideoProcessor {

  func onFrameReceived(frame: RTCVideoFrame) -> RTCVideoFrame? {
    // Convert the frame to grayscale (replace with your desired processing)
    let processedFrame = convertToGrayscale(frame)
    return processedFrame
  }

  private func convertToGrayscale(_ frame: RTCVideoFrame) -> RTCVideoFrame? {
    // Implement grayscale conversion logic here
    var newFrame = frame
    // ... (grayscale conversion logic)
    return newFrame
  }
}

Setting the Video Processor

Once you have your custom processor class, use the setVideoProcessor method of the Meeting class to activate it. This tells VideoSDK to use your processor for processing the video stream.

Example:

class MeetingViewController {
  @IBAction func setProcessorTapped(_ sender: Any) {
    let processor = CustomVideoProcessor()
    meeting.setVideoProcessor(processor: processor)
  }
}

Removing the Processor

To stop applying the video effects, simply call setVideoProcessor again but pass nil as the argument. This will disable your custom processor and revert to the original video stream.