Video Rendering - iOS

The SDK has its own types for showing video. You get a participant's tracks from their RTCStream: track, videoTracks and audioTracks. You do not create tracks or frames yourself.

• To show a video track, add an RTCMTLVideoView to it, or your own view that conforms to RTCVideoRenderer.
• createRenderView() of RTCStream returns an RTCMTLVideoView that already shows the stream.

These types come from VideoSDKRTC. Only the members listed on this page are available on them.

RTCMediaStreamTrack

class RTCMediaStreamTrack

The base class of RTCVideoTrack and RTCAudioTrack.

trackId

• type: String
• The identifier of the track.

kind

• type: RTCMediaStreamTrackKind
• .audio or .video. A screen-share track is .video too. Use isShare, or the stream's kind, to tell a camera and a screen share apart.
• A switch on kind needs an @unknown default: case. Without it, Swift 5 warns and Swift 6 does not compile.

isEnabled

• type: Bool
• defaultValue: true
• On an RTCVideoTrack, false stops passing frames to the track's renderers on this device. The renderers keep showing the last frame.
• On a remote participant's RTCAudioTrack, false asks the server to stop sending you that participant's audio, and true resumes it. Other participants still hear them. To mute your own microphone, call muteMic().

readyState

• type: RTCMediaStreamTrackState
• .live or .ended. A video track becomes .ended when it ends for good, for example when the participant leaves the room, or when the custom camera track it belongs to ends. It does not become .live again.
• A video track stays .live while the participant turns their camera off and on.

---

RTCVideoTrack

final class RTCVideoTrack: RTCMediaStreamTrack

The video track of a participant's camera or screen share.

isShare

• type: Bool
• true for a screen-share track, and false for a camera track.

add()

func add(_ renderer: RTCVideoRenderer)

• Adds a renderer, such as an RTCMTLVideoView, so it starts to receive the track's frames.
• The track keeps a strong reference to the renderer until you call remove(_:). Remove the renderer when you stop showing the video, or it is not released.
• Adding the same renderer again does nothing. Adding a renderer to an ended track does nothing.

remove()

func remove(_ renderer: RTCVideoRenderer)

• Removes a renderer. The renderer gets one last renderFrame(nil) call, so it can clear its view.
• Removing a renderer that was not added does nothing.

Example

extension RoomViewController: ParticipantEventListener {
    func onStreamEnabled(_ stream: RTCStream, forParticipant participant: Participant) {
        if let videoTrack = stream.track as? RTCVideoTrack {
            videoTrack.add(participantVideoView)
        }
    }

    func onStreamDisabled(_ stream: RTCStream, forParticipant participant: Participant) {
        if let videoTrack = stream.track as? RTCVideoTrack {
            videoTrack.remove(participantVideoView)
        }
    }
}

---

RTCAudioTrack

final class RTCAudioTrack: RTCMediaStreamTrack

• The audio track of a participant's microphone. The SDK plays remote audio for you, so you do not need to add anything to it.
• Use isEnabled to stop and resume receiving a remote participant's audio.

---

RTCMTLVideoView

class RTCMTLVideoView: UIView, RTCVideoRenderer

• A view that shows the frames of an RTCVideoTrack. Add it to the track with add(_:), and remove it with remove(_:) when you stop showing the video.
• Create it in code with RTCMTLVideoView(frame:). In a storyboard or XIB, use a UIView, set its class to RTCMTLVideoView, and set Module to VideoSDKRTC in the Identity inspector.
• It cannot be subclassed.

videoContentMode

• type: UIView.ContentMode
• How the video fits the view, for example .scaleAspectFit or .scaleAspectFill.

isEnabled

• type: Bool
• defaultValue: true
• When false, the view ignores new frames and keeps showing the current one.

setMirror()

func setMirror(_ mirror: Bool)

• Flips the picture horizontally when true, for example for a front-camera self-view. Pass false to show frames as they arrive.
• Any transform your app set on the view is kept.
• Call it on the main thread.

Example

class ParticipantViewController: UIViewController {
    let videoView = RTCMTLVideoView(frame: .zero)

    override func viewDidLoad() {
        super.viewDidLoad()
        videoView.frame = view.bounds
        videoView.videoContentMode = .scaleAspectFill
        videoView.setMirror(true) // self-view from the front camera
        view.addSubview(videoView)
    }

    func showVideo(of stream: RTCStream) {
        if let videoTrack = stream.track as? RTCVideoTrack {
            videoTrack.add(videoView)
        }
    }

    func hideVideo(of stream: RTCStream) {
        if let videoTrack = stream.track as? RTCVideoTrack {
            videoTrack.remove(videoView)
        }
    }
}

---

RTCVideoRenderer

protocol RTCVideoRenderer: AnyObject

Conform to it to draw video frames yourself, for example with Metal, or to read them. Add your renderer to a track with add(_:).

setSize()

func setSize(_ size: CGSize)

• Called on the main thread when the frame size changes. add(_:) also calls it when the track already has a frame size.
• Use it to update your layout, for example the aspect ratio of your view.

renderFrame()

func renderFrame(_ frame: RTCVideoFrame?)

• Called for every frame. It can be called on a background thread, so return quickly and do not block.
• nil means clear your view: the video stopped, the renderer was removed, or the track ended.
• The frame's pixel buffer is valid only during the call. To use the image later, copy it.

Example

final class FrameSizeRenderer: RTCVideoRenderer {

    func setSize(_ size: CGSize) {
        print("Video size: \(size.width) x \(size.height)")
    }

    func renderFrame(_ frame: RTCVideoFrame?) {
        guard let frame else {
            // clear your view
            return
        }
        switch frame.buffer {
        case .cvPixelBuffer(let pixelBuffer):
            // draw the pixel buffer
            _ = pixelBuffer
        case .i420(let i420Buffer):
            // draw the I420 planes
            _ = i420Buffer
        @unknown default:
            break
        }
    }
}

---

RTCVideoFrame

struct RTCVideoFrame

A video frame that a renderer receives. Only the SDK creates frames.

• width: Int32 - The frame width, in pixels.
• height: Int32 - The frame height, in pixels.
• rotation: RTCVideoRotation - The clockwise rotation to apply before you show the frame: ._0, ._90, ._180 or ._270. The SDK sets it to ._0.
• timestampNs: Int64 - When the SDK delivered the frame, in nanoseconds. Compare it only with other frames of the same track.
• buffer: RTCVideoFrameBuffer - The pixels of the frame.

---

RTCVideoFrameBuffer

enum RTCVideoFrameBuffer

• .cvPixelBuffer(CVPixelBuffer): The pixels in a CVPixelBuffer. The SDK sends frames in this form. Read the pixel format with CVPixelBufferGetPixelFormatType(_:).
• .i420(RTCI420Buffer): The pixels as I420 planes, with width, height, dataY, dataU, dataV, strideY, strideU and strideV.
• width and height give the size of either form, in pixels.
• A switch on the buffer needs an @unknown default: case, as in the renderFrame() example.