Optimize Audio Track - Flutter

While optimizing for the best listening experience, it is necessary to fine-tune the audio track that is being used during the calls.

You can pass a custom audio track for the participant's media before and during the room.

Custom Audio Track

This feature can be used to add custom layers like background noise removal, echo cancellation, etc. on audio and send it to other participants.

Once you set a custom audio track created using createMicrophoneAudioTrack(), the SDK remembers its settings and automatically reuses them whenever the mic is turned back on. To change the track configuration, call muteMic(), then pass a new track created using createMicrophoneAudioTrack() to unmuteMic().

Encoder Config Reference

Encoder Config	Bitrate	Auto Gain	Echo Cancellation	Noise Suppression
SPEECH_LOW_QUALITY	24 kbps	TRUE	TRUE	TRUE
SPEECH_STANDARD	24 kbps	TRUE	TRUE	TRUE
MUSIC_STANDARD	40 kbps	FALSE	FALSE	FALSE
STANDARD_STEREO	64 kbps	FALSE	FALSE	FALSE
HIGH_QUALITY	128 kbps	FALSE	FALSE	FALSE
HIGH_QUALITY_STEREO	192 kbps	FALSE	FALSE	FALSE

note

If you face issues related to echo, you can pass NoiseConfig(echoCancellation: true) as noiseConfig. Echo cancellation attempts to prevent echo effects on a two-way audio connection by reducing or eliminating crosstalk between the user's output device and their input device — for example, filtering out sound produced by the speakers from being captured by the microphone input track.

How to Create Custom Audio Track ?

• You can create a Audio Track using createMicrophoneAudioTrack() method of VideoSDK.

• This method can be used to create audio track using different encoding parameters and noise cancellation configuration.

• This method does not ask for microphone permission, so call VideoSDK.requestPermissions() before you create the track.

• If the track cannot be created, the method throws a VideoSDKError, for example 3018 without microphone permission, 3022 when there is no microphone, 3024 when the microphone is in use, or 3071 for an unknown microphoneId.

• The microphone opens when the track is created. If you do not use the track, call stop() on it to close the microphone.

Example

import 'package:videosdk/videosdk.dart';

// The SDK does not ask for microphone permission.
final permissions = await VideoSDK.requestPermissions(Permissions.AUDIO);
if (permissions['audio'] != true) return;

try {
  CustomStreamTrack customTrack = await VideoSDK.createMicrophoneAudioTrack(
    // It will be the id of the mic from which the voice should be captured.
    microphoneId: 'mic-id', // OPTIONAL: the deviceId of a mic from VideoSDK.getMics()

    // This will accept the voice profile you want to capture.
    encoderConfig: CustomMicrophoneTrackConfig.SPEECH_STANDARD, // `HIGH_QUALITY` | `MUSIC_STANDARD`,  Default : `SPEECH_STANDARD`

    noiseConfig: NoiseConfig(
      // It is used to improve the quality of audio by removing background noise
      // that can interfere with the clarity of speech.
      noiseSuppression: true,

      // It is used to remove unwanted echoes from voice.
      echoCancellation: true,

      // It is used to maintain a consistent level of loudness or amplitude in a voice.
      autoGainControl: true,

      // It is used to cut low-frequency noise, such as hum. Default : false
      highPassFilter: false,
    ),
  );
} on VideoSDKError catch (error) {
  // For example 3018 (permission denied), 3022 (no microphone) or 3024 (microphone in use)
  print("Could not create the track: ${error.code} ${error.name}");
}

• SPEECH_STANDARD : This config is optimised for normal voice communication.

• HIGH_QUALITY : This config is used for getting RAW audio, where you can apply your noiseConfig.

• MUSIC_STANDARD : This config is optimised for communication, where sharing of musical notes such as songs or instrumental sounds, is important.

If you pass a NoiseConfig, every flag you leave out is turned on, except highPassFilter, which stays off. Pass no noiseConfig to use the values of the encoderConfig profile.

How to Setup Custom Audio Track ?

The custom track can be set up both before and after the initialization of the room.

1. Setting up a Custom Track during the initialization of a room
2. Setting up a Custom Track with methods

1. Setting up a Custom Track during the initialization of a room

If you are passing micEnabled: true in the createRoom and want to use custom tracks from start of the room, you can pass custom track in the customMicrophoneTrack as shown below.

caution

Custom Track will not apply on micEnabled: false configuration.

Example

import 'dart:async' show unawaited;
import 'package:flutter/material.dart';
import 'package:videosdk/videosdk.dart';

class RoomScreen extends StatefulWidget {
  final String roomId;
  final String token;

  const RoomScreen(
      {super.key, required this.roomId, required this.token});

  @override
  State<RoomScreen> createState() => _RoomScreenState();
}

class _RoomScreenState extends State<RoomScreen> {
  late Room _room;

  @override
  void initState() {
    super.initState();
    unawaited(initRoom());
  }

  Future<void> initRoom() async {
    // On Android, the SDK does not ask for camera and microphone permission.
    await VideoSDK.requestPermissions();

    //Creating Custom Audio Track
    CustomStreamTrack? audioTrack;
    try {
      audioTrack = await VideoSDK.createMicrophoneAudioTrack(
          encoderConfig: CustomMicrophoneTrackConfig.SPEECH_STANDARD);
    } on VideoSDKError catch (error) {
      print("Could not create the track: ${error.code} ${error.name}");
    }

    //Creating a new Room with custom audio track
    // create room
    _room = VideoSDK.createRoom(
      roomId: widget.roomId,
      token: widget.token,
      participantName: "John Doe",
      micEnabled: true,
      camEnabled: true,
      defaultCamera: FacingMode.FRONT, // camera used when the room starts
      customMicrophoneTrack: audioTrack, // custom audio track :: optional
    );
  }

  @override
  Widget build(BuildContext context) {
    return YourRoomWidget();
  }
}

2. Setting up a Custom Track with methods

In order to switch tracks during the room, you have to pass the CustomStreamTrack in the unmuteMic() method of Room. You can also switch to it in one step with changeMic(customMicrophoneTrack: audioTrack), which unmutes the mic.

tip

Call muteMic() before you create a new track. While the mic is on, unmuteMic() with a track throws 3054 and releases the track.

Example

import 'dart:async' show unawaited;
import 'package:flutter/material.dart';
import 'package:videosdk/videosdk.dart';

class RoomScreen extends StatefulWidget {
  final String roomId;
  final String token;

  const RoomScreen(
      {super.key, required this.roomId, required this.token});

  @override
  State<RoomScreen> createState() => _RoomScreenState();
}

class _RoomScreenState extends State<RoomScreen> {
  late Room _room;

  @override
  void initState() {
    super.initState();
    unawaited(initRoom());
  }

  Future<void> initRoom() async {
    ...
  }

  @override
  Widget build(BuildContext context) {
    return Column(
      children: [
        ElevatedButton(
          onPressed: () async {
            try {
              // The mic must be off before you pass a new track
              if (_room.isMicEnabled) await _room.muteMic();

              //Creating Custom Audio Track
              CustomStreamTrack audioTrack = await VideoSDK.createMicrophoneAudioTrack(
                  encoderConfig: CustomMicrophoneTrackConfig.SPEECH_STANDARD);

              await _room.unmuteMic(audioTrack);
            } on VideoSDKError catch (error) {
              print("Could not unmute with the custom track: ${error.code} ${error.name}");
            }
          },
          child: const Text("Unmute with Custom Track"),
        ),
      ],
    );
  }
}

API Reference

The API references for all the methods and events utilised in this guide are provided below.

• Custom Audio Track
• requestPermissions()
• unmuteMic()
• muteMic()