Optimize Audio Track - Android

While optimizing for the best listening experience, it is necessary to fine-tune the audio track that is being used during the calls.

You can pass a custom audio track for the participant's media before and after joining the room.

Custom Audio Track

Use this to create a custom audio track with a different encoder configuration and send it to other participants.

Once you publish a custom audio track created using createMicrophoneAudioTrack(), the SDK remembers its settings and reuses them whenever the mic is turned back on with unmuteMic(), for as long as you stay in the room. To change the track configuration, pass a new track created using createMicrophoneAudioTrack() to changeMic().

Encoder Config Reference

CustomMicrophoneTrackConfig	Auto Gain	Echo Cancellation	Noise Suppression
SPEECH_LOW_QUALITY	TRUE	TRUE	TRUE
SPEECH_STANDARD	TRUE	TRUE	TRUE
MUSIC_STANDARD	FALSE	FALSE	FALSE
STANDARD_STEREO	FALSE	FALSE	FALSE
HIGH_QUALITY	FALSE	FALSE	FALSE
HIGH_QUALITY_STEREO	FALSE	FALSE	FALSE

Note: If you face issues related to echo, you can pass a NoiseConfig with echoCancellation = true, as shown below. Echo cancellation attempts to prevent echo effects on a two-way audio connection by reducing or eliminating crosstalk between the user's output device and their input device — for example, filtering out sound produced by the speakers from being captured by the microphone input track.

How to Create Custom Audio Track ?

• Create an audio track with the createMicrophoneAudioTrack() method of VideoSDK. encoderConfig sets the audio profile.

• The mic opens when the track is created. Call stop() on the track to release the mic if you don't publish it.

• To capture from a specific microphone, such as a Bluetooth headset, pass its deviceId from VideoSDK.getMics() as microphoneId. The SDK switches to that microphone when the track opens, and again when you publish the track with unmuteMic() or changeMic().

• It throws VideoSDKException when the track can't be created, for example with 3018 (ERROR_MICROPHONE_ACCESS_DENIED_OR_DISMISSED) when the RECORD_AUDIO permission is not granted. The method doesn't ask for the permission, so request it first.

• It is a suspend function. It returns once the mic is open, and throws 3024 (ERROR_MICROPHONE_IN_USE) when the mic will not open or record. The work runs on a background thread, so you can call it from the main thread.

• A mic that fails after the track is created ends the track. Set onEnded on the track to find out.

• It throws IllegalStateException if VideoSDK.initialize() has not run.

Example

lifecycleScope.launch {
  try {
    val audioCustomTrack = VideoSDK.createMicrophoneAudioTrack(
      encoderConfig = CustomMicrophoneTrackConfig.SPEECH_STANDARD // CustomMicrophoneTrackConfig.HIGH_QUALITY | CustomMicrophoneTrackConfig.MUSIC_STANDARD, Default : CustomMicrophoneTrackConfig.SPEECH_STANDARD
    )
  } catch (e: VideoSDKException) {
    Log.e("VideoSDK", "Microphone track failed: ${e.code} ${e.name}")
  }
}

• SPEECH_STANDARD : This config is optimised for normal voice communication.

• HIGH_QUALITY : This config is used for getting RAW audio, where you can apply your noiseConfig.

• MUSIC_STANDARD : This config is optimised for communication, where sharing of musical notes such as songs or instrumental sounds, is important.

Audio processing with NoiseConfig

• Pass a NoiseConfig as the noiseConfig parameter to turn audio processing on or off for the track. It has four flags: noiseSuppression, echoCancellation, autoGainControl and highPassFilter.

• If you pass a NoiseConfig, it decides every flag. A flag you leave out takes its default: true for noiseSuppression, echoCancellation and autoGainControl, and false for highPassFilter. Set a flag to null to keep the value of the encoderConfig instead.

• If you don't pass one, the values of the encoderConfig in the table above are used.

lifecycleScope.launch {
  try {
    val audioCustomTrack = VideoSDK.createMicrophoneAudioTrack(
      encoderConfig = CustomMicrophoneTrackConfig.HIGH_QUALITY,
      // Turns echo cancellation on. null keeps the HIGH_QUALITY value of the other flags.
      noiseConfig = NoiseConfig(
        noiseSuppression = null,
        echoCancellation = true,
        autoGainControl = null,
        highPassFilter = null
      )
    )
  } catch (e: VideoSDKException) {
    Log.e("VideoSDK", "Microphone track failed: ${e.code} ${e.name}")
  }
}

How to Setup Custom Audio Track ?

The custom track can be set up both before and after the initialization of the room.

1. Setting up a Custom Track during the initialization of a room
2. Setting up a Custom Track with methods

1. Setting up a Custom Track during the initialization of a room

If you are passing micEnabled = true in the createRoom of VideoSDK and want to use custom tracks from the start of the room, you can pass the custom track as customMicrophoneTrack in the createRoom as shown below.

caution

The track is published when you join only if micEnabled is true. With micEnabled set to false, publish it later with unmuteMic(), or call stop() on it to release the mic.

Example

override fun onCreate(savedInstanceState: Bundle?) {
  //..

  lifecycleScope.launch {
    try {
      val audioCustomTrack = VideoSDK.createMicrophoneAudioTrack(encoderConfig = CustomMicrophoneTrackConfig.HIGH_QUALITY)

      // create a new room instance
      room = VideoSDK.createRoom(
        roomId = roomId,
        token = token,
        participantName = participantName,
        // If true, it will use the passed custom track to turn the mic on
        micEnabled = true,
        camEnabled = true,
        //Pass the custom track here
        customMicrophoneTrack = audioCustomTrack,
      )
    } catch (e: VideoSDKException) {
      Log.e("VideoSDK", "Room setup failed: ${e.code} ${e.name}")
    }
  }
}

2. Setting up a Custom Track with methods

To switch tracks during the call, pass the CustomStreamTrack to room.unmuteMic() while the mic is muted.

You can also pass custom track in changeMic() method of Room, while the mic is on. It turns the mic off and on again with the new track, so onStreamDisabled and then onStreamEnabled fire.

tip

unmuteMic() throws VideoSDKException with 3054 (ERROR_ALREADY_IN_REQUESTED_STATE) when the mic is already on. Use changeMic() to switch the track of a mic that is on.

Example

lifecycleScope.launch {
  try {
    val audioCustomTrack = VideoSDK.createMicrophoneAudioTrack(
      encoderConfig = CustomMicrophoneTrackConfig.HIGH_QUALITY
    )

    if (room!!.isMicEnabled) {
      room!!.changeMic(audioCustomTrack)
    } else {
      room!!.unmuteMic(audioCustomTrack)
    }
  } catch (e: VideoSDKException) {
    Log.e("VideoSDK", "Switching the mic track failed: ${e.code} ${e.name}")
  }
}

API Reference

The API references for all the methods and events utilised in this guide are provided below.

• Custom Audio Track
• unmuteMic()
• changeMic()