Optimize Audio Track - React

To optimize the listening experience, it's essential to fine-tune the audio tracks used during calls.

For an enhanced fine-tuning experience, we've introduced the capability to provide a custom audio track for a participant's media before and during the meeting.

Custom Audio Track

This feature allows you to incorporate custom layers like background noise removal, echo cancellation, etc. and send these modifications to other participants.

Encoder Config Reference

Encoder Config	Bitrate	Auto Gain	Echo Cancellation	Noise Suppression
speech_low_quality	16 kbps	TRUE	TRUE	TRUE
speech_standard	24 kbps	TRUE	TRUE	TRUE
music_standard	32 kbps	FALSE	FALSE	FALSE
standard_stereo	64 kbps	FALSE	FALSE	FALSE
high_quality	128 kbps	FALSE	FALSE	FALSE
high_quality_stereo	192 kbps	FALSE	FALSE	FALSE

note

If you face issues related to echo, you can use noiseConfig with echoCancellation: true. Echo cancellation attempts to prevent echo effects on a two-way audio connection by reducing or eliminating crosstalk between the user's output device and their input device — for example, filtering out sound produced by the speakers from being captured by the microphone input track.

How to Create a Custom Audio Track ?

• You can create a Custom Audio Track using createMicrophoneAudioTrack() method of @videosdk.live/react-sdk.

• This method enables the creation of a audio track with various encoding parameters and noise cancellation configuration, ultimately returning a MediaStream.

Example

import { createMicrophoneAudioTrack } from "@videosdk.live/react-sdk";

let customTrack = await createMicrophoneAudioTrack({
  // It will be the id of the mic from which the voice should be captured.
  microphoneId : 'mic-id' // OPTIONAL

  // This will accept the voice profile you want to capture.
  encoderConfig: "speech_standard", // `high_quality` | `music_standard`,  Default : `speech_standard`

  noiseConfig: {
  // It is used to improve the quality of audio by removing background noise
  // that can interfere with the clarity of speech.
    noiseSuppression: true,

  // It is used to remove unwanted echoes from voice.
    echoCancellation: true,

  // It is used to maintain a consistent level of loudness or amplitude in a voice.
    autoGainControl: true,
  },
});

Here are different configurations for customizing audio tracks based on specific use cases:

• speech_standard : This config is optimised for normal voice communication.

• high_quality : This config is used for obtaining RAW audio, allowing you to apply your noiseConfig.

• music_standard : This config is optimised for communication scenarios, where the sharing of musical elements, such as songs or instrumental sounds, is crucial.

How to Setup Custom Audio Track ?

The custom track can be configured both before and after the meeting is initialized. Following are the methods that help in doing so:

1. Setting up a Custom Track during the initialization of a meeting
2. Setting up a Custom Track with methods

1. Setting up a Custom Track during the initialization of a meeting

If you are enabling the mic (micEnabled: true) in the config of MeetingProvider and wish to use custom tracks from the start of the meeting, you can pass a custom track in the config as demonstrated below.

caution

Custom Track will not apply on micEnabled: false configuration.

Example

import {
  createMicrophoneAudioTrack,
  MeetingProvider,
} from "@videosdk.live/react-sdk";

function App() {
  const getTrack = async () => {
    const track = await createMicrophoneAudioTrack({
      encoderConfig: "speech_standard",
      noiseConfig: {
        noiseSuppression: true,
        echoCancellation: true,
        autoGainControl: true,
      },
    });
    setCustomTrack(track);
  };

  let [customTrack, setCustomTrack] = useState();

  useEffect(() => {
    getTrack();
  }, []);

  return (
    customTrack && (
      <MeetingProvider
        config={{
          meetingId,
          micEnabled: true, //If true, it will use the passed custom track to turn mic on
          webcamEnabled: true,
          //Pass the custom audio track here
          customMicrophoneAudioTrack: customTrack,
        }}
        token={token}
      >
        <MeetingView />
      </MeetingProvider>
    )
  );
}

2. Setting up a Custom Track with methods

To switch tracks during the meeting, you need to pass the MediaStream in the unmuteMic() or toggleMic() method of the useMeeting hook.

tip

Make sure to call the muteMic() method before you create a new track as it may lead to unexpected behavior.

Example

import {
  createMicrophoneAudioTrack,
  useMeeting,
} from "@videosdk.live/react-sdk";

const MeetingControls = () => {
  const { localMicOn, unmuteMic, muteMic, toggleMic } = useMeeting();

  const handleToggleMic = async () => {
    if (localMicOn) {
      toggleMic();
    } else {
      let customTrack = await createMicrophoneAudioTrack({
        encoderConfig: "speech_standard",
        noiseConfig: {
          noiseSuppression: true,
          echoCancellation: true,
          autoGainControl: true,
        },
      });

      toggleMic(customTrack);
    }
  };

  const handleUnmuteMic = async () => {
    if (localMicOn) {
      muteMic();
    } else {
      let customTrack = await createMicrophoneAudioTrack({
        encoderConfig: "speech_standard",
        noiseConfig: {
          noiseSuppression: true,
          echoCancellation: true,
          autoGainControl: true,
        },
      });

      unmuteMic(customTrack);
    }
  };

  return (
    <>
      <button onClick={handleToggleMic}>Toggle Mic</button>
      <button onClick={handleUnmuteMic}>Unmute Mic</button>
    </>
  );
};

API Reference

The API references for all the methods and events utilized in this guide are provided below.

• Custom Audio Track