Skip to main content

Agent Runtime with JavaScript

VideoSDK empowers you to seamlessly integrate AI agents with real-time voice interaction into your JavaScript application within minutes. This guide shows you how to connect a JavaScript frontend with an AI agent created and configured entirely from the VideoSDK dashboard.

Prerequisites​

Before proceeding, ensure that your development environment meets the following requirements:

important

You need a VideoSDK account to generate a token and an agent from the dashboard. Visit the VideoSDK dashboard to generate a token.

Project Structure​

First, create an empty project using mkdir folder_name on your preferable location for the JavaScript Frontend. Your final project structure should look like this:

Project Structure
  root
   ├── index.html
   ├── config.js
   └── index.js

You will be working on the following files:

  • index.html: Responsible for creating a basic UI for joining the meeting.
  • config.js: Responsible for storing the token, room ID, and agent credentials.
  • index.js: Responsible for rendering the meeting view and audio functionality.

Building the JavaScript Frontend​

Step 1: Install VideoSDK​

1
Install VideoSDK

Import VideoSDK using the <script> tag or install it using npm/yarn.

<html>
  <head>
    <!--.....-->
  </head>
  <body>
    <!--.....-->
    <script src="https://sdk.videosdk.live/js-sdk/0.3.6/videosdk.js"></script>
  </body>
</html>

Step 2: Design the User Interface​

2
Design the User Interface

Create an index.html file containing join-screen and grid-screen for audio-only interaction. The "Connect Agent" button will be used to call the AI agent into the meeting.

index.html
<!DOCTYPE html>
<html>
<head> </head>
<body>
    <div id="join-screen">
        <button id="createMeetingBtn">Join Agent Meeting</button>
    </div>
    <div id="textDiv"></div>
    <div id="grid-screen" style="display: none">
        <h3 id="meetingIdHeading"></h3>
        <button id="leaveBtn">Leave</button>
        <button id="toggleMicBtn">Toggle Mic</button>
        <button id="connectAgentBtn">Connect Agent</button>
        <div id="audioContainer"></div>
    </div>
    <script src="https://sdk.videosdk.live/js-sdk/0.3.6/videosdk.js"></script>
    <script src="config.js"></script>
    <script src="index.js"></script>
</body>
</html>

Step 3: Configure the Frontend​

3
Configure the Frontend

Create a meeting room using the VideoSDK API:

curl -X POST https://api.videosdk.live/v2/rooms \
  -H "Authorization: YOUR_JWT_TOKEN_HERE" \
  -H "Content-Type: application/json"

Copy the roomId from the response and configure it in config.js. You will get the Agent and Version IDs in the next section.

config.js
TOKEN = "your_videosdk_auth_token_here";
ROOM_ID = "YOUR_MEETING_ID";
AGENT_ID = "YOUR_AGENT_ID";
VERSION_ID = "YOUR_VERSION_ID";

Step 4: Implement Meeting Logic​

4
Implement Meeting Logic

In index.js, retrieve DOM elements, declare variables, and add the core meeting functionalities, including the logic to connect the agent.

index.js
// getting Elements from Dom
const leaveButton = document.getElementById("leaveBtn");
const toggleMicButton = document.getElementById("toggleMicBtn");
const createButton = document.getElementById("createMeetingBtn");
const connectAgentButton = document.getElementById("connectAgentBtn");
const audioContainer = document.getElementById("audioContainer");
const textDiv = document.getElementById("textDiv");

// declare Variables
let meeting = null;
let meetingId = "";
let isMicOn = false;

// Join Agent Meeting Button Event Listener
createButton.addEventListener("click", async () => {
  document.getElementById("join-screen").style.display = "none";
  textDiv.textContent = "Please wait, we are joining the meeting";
  meetingId = ROOM_ID;
  initializeMeeting();
});

// Initialize meeting
function initializeMeeting() {
  window.VideoSDK.config(TOKEN);

  meeting = window.VideoSDK.initMeeting({
    meetingId: meetingId,
    name: "C.V.Raman",
    micEnabled: true,
    webcamEnabled: false,
  });

  meeting.join();

  meeting.localParticipant.on("stream-enabled", (stream) => {
    if (stream.kind === "audio") {
      setAudioTrack(stream, meeting.localParticipant, true);
    }
  });

  meeting.on("meeting-joined", () => {
    textDiv.textContent = null;
    document.getElementById("grid-screen").style.display = "block";
    document.getElementById("meetingIdHeading").textContent = `Meeting Id: ${meetingId}`;
  });

  meeting.on("meeting-left", () => {
    audioContainer.innerHTML = "";
  });

  meeting.on("participant-joined", (participant) => {
    let audioElement = createAudioElement(participant.id);
    participant.on("stream-enabled", (stream) => {
      if (stream.kind === "audio") {
        setAudioTrack(stream, participant, false);
        audioContainer.appendChild(audioElement);
      }
    });
  });

  meeting.on("participant-left", (participant) => {
    let aElement = document.getElementById(`a-${participant.id}`);
    if (aElement) aElement.remove();
  });
}

// Create audio elements for participants
function createAudioElement(pId) {
  let audioElement = document.createElement("audio");
  audioElement.setAttribute("autoPlay", "false");
  audioElement.setAttribute("playsInline", "true");
  audioElement.setAttribute("controls", "false");
  audioElement.setAttribute("id", `a-${pId}`);
  audioElement.style.display = "none";
  return audioElement;
}

// Set audio track
function setAudioTrack(stream, participant, isLocal) {
  if (stream.kind === "audio") {
    if (isLocal) {
      isMicOn = true;
    } else {
      const audioElement = document.getElementById(`a-${participant.id}`);
      if (audioElement) {
        const mediaStream = new MediaStream();
        mediaStream.addTrack(stream.track);
        audioElement.srcObject = mediaStream;
        audioElement.play().catch((err) => console.error("audioElem.play() failed", err));
      }
    }
  }
}

// Implement controls
leaveButton.addEventListener("click", async () => {
  meeting?.leave();
  document.getElementById("grid-screen").style.display = "none";
  document.getElementById("join-screen").style.display = "block";
});

toggleMicButton.addEventListener("click", async () => {
  if (isMicOn) meeting?.muteMic();
  else meeting?.unmuteMic();
  isMicOn = !isMicOn;
});

connectAgentButton.addEventListener("click", async () => {
  try {
    const response = await fetch("https://api.videosdk.live/v2/agent/general/dispatch", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: TOKEN,
      },
      body: JSON.stringify({ agentId: AGENT_ID, meetingId: ROOM_ID, versionId: VERSION_ID }),
    });
    if (response.ok) {
      alert("Agent connected successfully!");
      connectAgentButton.style.display = "none";
    } else {
      alert("Failed to connect agent.");
    }
  } catch (error) {
    console.error("Error connecting agent:", error);
    alert("An error occurred while connecting the agent.");
  }
});

Creating the AI Agent from Dashboard (No-Code)​

You can create and configure a powerful AI agent directly from the VideoSDK dashboard.

Step 1: Create Your Agent​

1
Create Your Agent

First, follow our detailed guide to Build a Custom Voice AI Agent in Minutes. This will walk you through creating the agent's persona, configuring its pipeline (Realtime or Cascading), and testing it directly from the dashboard.

Step 2: Get Agent and Version ID​

2
Get Agent and Version ID

Once your agent is created, you need to get its agentId and versionId to connect it to your frontend application.

  1. After creating your agent, go to the agent's page and find the JSON editor on right side. Copy the agentId.

  2. To get the versionId, click on 3 dots besides Deploy button and click on "Version History" in it. Copy the version id via copy button of the version you want.

Get agentId and versionId

Step 3: Configure IDs in Frontend​

3
Configure IDs in Frontend

Now, update your config.js file with these IDs.

config.js
TOKEN = "your_videosdk_auth_token_here";
ROOM_ID = "YOUR_MEETING_ID";
AGENT_ID = "paste_your_agent_id_here";
VERSION_ID = "paste_your_version_id_here";

Run the Application​

Step 1: Start the Frontend​

1
Start the Frontend

Once you have completed all the steps, serve your frontend files:

# Using Python's built-in server
python3 -m http.server 8000

# Or using Node.js http-server
npx http-server -p 8000

Open http://localhost:8000 in your web browser.

Step 2: Connect and Interact​

2
Connect and Interact

  1. Join the meeting from the frontend:

    • Click the "Join Agent Meeting" button in your browser.
    • Allow microphone permissions when prompted.

  2. Connect the agent:

    • Once you join, click the "Connect Agent" button.
    • You should see an alert confirming the agent was connected.
    • The AI agent will join the meeting and greet you.

  3. Start playing:

    • Interact with your AI agent using your microphone.

Final Output​

You have completed the implementation of an AI agent with real-time voice interaction using VideoSDK and a no-code agent from the dashboard.

Troubleshooting​

Common Issues:​

  1. Agent not joining:

    • Check that the ROOM_ID, AGENT_ID, and VERSION_ID are correctly set in config.js.
    • Verify your VideoSDK token is valid and has the necessary permissions.

  2. Audio not working:

    • Check browser permissions for microphone access.

  3. "Failed to connect agent" error:

    • Verify your AGENT_ID and VERSION_ID are correct.
    • Check the browser's developer console for any network errors.

Got a Question? Ask us on discord

Previous
Agent Runtime with React
Next

Was this helpful?