Simli Avatar

The Simli Avatar plugin allows you to integrate a real-time, lip-synced AI avatar into your VideoSDK agent. This creates a more engaging and interactive experience for users by providing a visual representation of the AI agent.

Simli offers two avatar types: Legacy (30 FPS) and Trinity (25 FPS). When creating a SimliAvatar, set is_trinity_avatar=True if you're using a Trinity avatar (default is False). Always select the correct faceID from the Simli dashboard.

Installation

Install the Simli-enabled VideoSDK Agents package:

pip install "videosdk-plugins-simli"

Authentication

The Simli plugin requires an Simli API key.

Set SIMLI_API_KEY in your .env file.

Importing

from videosdk.agents.plugins import SimliAvatar, SimliConfig

Setup Credentials

To use Simli, you need an API key. You can get one from the Simli Dashboard.

Set up your credentials by exporting them as an environment variable:

export SIMLI_API_KEY="YOUR_SIMLI_API_KEY"

You can also provide a faceId if you have a custom one.

export SIMLI_FACE_ID="YOUR_FACE_ID"

Example Usage

Here's how you can integrate the Simli Avatar with Pipeline in both cascading and realtime modes.

Cascade Mode

This example shows how to add the Simli Avatar to a Pipeline (cascading mode).

import os
from videosdk.agents import Pipeline
from videosdk.agents.plugins import SimliAvatar, SimliConfig
# Import other necessary components like STT, LLM, TTS

# 1. Initialize SimliConfig
simli_config = SimliConfig(
    apiKey=os.getenv("SIMLI_API_KEY"),
    faceId=os.getenv("SIMLI_FACE_ID"), # This is optional and has a default value
)

# 2. Create a SimliAvatar instance
# For Legacy avatars (default)
simli_avatar = SimliAvatar(config=simli_config)

# For Trinity avatars
# simli_avatar = SimliAvatar(
#     config=simli_config,
#     is_trinity_avatar=True,
# )

# 3. Add the avatar to the pipeline
pipeline = Pipeline(
    # ... stt=stt, llm=llm, tts=tts
    avatar=simli_avatar
)

Real-time Pipeline

This example shows how to add the Simli Avatar to a Pipeline (realtime mode).

import os
from videosdk.agents import Pipeline
from videosdk.agents.plugins import SimliAvatar, SimliConfig
# from videosdk.agents.plugins import GeminiRealtime # Example model

# 1. Initialize SimliConfig
simli_config = SimliConfig(
    apiKey=os.getenv("SIMLI_API_KEY"),
)

# 2. Create a SimliAvatar instance
# For Legacy avatars (default)
simli_avatar = SimliAvatar(config=simli_config)

# For Trinity avatars
# simli_avatar = SimliAvatar(
#     config=simli_config,
#     is_trinity_avatar=True,
# )

# 3. Add the avatar to the pipeline
pipeline = Pipeline(
    llm=your_realtime_model, # e.g., GeminiRealtime()
    avatar=simli_avatar
)

note

When using an environment variable for credentials, you should still load it in your code using os.getenv("SIMLI_API_KEY") and pass it to SimliConfig.

Configuration Options

You can customize the avatar's behavior using the SimliConfig and SimliAvatar classes.

SimliConfig

• faceId: (str, required) The ID for the avatar face. You can find available faces in the Simli Docs or create your own. For a Trinity avatar you can specify "faceId/emotionId" to use a non-default emotion.
• handleSilence: (bool, optional) When True, the Simli server keeps sending silent video when the input buffer is depleted; turning it off freezes the video when no audio is sent. Defaults to True.
• maxSessionLength: (int, optional) Absolute maximum session duration in seconds; the avatar disconnects after this time even if it is speaking. Defaults to 600 (10 minutes).
• maxIdleTime: (int, optional) Maximum duration in seconds the avatar can be not speaking before it disconnects. Defaults to 30.
• model: (SimliModels, optional) The Simli model to use (SimliModels.fasttalk or SimliModels.artalk). Defaults to SimliModels.fasttalk.

SimliAvatar

• config: (SimliConfig) A SimliConfig object with your desired settings.
• api_key: (str, optional) Your Simli API key. Falls back to the SIMLI_API_KEY environment variable if not provided.
• transport_mode: (str, optional) The transport mode to use, either "P2P" or "LIVEKIT". Defaults to "P2P".
• simli_url: (str, optional) The Simli API URL. Defaults to "https://api.simli.ai".
• is_trinity_avatar: (bool, optional) Set to True when using Trinity avatars. Defaults to False for Legacy avatars.

Additional Resources

The following resources provide more information about using Simli with VideoSDK Agents SDK.

• Simli docs: Simli docs.

SDK Reference

GitHub Repository

Python Package