Google LLM

The Google LLM provider enables your agent to use Google's Gemini family of language models for text-based conversations and processing. It also supports vision input capabilities, allowing your agent to analyze and respond to images alongside text with the supported models.

Installation

Install the Google-enabled VideoSDK Agents package:

pip install "videosdk-plugins-google"

Importing

from videosdk.plugins.google import GoogleLLM

Authentication

The Google plugin requires an Gemini API key.

Set GOOGLE_API_KEY in your .env file.

Example Usage

from videosdk.plugins.google import GoogleLLM
from videosdk.agents import CascadingPipeline

# Initialize the Google LLM model
llm = GoogleLLM(
    model="gemini-3-flash-preview",
    # When GOOGLE_API_KEY is set in .env - DON'T pass api_key parameter
    api_key="your-google-api-key",
    temperature=0.7,
    tool_choice="auto",
    max_output_tokens=1000
)

#  Add llm to cascading pipeline
pipeline = CascadingPipeline(llm=llm)

note

When using .env file for credentials, don't pass them as arguments to model instances or context objects. The SDK automatically reads environment variables, so omit api_key and other credential parameters from your code.

Vertex AI Integration

You can also use Google's Gemini models through Vertex AI. This requires a different authentication and configuration setup.

Authentication for Vertex AI

For Vertex AI, you need to set up Google Cloud credentials. Create a service account, download the JSON key file, and set the path to this file in your environment.

export GOOGLE_APPLICATION_CREDENTIALS="/path/to/your/keyfile.json"

You should also configure your project ID and location. These can be set as environment variables or directly in the code. If not set, the project_id is inferred from the credentials file and the location defaults to us-central1.

export GOOGLE_CLOUD_PROJECT="your-gcp-project-id"
export GOOGLE_CLOUD_LOCATION="your-gcp-location"

Example Usage with Vertex AI

To use Vertex AI, set vertexai=True when initializing GoogleLLM. You can configure the project and location using VertexAIConfig, which will take precedence over environment variables.

from videosdk.plugins.google import GoogleLLM, VertexAIConfig
from videosdk.agents import CascadingPipeline
# Import other necessary components like STT and TTS
# from videosdk.plugins.deepgram import DeepgramSTT
# from videosdk.plugins.elevenlabs import ElevenLabsTTS

# Initialize GoogleLLM with Vertex AI configuration
llm = GoogleLLM(
    vertexai=True,
    vertexai_config=VertexAIConfig(
        project_id="videosdk",
        location="us-central1"
    )
)

# Add llm to a cascading pipeline
pipeline = CascadingPipeline(
    stt=DeepgramSTT(), # Example STT
    llm=llm,
    tts=ElevenLabsTTS() # Example TTS
)

Configuration Options

• model: (str) The Google model to use (e.g., "gemini-3-flash-preview", "gemini-3-pro-preview", "gemini-2.0-flash-001",) (default: "gemini-2.0-flash-001").
• api_key: (str) Your Google API key. Can also be set via the GOOGLE_API_KEY environment variable.
• temperature: (float) Sampling temperature for response randomness (default: 0.7).
• tool_choice: (ToolChoice) Tool selection mode ("auto", "required", "none") (default: "auto").
• max_output_tokens: (int) Maximum number of tokens in the completion response (optional).
• top_p: (float) The nucleus sampling probability (optional).
• top_k: (int) The top-k sampling parameter (optional).
• presence_penalty: (float) Penalizes new tokens based on whether they appear in the text so far (optional).
• frequency_penalty: (float) Penalizes new tokens based on their existing frequency in the text so far (optional).

Additional Resources

The following resources provide more information about using Google with VideoSDK Agents SDK.

• Gemini docs: Google Gemini documentation.

SDK Reference

GitHub Repository

Python Package