xAI (Grok)

The xAI (Grok) provider enables your agent to use xAI's powerful Grok models for real-time, multimodal AI interactions.

Installation

Install the xAI-enabled VideoSDK Agents package:

pip install "videosdk-plugins-xai"

Authentication

The xAI plugin requires an xAI API key.

Set XAI_API_KEY in your .env file.

Importing

from videosdk.agents.plugins import XAIRealtime, XAIRealtimeConfig

Example Usage

from videosdk.agents.plugins import XAIRealtime, XAIRealtimeConfig
from videosdk.agents import Pipeline

# Initialize the xAI Grok real-time model
model = XAIRealtime(
    model="grok-4-1-fast-non-reasoning",
    # When XAI_API_KEY is set in .env - DON'T pass api_key parameter
    api_key="your-xai-api-key",
    config=XAIRealtimeConfig(
        voice="Eve",
        # collection_id="your-collection-id" # Optional
    )
)

# Create the pipeline with the model
pipeline = Pipeline(llm=model)

note

When using a .env file for credentials, don't pass them as arguments to model instances. The SDK automatically reads environment variables, so omit the api_key parameter from your code.

Key Features

• Multi-modal Interactions: Utilize xAI's powerful Grok models for voice and text.
• Function Calling: Define custom tools to retrieve weather data, interact with external APIs, or perform other actions.
• Web Search: Enable real-time web search capabilities by setting enable_web_search=True.
• X Search: Access X (formerly Twitter) content by setting enable_x_search=True and providing allowed_x_handles.

Configuration Options

• model: The Grok model to use (e.g., "grok-4-1-fast-non-reasoning").
• api_key: Your xAI API key (can also be set via the XAI_API_KEY environment variable).
• config: An XAIRealtimeConfig object for advanced options:
  • voice: (str) The voice to use for audio output. Options: "Ara", "Rex", "Sal", "Eve", "Leo" (default: "Ara").
  • instructions: (str or None) System instructions for the agent.
  • turn_detection: (XAITurnDetection or None) Configuration for server-side VAD to detect when a user has finished speaking.
  • modalities: (List[str]) List of enabled response types (default: ["text", "audio"]).
  • enable_web_search: (bool) Enable or disable web search capabilities (default: False).
  • enable_x_search: (bool) Enable or disable search on X (Twitter) (default: False).
  • allowed_x_handles: (List[str], optional) A list of allowed X handles to search within.
  • collection_id: (str, optional) The ID of a custom collection from your xAI Console storage to provide additional context.
  • max_num_results: (int) Maximum number of results to retrieve from a collection (file search) (default: 10).
• XAITurnDetection: Configuration for server-side voice activity detection, passed to turn_detection:
  • type: (str or None) The detection type (default: "server_vad").
  • threshold: (float) Voice activity detection threshold from 0.0 to 1.0 (default: 0.5).
  • prefix_padding_ms: (int) Padding before speech start in milliseconds (default: 300).
  • silence_duration_ms: (int) Silence duration to mark end of turn in milliseconds (default: 200).

Collection Storage

xAI Grok supports using "collections" to provide additional context to your agent, grounding its responses in your own documents or data.

To use a collection:

1. Navigate to xAI Console: Go to your console.x.ai dashboard.
2. Access Storage: Click on the Storage section in the sidebar.
3. Create New Collection: Click the "Create New Collection" button.
4. Upload Files: Upload your relevant documents or data files to the new collection.
5. Get Collection ID: Once the collection is created, copy its Collection ID.
6. Use in Config: Pass the copied ID to your agent's configuration:

config=XAIRealtimeConfig(
    voice="Eve",
    collection_id="your-collection-id-from-console",
    # ... other config options
)

The agent will now use the content of this collection to inform its responses.

Additional Resources

The following resources provide more information about using xAI (Grok) with the VideoSDK Agents SDK.

SDK Reference

GitHub Repository

Python Package