OpenAI LLM

The OpenAI LLM provider enables your agent to use OpenAI's language models (like GPT-4o) for text-based conversations and processing. It also supports vision input capabilities, allowing your agent to analyze and respond to images alongside text with the supported models.

Installation

Install the OpenAI-enabled VideoSDK Agents package:

pip install "videosdk-plugins-openai"

Importing

from videosdk.agents.plugins import OpenAILLM

Authentication

The OpenAI plugin requires an OpenAI API key.

Set OPENAI_API_KEY in your .env file.

Example Usage

from videosdk.agents.plugins import OpenAILLM
from videosdk.agents import Pipeline

llm = OpenAILLM(
    model="gpt-4o",
    temperature=0.7,
    top_p=0.95,
    seed=42,
    parallel_tool_calls=True,
    max_completion_tokens=1024,
)

pipeline = Pipeline(llm=llm)

note

When using a .env file for credentials, don't pass them as arguments to model instances. The SDK automatically reads environment variables, so omit api_key and other credential parameters from your code.

Configuration Options

Core

• model — The OpenAI model to use (e.g. "gpt-4o", "gpt-4o-mini"). Default: "gpt-4o-mini".
• api_key — Your OpenAI API key. Falls back to the OPENAI_API_KEY environment variable.
• base_url — Custom base URL for the OpenAI API (optional).
• temperature — Sampling temperature (0.0 – 2.0). Default: 0.7.
• tool_choice — Tool selection mode: "auto", "required", "none", or a dict {"type": "function", "function": {"name": "my_tool"}} to force a specific tool. Default: "auto".
• max_completion_tokens — Maximum tokens in the completion response (optional).

Generation knobs

• top_p — Nucleus sampling: only the tokens comprising the top top_p probability mass are considered (float, optional).
• frequency_penalty — Penalises tokens that appear frequently in the response so far; reduces repetition (float, -2.0 – 2.0, optional).
• presence_penalty — Penalises tokens that have appeared at all in the response so far; encourages new topics (float, -2.0 – 2.0, optional).
• seed — Integer seed for deterministic sampling. The same seed + same inputs will produce the same output (optional).
• reasoning_effort — Controls reasoning depth for reasoning / GPT-5 models: "none", "low", "medium", or "high". Only applied for reasoning models. Default: None (uses the model's default).
• verbosity — Controls output verbosity for reasoning / GPT-5 models: "low", "medium", or "high" (optional).

Organisation and project

• organization — Your OpenAI organisation ID. Falls back to the OPENAI_ORG_ID environment variable (optional).
• project — Your OpenAI project ID. Falls back to the OPENAI_PROJECT_ID environment variable (optional).

Tool calling

• parallel_tool_calls — When True, allows the model to call multiple tools in a single turn. When False, forces one tool call at a time. Default: provider default (optional).

HTTP and client

• timeout — Custom httpx.Timeout for the underlying HTTP client (optional).
• extra_headers — Additional HTTP headers forwarded to every API call (dict, optional).
• extra_query — Additional query-string parameters forwarded to every API call (dict, optional).
• extra_body — Additional JSON body fields forwarded to every API call (dict, optional).
• client — Optional pre-built openai.AsyncOpenAI instance to use instead of creating a new one. When provided, api_key, base_url, organization, project, timeout, and max_retries are ignored (optional).
• max_retries — Number of automatic retries on transient errors. Default: 0.

Streaming (WebSocket Responses API)

• streaming — When True, use OpenAI's WebSocket Responses API instead of the standard HTTP chat completions endpoint. The connection is reused across turns for lower per-turn latency. Default: False (HTTP mode).
• store — Only used when streaming=True. Controls whether responses are persisted server-side. Default: False (ZDR-friendly).
• wss_url — Override the WebSocket Responses URL. Defaults to OpenAI's public endpoint (optional).

Advanced Example

from videosdk.agents.plugins import OpenAILLM
from videosdk.agents import Pipeline

llm = OpenAILLM(
    model="gpt-4o",
    temperature=0.7,
    top_p=0.95,
    frequency_penalty=0.1,
    presence_penalty=0.1,
    seed=42,
    parallel_tool_calls=True,
    max_completion_tokens=2048,
)

pipeline = Pipeline(llm=llm)

Additional Resources

• OpenAI docs: OpenAI documentation.

SDK Reference

GitHub Repository

Python Package