OpenAI

Large Language Model services using OpenAI’s chat completion API

Overview

OpenAILLMService provides chat completion capabilities using OpenAI’s API, supporting streaming responses, function calling, vision input, and advanced context management for conversational AI applications with state-of-the-art language models.

OpenAI LLM API Reference

Pipecat’s API methods for OpenAI integration

Example Implementation

Function calling example with weather API

OpenAI Documentation

Official OpenAI API documentation

OpenAI Platform

Access models and manage API keys

Installation

To use OpenAI services, install the required dependencies:

pip install "pipecat-ai[openai]"

Prerequisites

OpenAI Account Setup

Before using OpenAI LLM services, you need:

1. OpenAI Account: Sign up at OpenAI Platform
2. API Key: Generate an API key from your account dashboard
3. Model Selection: Choose from available models (GPT-4.1, GPT-4o, GPT-4o-mini, etc.)
4. Usage Limits: Set up billing and usage limits as needed

Required Environment Variables

• OPENAI_API_KEY: Your OpenAI API key for authentication

Configuration

model · str · default: "None" · deprecated

OpenAI model name to use (e.g., "gpt-4.1", "gpt-4o", "gpt-4o-mini"). Deprecated in v0.0.105. Use settings=OpenAILLMService.Settings(...) instead.

api_key · str · default: "None"

OpenAI API key. If None, uses the OPENAI_API_KEY environment variable.

base_url · str · default: "None"

Custom base URL for the OpenAI API. Override for proxied or self-hosted deployments.

organization · str · default: "None"

OpenAI organization ID.

project · str · default: "None"

OpenAI project ID.

default_headers · Mapping[str, str] · default: "None"

Additional HTTP headers to include in every request.

settings · OpenAILLMService.Settings · default: "None"

Runtime-configurable model settings. See Settings below.

params · InputParams · default: "None" · deprecated

Runtime-configurable model settings. See Settings below. Deprecated in v0.0.105. Use settings=OpenAILLMService.Settings(...) instead.

retry_timeout_secs · float · default: "5.0"

Request timeout in seconds. Used when retry_on_timeout is enabled to determine when to retry.

retry_on_timeout · bool · default: "False"

Whether to retry the request once if it times out. The retry attempt has no timeout limit.

Settings

Runtime-configurable settings passed via the settings constructor argument using OpenAILLMService.Settings(...). These can be updated mid-conversation with LLMUpdateSettingsFrame. See Service Settings for details.

Parameter	Type	Default	Description
model	str	None	OpenAI model identifier. (Inherited from base settings.)
system_instruction	str	None	System instruction/prompt for the model. (Inherited from base settings.)
temperature	float	NOT_GIVEN	Sampling temperature (0.0 to 2.0). Lower values are more focused, higher values are more creative.
max_tokens	int	NOT_GIVEN	Maximum tokens to generate.
top_p	float	NOT_GIVEN	Top-p (nucleus) sampling (0.0 to 1.0). Controls diversity of output.
top_k	int	NOT_GIVEN	Top-k sampling parameter.
frequency_penalty	float	NOT_GIVEN	Penalty for frequent tokens (-2.0 to 2.0). Positive values discourage repetition.
presence_penalty	float	NOT_GIVEN	Penalty for new topics (-2.0 to 2.0). Positive values encourage the model to talk about new topics.
seed	int	NOT_GIVEN	Random seed for deterministic outputs.
max_completion_tokens	int	NOT_GIVEN	Maximum completion tokens to generate.

NOT_GIVEN values are omitted from the API request entirely, letting the OpenAI API use its own defaults. This is different from None, which would be sent explicitly.

Usage

Basic Setup

from pipecat.services.openai import OpenAILLMService

llm = OpenAILLMService(
    api_key=os.getenv("OPENAI_API_KEY"),
    model="gpt-4o",
)

With Custom Settings

from pipecat.services.openai import OpenAILLMService

llm = OpenAILLMService(
    api_key=os.getenv("OPENAI_API_KEY"),
    settings=OpenAILLMService.Settings(
        model="gpt-4.1",
        temperature=0.7,
        max_completion_tokens=1000,
        frequency_penalty=0.5,
    ),
)

Updating Settings at Runtime

Model settings can be changed mid-conversation using LLMUpdateSettingsFrame:

from pipecat.frames.frames import LLMUpdateSettingsFrame
from pipecat.services.openai.base_llm import OpenAILLMSettings

await task.queue_frame(
    LLMUpdateSettingsFrame(
        delta=OpenAILLMSettings(
            temperature=0.3,
            max_completion_tokens=500,
        )
    )
)

The InputParams / params= pattern is deprecated as of v0.0.105. Use Settings / settings= instead. See the Service Settings guide for migration details.

Notes

• OpenAI-compatible providers: Many third-party LLM providers offer OpenAI-compatible APIs. You can use OpenAILLMService with them by setting base_url to the provider’s endpoint.
• Retry behavior: When retry_on_timeout=True, the first attempt uses the retry_timeout_secs timeout. If it times out, a second attempt is made with no timeout limit.
• Function calling: Supports OpenAI’s tool/function calling format. Register function handlers on the pipeline task to handle tool calls automatically.
• System instruction precedence: If both system_instruction (from the constructor) and a system message in the context are set, the constructor’s system_instruction takes precedence and a warning is logged.

Event Handlers

OpenAILLMService supports the following event handlers, inherited from LLMService:

Event	Description
on_completion_timeout	Called when an LLM completion request times out
on_function_calls_started	Called when function calls are received and execution is about to start

@llm.event_handler("on_completion_timeout")
async def on_completion_timeout(service):
    print("LLM completion timed out")