Ollama

LLM service implementation using Ollama with OpenAI-compatible interface

Overview

OLLamaLLMService provides access to locally-run Ollama models through an OpenAI-compatible interface. It inherits from BaseOpenAILLMService and allows you to run various open-source models locally while maintaining compatibility with OpenAI’s API format for privacy and cost control.

Ollama LLM API Reference

Pipecat’s API methods for Ollama integration

Example Implementation

Browse examples using Ollama models

Ollama Documentation

Official Ollama documentation and model library

Download Ollama

Download and setup instructions for Ollama

Installation

To use Ollama services, you need to install both Ollama and the Pipecat dependency:

1. Install Ollama on your system from ollama.com/download
2. Install Pipecat dependency:

pip install "pipecat-ai[ollama]"

3. Pull a model (first time only):

ollama pull llama2

Prerequisites

Ollama Local Setup

Before using Ollama LLM services, you need:

1. Ollama Installation: Download and install Ollama from ollama.com
2. Model Selection: Pull your desired models (llama2, mistral, codellama, etc.)
3. Local Service: Ensure Ollama service is running (default port 11434)
4. Hardware: Sufficient RAM and storage for your chosen models

Configuration

• No API Keys Required: Ollama runs entirely locally
• Model Management: Use ollama pull <model> to download models
• Service URL: Default is http://localhost:11434 (configurable)

Ollama runs as a local service on port 11434. No API key required for complete privacy!

Configuration

model · str · default: "None" · deprecated

The Ollama model to use. Must be pulled locally first with ollama pull.

Deprecated in v0.0.105. Use settings=OLLamaLLMService.Settings(model=...) instead.

settings · OLLamaLLMService.Settings · default: "None"

Runtime-configurable settings. See Settings below.

base_url · str · default: "http://localhost:11434/v1"

Base URL for the Ollama API endpoint.

Settings

Runtime-configurable settings passed via the settings constructor argument using OLLamaLLMService.Settings(...). These can be updated mid-conversation with LLMUpdateSettingsFrame. See Service Settings for details.

This service uses the same settings as OpenAILLMService. See OpenAI LLM Settings for the full parameter reference.

Usage

Basic Setup

from pipecat.services.ollama import OLLamaLLMService

llm = OLLamaLLMService(
    model="llama2",
)

With Custom Model and URL

from pipecat.services.ollama import OLLamaLLMService

llm = OLLamaLLMService(
    model="mistral",
    base_url="http://localhost:11434/v1",
)

With Custom Settings

from pipecat.services.ollama import OLLamaLLMService

llm = OLLamaLLMService(
    base_url="http://localhost:11434/v1",
    settings=OLLamaLLMService.Settings(
        model="mistral",
        temperature=0.7,
    ),
)

Notes

• No API key is required. The service automatically uses a placeholder key ("ollama") for OpenAI client compatibility.
• The Ollama service must be running locally before starting your pipeline. Start it with ollama serve if it is not already running.
• Model capabilities (function calling, vision, etc.) depend on the specific model you pull. Check the Ollama model library for details.

The InputParams / params= pattern is deprecated as of v0.0.105. Use Settings / settings= instead. See the Service Settings guide for migration details.