LLM Integration

Overview

Compozy agents integrate with multiple LLM providers through a unified interface, allowing seamless switching between models and providers. This abstraction enables you to choose the best model for each task while maintaining consistent configuration patterns.

Supported Providers

Provider Configuration

Basic Configuration

Each agent requires a provider configuration that specifies the LLM to use. The model property accepts any valid model string for the chosen provider - it's not limited to a predefined list:

config:
  provider: openai
  model: gpt-4-turbo-preview
  api_key: "{{ .env.OPENAI_API_KEY }}"
  params:
    temperature: 0.7
    max_tokens: 2000

Provider-Specific Examples

config:
  provider: openai
  model: gpt-4-turbo-preview
  api_key: "{{ .env.OPENAI_API_KEY }}"
  params:
    temperature: 0.7
    max_tokens: 4000
    top_p: 0.9
    frequency_penalty: 0.1
    presence_penalty: 0.1

Model Parameters

Common Parameters

Most providers support these standard parameters. For the complete and most up-to-date parameter schema, see the Provider Schema Documentation.

Provider-Specific Parameters

Global Model Configuration

Define reusable model configurations in your project's compozy.yaml:

models:
  - provider: openai
    model: gpt-4-turbo-preview
    api_key: "{{ .env.OPENAI_API_KEY }}"
    params:
      temperature: 0.7

  - provider: anthropic
    model: claude-4-opus
    api_key: "{{ .env.ANTHROPIC_API_KEY }}"

  - provider: ollama
    model: llama2:13b
    api_url: "http://localhost:11434"

Reference global models in agents:

agents:
  - id: code-reviewer
    config:
      $ref: global::models.#(provider=="anthropic")
    instructions: "Review code for quality..."

Next Steps