Anvia Docs

Ollama provides partial OpenAI API compatibility for local models. The local OpenAI-compatible base URL is usually http://localhost:11434/v1.

Create the Client

import { AgentBuilder } from "@anvia/core";
import { OpenAIClient } from "@anvia/openai";

const client = new OpenAIClient({
  baseUrl: "http://localhost:11434/v1",
  apiKey: "ollama",
});

const model = client.completionModel("llama3.1");

const agent = new AgentBuilder("local-support", model)
  .instructions("Answer support questions clearly.")
  .build();

const response = await agent.prompt("Hello!").send();

console.log(response.output);

Ollama does not require an API key for local use, but the OpenAI SDK expects one. Use a placeholder value.

Get the Model List

Use the OpenAI-compatible models endpoint through listModels():

const models = await client.listModels();

console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy })));

Notes

Ollama compatibility covers common OpenAI-style chat flows, but not every OpenAI parameter or advanced capability.
Local model availability depends on which models are pulled into your Ollama runtime.

For current Ollama API details, see Ollama OpenAI compatibility.