LiteLLM

LiteLLM can run as a proxy server that exposes OpenAI-compatible endpoints for many upstream providers. The local proxy base URL is commonly http://localhost:4000/v1.

Create the Client

import { AgentBuilder } from "@anvia/core";
import { OpenAIClient } from "@anvia/openai";

const client = new OpenAIClient({
  baseUrl: process.env.LITELLM_BASE_URL ?? "http://localhost:4000/v1",
  apiKey: process.env.LITELLM_API_KEY ?? "local",
});

const model = client.completionModel("gpt-5-mini");

const agent = new AgentBuilder("support", model)
  .instructions("Answer support questions clearly.")
  .build();

const response = await agent.prompt("Hello!").send();

console.log(response.output);

The model id must match a model configured in your LiteLLM proxy. Depending on your proxy configuration, that might be a public provider model id or a local alias.

Get the Model List

When the LiteLLM proxy exposes /models, listModels() reads the configured model list through the same base URL.

const models = await client.listModels();

console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy })));

Notes

• Keep LiteLLM model aliases in your proxy configuration and pass those aliases to completionModel(...).
• Advanced features such as tools, structured output, images, and reasoning metadata depend on the upstream provider and proxy configuration.
• If your proxy requires a master key or virtual key, pass it as apiKey.

For current LiteLLM details, see the LiteLLM documentation.