Anvia Docs

Use multiple model providers without scattering provider decisions through product code.

Use multiple providers when different workflows need different model capabilities, cost profiles, latency, deployment regions, or fallback behavior.

Keep provider selection in one small module. Routes, tools, and agent runners should ask for a named model role instead of constructing provider clients directly.

Model Roles

Start by naming what each model is used for:

Role	Typical use
`primary`	normal agent turns, tool calls, and product workflows
`fast`	lightweight classification, routing, summarization, or titles
`reasoning`	high-stakes planning, analysis, or multi-step decisions
`multimodal`	image or document input
`fallback`	backup when the primary provider is unavailable

Do not encode provider names across the app. Encode product intent, then map that intent to providers centrally.

Centralize Providers

Create provider clients once and export stable model roles.

import { AnthropicClient } from "@anvia/anthropic";
import { GeminiClient } from "@anvia/gemini";
import { OpenAIClient } from "@anvia/openai";

const openai = new OpenAIClient({
  apiKey: process.env.OPENAI_API_KEY,
});

const anthropic = new AnthropicClient({
  apiKey: process.env.ANTHROPIC_API_KEY,
});

const gemini = new GeminiClient({
  apiKey: process.env.GEMINI_API_KEY,
});

export const models = {
  primary: openai.completionModel("gpt-5.5"),
  fast: gemini.completionModel("gemini-3.1-flash-lite-preview"),
  reasoning: anthropic.completionModel("claude-opus-4-6"),
};

Keep API keys, base URLs, gateway configuration, and model ids in this layer. Application workflows should import models.primary, not new OpenAIClient(...).

Select by Workflow

Let the harness choose a model role based on the workflow, tenant, feature flag, or request shape.

import { AgentBuilder } from "@anvia/core";
import { models } from "./models";

type WorkflowKind = "support" | "research" | "intake";

function modelForWorkflow(kind: WorkflowKind) {
  switch (kind) {
    case "intake":
      return models.fast;
    case "research":
      return models.reasoning;
    case "support":
      return models.primary;
  }
}

export function createWorkflowAgent(kind: WorkflowKind) {
  return new AgentBuilder(kind, modelForWorkflow(kind))
    .instructions("Follow the workflow instructions and use tools when needed.")
    .defaultMaxTurns(kind === "research" ? 6 : 3)
    .build();
}

The prompt, tools, storage, and trace metadata can stay the same while the model role changes.

Capability Boundaries

Not every provider supports every request shape. Keep capability-sensitive behavior near model selection.

Capability	Practice
tools	use models that support tool calling for agent workflows
image input	route multimodal requests to a model that accepts images
document input	route file-based workflows to a provider that supports the attachment shape
structured output	test schema-heavy prompts against each configured provider
streaming	verify UI event handling for every provider used in streaming routes

Avoid silent downgrades. If a workflow requires images, tools, or structured output, reject or reroute unsupported provider choices before calling the model.

Fallbacks

Fallbacks are product policy, not provider magic. Decide when a retry is safe.

import { AgentBuilder, Message } from "@anvia/core";
import { models } from "./models";

export async function runWithFallback(prompt: string) {
  const messages = [Message.user(prompt)];

  try {
    return await new AgentBuilder("support", models.primary)
      .instructions("Answer clearly and cite tool results when available.")
      .defaultMaxTurns(3)
      .build()
      .prompt(messages)
      .send();
  } catch (error) {
    return new AgentBuilder("support-fallback", models.reasoning)
      .instructions("Answer clearly. Mention when live provider fallback was used.")
      .defaultMaxTurns(3)
      .build()
      .prompt(messages)
      .send();
  }
}

Use fallbacks mostly for read-only or idempotent workflows. For side-effect-heavy agents, retry only before tools have executed or behind application idempotency records.

Observability

Trace the selected provider role, not just the agent name.

const response = await agent
  .prompt(input.message)
  .withTrace({
    name: "support-chat",
    metadata: {
      modelRole: "primary",
      workflow: "support",
      tenantId: input.tenantId,
    },
  })
  .send();

Use traces and evals to compare providers on real prompts before moving traffic. Provider swaps should be observable, reversible, and covered by workflow-level regression tests.

Checklist

Keep provider clients in one module.
Export model roles such as primary, fast, and reasoning.
Select models at the harness boundary, not inside tools.
Test capability-sensitive workflows against every provider role.
Record model role and workflow in trace metadata.
Use fallbacks only where retry semantics are safe.