# Agent Structure (/docs/best-practices/common-patterns/agent-structure) Build agents from explicit TypeScript values. A built agent is an immutable runtime configuration: id, model, instructions, static context, tools, hooks, observers, memory, and defaults. It should be safe to import from routes, jobs, tests, and Studio when its tools do not capture the current request. When tools or context need the current user, tenant, request, transaction, or permission state, create a scoped agent in a factory or runner. Do not hide request state in module-level variables. ## Stable Agent Module [#stable-agent-module] Use a shared built agent when every registered tool is context-free, read-only, or already enforces its own context through injected services. ```ts // src/ai/support-agent.ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { supportTools } from "./support-tools"; const client = new OpenAIClient({ apiKey: process.env.OPENAI_API_KEY }); const model = client.completionModel("gpt-5.5"); export const supportAgent = new AgentBuilder("support", model) .name("Support Agent") .description("Answers support questions and uses support tools.") .instructions( "Answer support questions clearly. Ask for missing details before guessing.", ) .tools(supportTools) .defaultMaxTurns(3) .build(); ``` The stable agent id is part of the runtime contract. It appears in traces, Studio, sessions, nested-agent tools, and workflows that reference the agent. ## Request Boundary [#request-boundary] Keep user input, auth state, tenant data, message history, trace metadata, route-specific limits, and application error mapping at the call site or runner. ```ts import { Message } from "@anvia/core"; import { supportAgent } from "./support-agent"; export async function runSupportTurn(input: SupportTurnInput) { const history = await input.conversations.loadMessages(input.conversationId); const response = await supportAgent .prompt([...history, Message.user(input.message)]) .withTrace({ name: "support-chat", userId: input.user.id, metadata: { tenantId: input.user.tenantId, conversationId: input.conversationId, }, }) .maxTurns(2) .send(); await input.conversations.append(input.conversationId, response.messages); return response.output; } ``` This keeps the agent reusable. A background job can call the same agent with a different trace name, a test can replace history with fixtures, and Studio can run the built agent without product-route state. ## Scoped Agent Factory [#scoped-agent-factory] Use a factory when tools or context need current user data, tenant data, feature flags, service handles, or a transaction. ```ts import type { CompletionModel } from "@anvia/core"; import { AgentBuilder } from "@anvia/core"; import { createSupportTools } from "./support-tools"; type SupportAgentScope = { userId: string; tenantId: string; plan: "free" | "pro" | "enterprise"; services: { orders: OrdersService; tickets: TicketsService; }; }; export function createSupportAgent(model: CompletionModel, scope: SupportAgentScope) { return new AgentBuilder("support", model) .instructions("Answer support questions clearly. Use tools for account data.") .tools( createSupportTools({ userId: scope.userId, tenantId: scope.tenantId, orders: scope.services.orders, tickets: scope.services.tickets, }), ) .context(`Current customer plan: ${scope.plan}`, "customer-plan") .defaultMaxTurns(3) .build(); } ``` The factory should receive explicit dependencies. Avoid reading the current user, request, tenant, or transaction from global state inside the agent module. ## Shared Configuration Helper [#shared-configuration-helper] If stable and scoped agents share most behavior, extract a small builder helper. ```ts import type { AgentBuilder, CompletionModel } from "@anvia/core"; function configureSupportBehavior(builder: AgentBuilder) { return builder .name("Support Agent") .description("Answers support questions and performs safe support actions.") .instructions("Answer clearly. Ask before guessing. Use tools for account data.") .defaultMaxTurns(3); } export function createScopedSupportAgent(model: CompletionModel, scope: SupportAgentScope) { return configureSupportBehavior(new AgentBuilder("support", model)) .tools(createSupportTools(scope)) .context(`Current customer plan: ${scope.plan}`, "customer-plan") .build(); } ``` Prefer factories and helpers over mutating a built agent. Build once for stable configuration, or build per request when the tool set captures request-local state. ## Decision Table [#decision-table] | Put it here | When | | -------------------- | --------------------------------------------------------------------------------------------- | | `AgentBuilder` | stable identity, instructions, static context, context-free tools, default limits, observers | | scoped agent factory | request-scoped tools, tenant-aware context, feature-flagged tools, transaction-bound services | | runner | validation, auth, history, trace metadata, persistence, error mapping | | prompt request | current input, one-off max turns, tool concurrency, request trace metadata | | tool factory | user id, tenant id, service handles, permission scope | | route or job | transport parsing, response shape, job retry policy, caller-specific timeouts | # Context and Memory (/docs/best-practices/common-patterns/context-and-memory) Context works best when each layer has a clear job. Keep durable behavior in instructions, small stable facts in static context, request facts in the scoped agent or prompt text, prompt-dependent knowledge in retrieval, and conversation continuity in history or sessions. Do not treat context as permission. Filter records by tenant, user, and access level before they can be included in a prompt or retrieval index. ## Choose the Right Layer [#choose-the-right-layer] | Layer | Use it for | Owner | | --------------- | -------------------------------------------------------------- | ---------------------------------- | | instructions | behavior rules that apply to every run | stable agent | | static context | short stable facts always relevant to the agent | stable agent | | request facts | current user, tenant, plan, locale, feature flags, route state | runner or scoped factory | | dynamic context | retrieved documents or records selected for the prompt | index plus runner filtering | | message history | previous turns needed to continue a conversation | application storage | | sessions | durable memory managed through an agent memory store | Anvia memory plus app-chosen store | | tools | data that should be fetched or changed only when needed | model decision, app execution | ## Instructions vs Facts [#instructions-vs-facts] Put behavior in instructions. Put facts in context or loaded prompt data. ```ts const agent = new AgentBuilder("billing", model) .instructions(` Answer billing questions clearly. Use billing tools for account-specific data. Do not guess invoice status. `) .context("Invoices are generated on the first day of each month.", "invoice-cycle") .build(); ``` Avoid hiding changing account facts in instructions. If a value depends on the current request, load it in the runner and attach it to the scoped agent or prompt. ```ts const agent = new AgentBuilder("billing", model) .instructions("Answer billing questions clearly.") .context(`Current account plan: ${account.plan}`, "account-plan") .context(`Billing country: ${account.country}`, "billing-country") .build(); ``` ## Request Context [#request-context] Use request context for small facts the application already knows are relevant. ```ts const prompt = [ ...history, Message.user(` Current route: support chat Current plan: ${account.plan} Open ticket count: ${openTickets.length} User message: ${message} `), ]; const response = await agent.prompt(prompt).send(); ``` Keep request context compact. If the agent only needs the current plan, do not paste the whole account record. If the model may need additional fields conditionally, expose a tool instead. ## Dynamic Context [#dynamic-context] Use dynamic context when relevant facts depend on the prompt. ```ts const agent = new AgentBuilder("support", model) .instructions("Use retrieved support context when it is relevant.") .dynamicContext(supportKnowledgeIndex, { topK: 5, threshold: 0.72, format: (result) => ({ id: result.id, text: `${result.document.title}\n${result.document.body}`, }), }) .build(); ``` Dynamic context should be tenant-safe before search. Use separate indexes, metadata filters, or a prefiltered index handle so records from another tenant cannot be retrieved. ## History and Sessions [#history-and-sessions] Use explicit `Message[]` history when your application already owns conversation storage. ```ts import { Message } from "@anvia/core"; const history = await conversations.loadMessages(conversationId); const response = await agent .prompt([...history, Message.user(message)]) .send(); await conversations.append(conversationId, response.messages); ``` Use `agent.session(...)` when the agent has a memory store configured and you want Anvia to manage durable conversation state for that session. ```ts const response = await agent .session(sessionId) .prompt("What did we decide last time?") .send(); ``` Use one approach per workflow unless there is a clear reason to combine them. Mixing explicit history and sessions casually makes prompts harder to inspect. ## Context Assembly Checklist [#context-assembly-checklist] | Question | Default | | --------------------------------------------- | ---------------------------------- | | Is it a rule for every run? | instruction | | Is it a short stable fact? | static `.context(...)` | | Is it current user or tenant data? | runner or scoped agent context | | Is it large or frequently changing knowledge? | retrieval | | Is it previous conversation state? | explicit history or session memory | | Does access need to be checked at call time? | tool or service | | Could it leak another tenant's data? | filter before context assembly | ## Practical Rule [#practical-rule] Context should explain the current run, not become a data dump. If a fact is needed every time, make it stable. If a fact changes per request, load it in the runner. If a fact is large, search for it. If a fact requires permission, put it behind a tool or prefilter it before retrieval. ## Related Patterns [#related-patterns] * Use [Support Agent](/docs/best-practices/real-cases/support-agent) for account-aware chat with history and retrieval. * Use [Research Agent](/docs/best-practices/real-cases/research-agent) for retrieval-heavy workflows with many read-only tools. * Use [Dynamic Tool Catalogs](/docs/best-practices/tool-patterns/dynamic-tool-catalogs) when searchable tools are a better fit than broad context. # Harness Blueprint (/docs/best-practices/common-patterns/harness-blueprint) An agent harness is the application-owned code around the Anvia runtime. It accepts a product request, prepares the agent run, controls access to application behavior, records what happened, and returns a product response. The harness is not a new framework layer. It is ordinary TypeScript that makes the boundary explicit: Anvia runs the prompt, tools, hooks, retrieval, memory, streaming, and observers; your application owns auth, data, permissions, transactions, storage, retries, deployment, and side effects. ## Ownership Map [#ownership-map] | Layer | Owns | | ------------------------------------------ | -------------------------------------------------------------------------------------------- | | route, server action, job, or queue worker | transport parsing, auth entrypoint, response shape, job retry policy | | harness runner | one product workflow, request validation, scoped dependencies, trace metadata, error mapping | | stable agent factory | agent id, instructions, model behavior, static defaults, reusable observers | | request-scoped tools | product service calls, permissions, tenant filtering, side-effect safety | | context assembly | static facts, request facts, retrieval results, history, session id | | storage and audit | conversation history, event stream, memory store, idempotency records, approval decisions | | Anvia runtime | model-tool loop, schemas, hooks, turn limits, tool calls, streaming events, observer events | | model provider | completion behavior and provider usage accounting | Keep the ownership split boring. If a decision affects product correctness or security, it belongs in app code or a tool. If it affects how the model is prompted, constrained, or observed, configure it on the agent or prompt request. ## Recommended Modules [#recommended-modules] ```txt src/ ai/ model.ts # provider clients and reusable models support-agent.ts # stable agent factory or shared built agent support-tools.ts # request-scoped tool factories support-runner.ts # harness runner for one product workflow services/ orders.ts # product services and permission-aware data access tickets.ts storage/ conversations.ts # history, sessions, events, and audit records routes/ support.ts # thin transport boundary ``` This layout is a convention, not a requirement. The important part is the direction of dependencies: routes call runners, runners create scoped tools and agents, tools call services, and services own product data. ## Request Lifecycle [#request-lifecycle] | Step | Harness responsibility | Anvia responsibility | | ------------------- | ------------------------------------------------------------------------- | ------------------------------------------- | | validate input | parse payload, reject missing fields, normalize product ids | none | | resolve app context | authenticate, load user, tenant, feature flags, services, transaction | none | | load state | read history, session state, idempotency record, relevant product records | read memory only if configured | | create runtime | build or select agent, create request-scoped tools, add context | hold agent configuration | | run prompt | attach trace metadata, set turn limit, call `.send()` or `.stream()` | run model-tool loop | | handle effects | tools enforce permissions, services run transactions and audit writes | call tools and return tool results to model | | persist result | append messages, event logs, summaries, metrics, user-visible output | return response messages and usage | | map response | return HTTP body, job result, UI stream, or domain object | none | ## Minimal Harness [#minimal-harness] ```ts import { AgentBuilder, Message } from "@anvia/core"; import { model } from "./model"; import { createSupportTools } from "./support-tools"; export async function runSupportHarness(input: SupportHarnessInput) { const user = await input.auth.requireUser(); const history = await input.conversations.loadMessages(input.conversationId); const agent = new AgentBuilder("support", model) .name("Support Agent") .instructions("Answer support questions clearly. Use tools for account data.") .tools( createSupportTools({ userId: user.id, tenantId: user.tenantId, orders: input.services.orders, tickets: input.services.tickets, }), ) .context(`Current customer plan: ${user.plan}`, "customer-plan") .defaultMaxTurns(3) .build(); const response = await agent .prompt([...history, Message.user(input.message)]) .withTrace({ name: "support-chat", userId: user.id, metadata: { tenantId: user.tenantId, conversationId: input.conversationId, }, }) .send(); await input.conversations.append(input.conversationId, response.messages); return { output: response.output, usage: response.usage, }; } ``` The runner can be called from a route, background job, CLI script, test, or Studio setup. Keep transport details outside it unless the workflow itself is transport-specific. ## Harness Checklist [#harness-checklist] | Concern | Production default | | ------------- | ------------------------------------------------------------ | | identity | stable agent id and trace name | | input | validate before constructing the prompt | | permissions | enforce inside tools and services, not in prompt text | | context | attach only scoped, relevant facts | | side effects | use idempotency keys or transactions in service code | | persistence | save `response.messages` and any app-owned audit records | | observability | use stable traces and safe metadata | | failures | map known errors to product responses at the runner boundary | Start with this shape before adding more agents, nested agents, or pipelines. Most harness problems are easier to fix when the single-agent boundary is explicit. # Pipeline (/docs/best-practices/common-patterns/pipeline) Start with one runner around one agent. Add orchestration only when the job has a real boundary: a deterministic step, a specialist capability, a parallel branch, a typed extraction stage, batching, or a separately testable process. Many production workflows do not need a pipeline. They need a clear harness runner. ## One Agent in a Runner [#one-agent-in-a-runner] Use one agent when one promptable runtime can own the task and a small set of tools is enough. ```ts const agent = new AgentBuilder("support", model) .instructions("Answer support questions and use tools when account data is needed.") .tools(createSupportTools(scope)) .defaultMaxTurns(3) .build(); const response = await agent.prompt(message).send(); ``` This is the simplest shape to trace, test, and run in Studio. Prefer it until the workflow has a boundary that should be named and tested separately. ## Runner Before Pipeline [#runner-before-pipeline] Use a runner when the workflow is mostly application wiring: auth, input validation, history, scoped tools, context, trace metadata, persistence, and error mapping. ```ts export async function runSupportTurn(input: SupportRunnerInput) { const user = await input.auth.requireUser(); const history = await input.conversations.loadMessages(input.conversationId); const agent = createSupportAgent(model, { user, services: input.services }); const response = await agent .prompt([...history, Message.user(input.message)]) .withTrace({ name: "support-chat", userId: user.id }) .send(); await input.conversations.append(input.conversationId, response.messages); return response.output; } ``` Do not add a pipeline just to hide application setup. If the steps are not reusable or independently testable, the runner is the right abstraction. ## Agent as a Tool [#agent-as-a-tool] Use an agent tool when a subdomain has stable instructions and should be callable by another agent. ```ts const refundsAgent = new AgentBuilder("refunds", model) .description("Answers refund-policy questions.") .instructions("Answer only refund-related questions.") .build(); const triageAgent = new AgentBuilder("triage", model) .instructions("Route refund questions to the refund specialist.") .tool( refundsAgent.asTool({ name: "ask_refunds_agent", maxTurns: 2, }), ) .defaultMaxTurns(3) .build(); ``` Keep nested agents narrow. A broad agent calling another broad agent usually makes traces harder to understand and quality harder to evaluate. ## Pipeline [#pipeline] Use a pipeline when the job has explicit typed stages that should be tested, reused, or observed independently. ```ts const pipeline = new PipelineBuilder() .step((input) => input.trim()) .prompt(triageAgent) .extract(ticketExtractor) .step((ticket) => ({ ...ticket, needsEscalation: ticket.priority === "high", })) .build(); ``` Pipelines are a good fit for preprocessing, enrichment, extraction, branching, batching, and post-processing. They are a poor fit for hiding route logic or permission policy. ## Decision Table [#decision-table] | Use | When | | ------------------- | --------------------------------------------------------------------- | | one agent | one conversational runtime can own the task | | runner | the complexity is app context, persistence, tracing, or error mapping | | agent tool | another agent owns a narrow specialist boundary | | pipeline | the job has explicit typed stages or deterministic steps | | direct service call | the step does not need a model | | extractor | model output must match a schema for downstream code | | parallel branches | independent model or service work can run at the same time | ## Practical Rule [#practical-rule] Reach for the smallest named boundary that matches the job. A runner names a product workflow. An agent names model behavior. A tool names product capability. A pipeline names typed process stages. # Production Guardrails (/docs/best-practices/common-patterns/production-guardrails) Production guardrails should live at the boundary they protect. Agents can own default runtime limits and hooks. Tools should own side-effect safety. The application should own retries, timeouts, idempotency records, persistence, audit logs, and response shape. Do not try to solve product safety with prompt text alone. Prompt instructions help the model choose behavior, but product code must enforce the boundary. ## Turn Limits [#turn-limits] Keep tool-call loops bounded. Set a conservative default on the agent and override it per request only when the workflow needs more room. ```ts const agent = new AgentBuilder("support", model) .instructions("Use tools only when account data is required.") .tools(supportTools) .defaultMaxTurns(3) .build(); const response = await agent.prompt(message).maxTurns(2).send(); ``` Low limits make failures faster and traces easier to inspect. If a workflow often needs a high turn limit, check whether it should be split into deterministic steps, a pipeline, or a narrower tool. ## Permission Checks [#permission-checks] Use tool or service code for normal permission checks. The model should never be the authority for whether a user can read data or perform a side effect. ```ts async execute({ orderId }) { await orders.requireAccess({ userId: scope.userId, tenantId: scope.tenantId, orderId, }); return orders.find(orderId); } ``` Permission failures can either return a typed state when the model can continue, or throw when the product request should fail. ## Hooks and Approvals [#hooks-and-approvals] Use hooks when the run should decide whether to run, skip, or cancel a tool call. Use application code for reviewer identity, storage, notification, timeout, and audit policy. ```ts import { createHook } from "@anvia/core"; const approvalHook = createHook({ async onToolCall({ toolName, tool }) { if (toolName !== "issue_refund") { return tool.run(); } const approved = await approvals.waitForDecision({ toolName, reason: "Refunds require reviewer approval.", }); return approved ? tool.run() : tool.cancel("Refund was not approved."); }, }); const agent = new AgentBuilder("support", model) .instructions("Use refund tools only when policy allows it.") .tools(refundTools) .hook(approvalHook) .build(); ``` Studio is useful for local approval iteration. Production approval storage, notifications, reviewer identity, and audit logs stay in your application. ## Timeouts and Cancellation [#timeouts-and-cancellation] Use hooks to cancel unsafe prompt runs. Use app-level timeout wrappers when the caller needs bounded latency. ```ts import { createHook } from "@anvia/core"; const policyHook = createHook({ async onCompletionCall({ prompt, run }) { if (containsSensitiveExportRequest(prompt)) { return run.cancel("Sensitive exports are not allowed in this workflow."); } return run.continue(); }, }); ``` ```ts const result = await withTimeout(runSupportTurn(input), 30_000); ``` Use cancellation for policy decisions and user-denied workflows. Use timeouts for caller latency. Use retries for transient infrastructure failures only when the operation is safe to retry. ## Idempotent Tools [#idempotent-tools] For side effects, pass an application-generated operation id into the service layer and let the service deduplicate. ```ts async execute({ orderId, amount }) { return billing.issueRefund({ userId: scope.userId, tenantId: scope.tenantId, orderId, amount, operationId: `refund:${scope.tenantId}:${orderId}:${amount}`, }); } ``` The model should not invent the idempotency boundary. Your application should. ## Retries [#retries] Retry at the runner or job boundary, not inside individual tools by default. ```ts export async function runSupportJob(job: SupportJob) { return retryTransient( () => runSupportTurn(job.input), { attempts: 2, retryIf: (error) => isProviderTimeout(error) || isTemporaryStorageError(error), }, ); } ``` Do not retry non-idempotent write tools unless the service method has an idempotency key, transaction boundary, or durable operation record. ## Audit Records [#audit-records] Audit product decisions in application storage. Traces help debug agent behavior, but they should not be the only record of a sensitive product operation. ```ts await audit.write({ actorId: scope.userId, tenantId: scope.tenantId, action: "refund.issued", targetId: orderId, operationId, }); ``` Use traces for runtime inspection. Use audit records for product accountability. ## Decision Table [#decision-table] | Risk | Guardrail | | ----------------------------- | ------------------------------------------------------ | | endless tool loop | `.defaultMaxTurns(...)` and request `.maxTurns(...)` | | restricted read | permission check in tool or service code | | restricted write | permission check, hook, approval runtime, audit record | | human approval required | application-owned approval runtime called from a hook | | caller needs bounded latency | app-level timeout around the runner | | duplicate side effect | idempotency key or transaction in the service layer | | provider or transport failure | retry at the application boundary when safe | | sensitive operation | product audit record plus trace metadata | ## Related Patterns [#related-patterns] * Use [Side Effect Tools](/docs/best-practices/tool-patterns/side-effect-tools) for writes, approvals, idempotency, and audit records. * Use [Backoffice Agent](/docs/best-practices/real-cases/backoffice-agent) for admin workflows with approval-heavy operations. * Use [Production Readiness Checklist](/docs/best-practices/operations/production-readiness-checklist) before deploying a harness. # Request Runners (/docs/best-practices/common-patterns/request-runners) A request runner is the function your route, job, queue worker, or test calls. It owns the product workflow around one agent run. Keep it small enough to test directly and explicit enough that auth, context, tools, persistence, and error mapping are visible. ## Runner Responsibilities [#runner-responsibilities] | Responsibility | Why it belongs in the runner | | --------------------- | -------------------------------------------------------------------------------------------- | | validate input | reject bad product requests before model calls | | resolve app context | auth, tenant, feature flags, service handles, and transactions are app-owned | | load state | conversation history, memory ids, idempotency records, and product records are storage-owned | | create scoped runtime | request-scoped tools and context should not leak into globals | | call the agent | the runner controls trace metadata, limits, and streaming vs final response | | persist output | `response.messages`, events, audit records, and summaries are product state | | map failures | routes and jobs need predictable product errors | ## Standard Runner Shape [#standard-runner-shape] ```ts import { Message, PromptCancelledError } from "@anvia/core"; import { model } from "./model"; import { createSupportAgent } from "./support-agent"; type SupportRunnerInput = { conversationId: string; message: string; auth: AuthService; conversations: ConversationStore; services: { orders: OrdersService; tickets: TicketsService; }; }; export async function runSupportTurn(input: SupportRunnerInput) { const message = input.message.trim(); if (message.length === 0) { return { ok: false as const, error: "message_required" }; } const user = await input.auth.requireUser(); const history = await input.conversations.loadMessages(input.conversationId); const agent = createSupportAgent(model, { userId: user.id, tenantId: user.tenantId, plan: user.plan, services: input.services, }); try { const response = await agent .prompt([...history, Message.user(message)]) .withTrace({ name: "support-chat", userId: user.id, metadata: { tenantId: user.tenantId, conversationId: input.conversationId, }, }) .send(); await input.conversations.append(input.conversationId, response.messages); return { ok: true as const, output: response.output, usage: response.usage, }; } catch (error) { if (error instanceof PromptCancelledError) { return { ok: false as const, error: "cancelled" }; } throw error; } } ``` The route can stay thin: ```ts export async function POST(request: Request) { const body = await request.json(); const result = await runSupportTurn({ conversationId: body.conversationId, message: body.message, auth, conversations, services: { orders, tickets }, }); if (!result.ok) { return Response.json({ error: result.error }, { status: 400 }); } return Response.json({ output: result.output }); } ``` ## Load Context Before the Prompt [#load-context-before-the-prompt] Do not ask the model to discover product context that the application can resolve deterministically. Load the known values first and attach only the facts the agent needs. ```ts const account = await accounts.findForUser(user.id); const openTickets = await tickets.listOpen({ userId: user.id, limit: 5 }); const agent = createSupportAgent(model, { userId: user.id, tenantId: user.tenantId, plan: account.plan, services: input.services, }); const prompt = [ ...history, Message.user(` Current account plan: ${account.plan} Open ticket count: ${openTickets.length} User message: ${message} `), ]; const response = await agent.prompt(prompt).send(); ``` Use tools when the model needs to decide whether to fetch or change state. Use preloaded context when the application already knows the fact is required. ## Timeouts and Retries [#timeouts-and-retries] Anvia prompt requests do not replace product-level timeout or retry policy. Put bounded-latency behavior around the runner. ```ts export async function withTimeout(promise: Promise, timeoutMs: number): Promise { let timeout: ReturnType | undefined; const timeoutPromise = new Promise((_, reject) => { timeout = setTimeout(() => { reject(new Error("support_agent_timeout")); }, timeoutMs); }); try { return await Promise.race([promise, timeoutPromise]); } finally { if (timeout !== undefined) { clearTimeout(timeout); } } } const result = await withTimeout(runSupportTurn(input), 30_000); ``` Retry only when the whole operation is safe to retry. If tools can create side effects, the service layer should use idempotency keys or transactions before the runner retries the prompt. ## Testing Boundary [#testing-boundary] A runner is easy to test because every product dependency is passed in. ```ts it("rejects empty messages before calling the agent", async () => { const result = await runSupportTurn({ conversationId: "conv_123", message: " ", auth: fakeAuth(), conversations: fakeConversations(), services: fakeServices(), }); expect(result).toEqual({ ok: false, error: "message_required" }); }); ``` Test validation, auth failures, storage calls, known cancellations, and persistence without a provider call. Use provider-backed tests only for the model behavior the workflow depends on. # Testing and Observability (/docs/best-practices/common-patterns/testing-and-observability) Most correctness should be tested before a provider call is made. Tools, service wrappers, retrieval filters, pipeline steps, prompt wrappers, runners, and error mapping can all be checked with fakes. Use provider-backed tests, Studio, traces, and evals after the application boundaries are covered. ## Test Tools Directly [#test-tools-directly] Tools have schemas and executable handlers. Call them directly or through a `ToolSet` to verify validation, permissions, expected states, and output shape. ```ts const tools = createSupportToolSet({ userId: "user_123", tenantId: "tenant_123", orders: fakeOrders, tickets: fakeTickets, }); const result = await tools.call( "lookup_order", JSON.stringify({ orderId: "A-100" }), ); expect(JSON.parse(result)).toEqual({ status: "found", orderId: "A-100", fulfillmentStatus: "shipped", }); ``` This keeps product policy testable without asking a model to choose the right branch. ## Test Runners With Fakes [#test-runners-with-fakes] Runner tests should verify application behavior: validation, auth, history loading, scoped tool creation, trace metadata, persistence, and known errors. ```ts it("persists new messages after a successful support turn", async () => { const conversations = fakeConversations({ history: [Message.user("Earlier question")], }); const result = await runSupportTurn({ conversationId: "conv_123", message: "Where is order A-100?", auth: fakeAuth({ userId: "user_123", tenantId: "tenant_123" }), conversations, services: fakeServices(), }); expect(result.ok).toBe(true); expect(conversations.append).toHaveBeenCalledWith( "conv_123", expect.any(Array), ); }); ``` If the runner currently creates the real agent internally, extract a factory dependency for tests or keep provider-backed tests narrow. Do not make every route test depend on a live model. ## Test Retrieval Filters [#test-retrieval-filters] Retrieval bugs are often permission bugs. Test the filter or index handle before testing answer quality. ```ts const results = await tenantKnowledge.search("refund policy", { tenantId: "tenant_123", topK: 5, }); expect(results.every((result) => result.metadata.tenantId === "tenant_123")).toBe(true); ``` Then use Studio, traces, or evals to inspect whether the retrieved context produces the answer you expect. ## Observe Runs [#observe-runs] Attach observers when you need traces, usage records, external reporting, or debugging evidence. ```ts const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .observe(observer) .build(); ``` Use stable trace names such as `support-chat`, `ticket-summary`, or `retrieval-answer`. Include application identifiers in trace metadata when they are safe to store. ```ts const response = await agent .prompt(message) .withTrace({ name: "support-chat", userId, metadata: { tenantId, conversationId, channel: "web", }, }) .send(); ``` Trace metadata should help connect an agent run to application state without leaking secrets or large records. ## Use Studio During Iteration [#use-studio-during-iteration] Use Studio to inspect messages, tool calls, approvals, sessions, traces, and quick prompts while the workflow is still changing. ```ts import { Studio } from "@anvia/studio"; import { supportAgent } from "./ai/support-agent"; new Studio([supportAgent]).start({ port: 3000 }); ``` For request-scoped agents, create a safe development scope with fake or sandbox services and register that built agent in Studio. Do not wire Studio directly to production credentials just to test prompt behavior. ## Use Evals for Repeatable Behavior [#use-evals-for-repeatable-behavior] Use evals when the workflow has known prompts, regression cases, or output expectations that should be checked repeatedly. Keep eval targets narrow: | Eval target | Good use | | ---------------- | ---------------------------------------------- | | tool output | permission and state contracts | | runner output | product response shape and known error mapping | | agent output | answer quality for stable prompts | | extractor output | schema adherence and downstream contract | | pipeline output | stage composition and typed workflow behavior | Evals should complement unit tests, not replace them. Unit tests prove app-owned boundaries. Evals watch model-dependent behavior. ## Harness Test Matrix [#harness-test-matrix] | Area | Test without provider | Inspect with provider | | ----------------------- | -------------------------------------------------- | ---------------------------------------- | | input validation | empty, malformed, unsupported payloads | not needed | | auth and permissions | fake users, tenants, denied reads and writes | trace denied tool calls | | tools | direct `ToolSet.call(...)` with fake services | observe model tool choice | | context assembly | snapshot prompt messages or scoped facts | inspect retrieved documents in traces | | history and persistence | fake conversation store calls | verify multi-turn behavior | | guardrails | hook cancellation, approval decisions, idempotency | inspect approval and cancellation traces | | final output | runner response shape | eval answer quality | ## Practical Rule [#practical-rule] Test product policy with fakes and direct calls. Use provider tests, Studio, traces, and evals to inspect model behavior after the application-owned boundaries are already covered. ## Related Patterns [#related-patterns] * Use [Tool Validation and Contracts](/docs/best-practices/tool-patterns/tool-validation-and-contracts) for schema and direct-call tests. * Use [MCP Tool Inspection](/docs/best-practices/mcp-patterns/mcp-tool-inspection) for server capability checks. * Use [Eval Strategy](/docs/best-practices/quality-observability/eval-strategy) for repeatable model-dependent checks. * Use [Tracing and Debugging](/docs/best-practices/quality-observability/tracing-and-debugging) to connect traces, logs, and eval outcomes. * Use [Production Readiness Checklist](/docs/best-practices/operations/production-readiness-checklist) for deployment validation. # Tools and Services (/docs/best-practices/common-patterns/tools-and-services) Tools are the boundary between model decisions and product behavior. Keep tool names, descriptions, schemas, and result shapes model-friendly. Keep permissions, database access, side effects, transactions, and audit policy in application code. A good tool is a narrow adapter over a product capability. It should not be a second service layer, and it should not trust the model to enforce product policy. ## Tool Factory Pattern [#tool-factory-pattern] Use factories when a tool needs the current user, tenant, database transaction, feature flags, or service client. ```ts import { createTool } from "@anvia/core"; import { z } from "zod"; type OrderToolScope = { userId: string; tenantId: string; orders: OrdersService; }; export function createOrderTools(scope: OrderToolScope) { const lookupOrder = createTool({ name: "lookup_order", description: "Look up one order for the current customer.", input: z.object({ orderId: z.string().min(1), }), output: z.object({ status: z.enum(["found", "not_found"]), orderId: z.string(), fulfillmentStatus: z.string().optional(), }), async execute({ orderId }) { await scope.orders.requireAccess({ userId: scope.userId, tenantId: scope.tenantId, orderId, }); const order = await scope.orders.find(orderId); if (!order) { return { status: "not_found" as const, orderId }; } return { status: "found" as const, orderId, fulfillmentStatus: order.fulfillmentStatus, }; }, }); return [lookupOrder]; } ``` The model sees a small action. Your application still owns access checks, tenant scoping, service calls, and product states. ## Compose Tool Groups [#compose-tool-groups] Create groups by domain, then combine them in the scoped agent factory or runner. ```ts import { ToolSet } from "@anvia/core"; export function createSupportToolSet(scope: SupportToolScope) { return ToolSet.fromTools([ ...createOrderTools(scope), ...createTicketTools(scope), ...createPolicyTools(scope), ]); } ``` Use `.tools([...])` for a request-scoped list. Use `.useToolSet(...)` when you need a shared mutable catalog that can be inspected or updated at runtime. ```ts const agent = new AgentBuilder("support", model) .instructions("Use tools when account-specific data is required.") .useToolSet(createSupportToolSet(scope)) .defaultMaxTurns(3) .build(); ``` ## Return Expected States [#return-expected-states] Return structured expected states when the model can continue productively. Throw only when the workflow should fail, be retried, or be handled by the application error boundary. ```ts async execute({ ticketId }) { const ticket = await tickets.findForUser(scope.userId, ticketId); if (!ticket) { return { status: "not_found" as const, ticketId }; } if (ticket.locked) { return { status: "blocked" as const, reason: "ticket_locked", ticketId, }; } return { status: "ready" as const, ticketId, subject: ticket.subject, priority: ticket.priority, }; } ``` Expected states make the model's next step easier to inspect. Unexpected errors should still throw so the runner can log, retry, or return a stable product error. ## Keep Side Effects Behind Services [#keep-side-effects-behind-services] For write tools, the tool validates model input and calls a product service. The service owns permissions, transactions, idempotency, and audit records. ```ts export function createRefundTools(scope: RefundToolScope) { return [ createTool({ name: "issue_refund", description: "Issue an approved refund for a paid order.", input: z.object({ orderId: z.string(), amount: z.number().positive(), reason: z.string().min(1), }), async execute({ orderId, amount, reason }) { return scope.billing.issueRefund({ userId: scope.userId, tenantId: scope.tenantId, orderId, amount, reason, operationId: `refund:${scope.tenantId}:${orderId}:${amount}`, }); }, }), ]; } ``` The model can request the operation. It should not invent the permission check, transaction boundary, or idempotency key policy. ## Tool Contract Checklist [#tool-contract-checklist] | Concern | Good default | | ------------- | --------------------------------------------------------------------- | | name | verb-noun, stable, specific, such as `lookup_order` | | description | tell the model when to use it, not how the service works internally | | input schema | require product ids, enums, and bounded strings where possible | | output schema | return compact states the model can reason about | | permissions | enforce in service or tool code before reading or writing data | | side effects | route through service methods with idempotency and audit behavior | | errors | return expected states, throw unexpected failures | | tests | call tools directly or through `ToolSet.call(...)` with fake services | ## Decision Table [#decision-table] | Situation | Pattern | | ------------------------------------- | -------------------------------------------------------------- | | tool needs current user or tenant | create tools inside a scoped factory | | tool reads product data | filter and authorize in the service call | | product state is recoverable | return `not_found`, `blocked`, `ready`, or another typed state | | product operation failed unexpectedly | throw and let the runner boundary handle it | | write operation can be retried | require an idempotency key or transaction in the service layer | | many tools need reuse or direct tests | group them with `ToolSet` | ## Related Patterns [#related-patterns] * Use [Dynamic Tool Catalogs](/docs/best-practices/tool-patterns/dynamic-tool-catalogs) when the catalog is too large to send every turn. * Use [Tool Validation and Contracts](/docs/best-practices/tool-patterns/tool-validation-and-contracts) when tool output drives downstream code. * Use [Side Effect Tools](/docs/best-practices/tool-patterns/side-effect-tools) when a tool writes data or calls an external write API. # Best Practices (/docs/best-practices) Use these patterns when an agent is moving from a demo into product code. The goal is to keep model behavior explicit while your application keeps ownership of users, permissions, data, side effects, storage, deployment, and audit trails. This section is organized as a pattern library. Start with the common harness shape, then jump to the real case that matches the system you are building. ## Pattern Map [#pattern-map] | Need | Pattern | | -------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | | Understand the standard agent harness boundary | [Harness Blueprint](/docs/best-practices/common-patterns/harness-blueprint) | | Keep stable agent config separate from request state | [Agent Structure](/docs/best-practices/common-patterns/agent-structure) | | Wrap one route, job, or queue request | [Request Runners](/docs/best-practices/common-patterns/request-runners) | | Wrap product services as safe model-callable actions | [Tools and Services](/docs/best-practices/common-patterns/tools-and-services) | | Assemble instructions, facts, retrieval, history, and sessions | [Context and Memory](/docs/best-practices/common-patterns/context-and-memory) | | Choose one agent, nested agents, or a typed pipeline | [Pipeline](/docs/best-practices/common-patterns/pipeline) | | Add limits, approvals, idempotency, retries, and audit records | [Production Guardrails](/docs/best-practices/common-patterns/production-guardrails) | | Test deterministic boundaries and inspect model behavior | [Testing and Observability](/docs/best-practices/common-patterns/testing-and-observability) | ## Tool Patterns [#tool-patterns] | Real problem | Pattern | | ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | | The agent has dozens or hundreds of tools | [Dynamic Tool Catalogs](/docs/best-practices/tool-patterns/dynamic-tool-catalogs) | | Tool arguments, outputs, and product states need contracts | [Tool Validation and Contracts](/docs/best-practices/tool-patterns/tool-validation-and-contracts) | | A tool writes data, sends messages, or changes external state | [Side Effect Tools](/docs/best-practices/tool-patterns/side-effect-tools) | ## Long Process Pipelines [#long-process-pipelines] | Need | Pattern | | ------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------- | | Run Anvia pipelines outside the request path with BullMQ and Redis | [Overview](/docs/best-practices/long-process-pipelines) | | Enqueue validated pipeline work from an API boundary | [Enqueue Pipeline Jobs](/docs/best-practices/long-process-pipelines/enqueue-pipeline-jobs) | | Run pipeline jobs from a separate BullMQ worker | [Run Pipeline Workers](/docs/best-practices/long-process-pipelines/run-pipeline-workers) | | Persist status, handle retries, and test long-running jobs | [Status, Retries, and Testing](/docs/best-practices/long-process-pipelines/status-retries-testing) | ## Multi-agent Patterns [#multi-agent-patterns] | Need | Pattern | | ---------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | | Stream coordinator output and nested specialist progress | [Overview](/docs/best-practices/multi-agent-patterns) | | Build specialist agents and expose them as streaming tools | [Build Streaming Specialists](/docs/best-practices/multi-agent-patterns/build-streaming-specialists) | | Group parent and child stream events in a UI | [Consume Nested Events](/docs/best-practices/multi-agent-patterns/consume-nested-events) | | Persist, replay, and test multi-agent streams | [Persistence and Testing](/docs/best-practices/multi-agent-patterns/persistence-and-testing) | ## MCP Patterns [#mcp-patterns] | Real problem | Pattern | | --------------------------------------------------------------- | ------------------------------------------------------------------------------ | | You need to connect and reconnect external MCP servers | [MCP Server Lifecycle](/docs/best-practices/mcp-patterns/mcp-server-lifecycle) | | You need to inspect, validate, filter, or wrap MCP tools | [MCP Tool Inspection](/docs/best-practices/mcp-patterns/mcp-tool-inspection) | | You need an agent harness that combines app tools and MCP tools | [MCP Agent Harness](/docs/best-practices/mcp-patterns/mcp-agent-harness) | ## Knowledge Patterns [#knowledge-patterns] | Real problem | Pattern | | ----------------------------------------------------- | ------------------------------------------------------------------------------ | | You need to build or refresh a retrieval index safely | [RAG Ingestion](/docs/best-practices/knowledge-patterns/rag-ingestion) | | You need to decide how retrieval enters an agent run | [RAG Agent Context](/docs/best-practices/knowledge-patterns/rag-agent-context) | ## Real Cases [#real-cases] | System | Pattern | | ------------------------------------------------------------------------------ | -------------------------------------------------------------------- | | Customer support chat with account tools, retrieval, and history | [Support Agent](/docs/best-practices/real-cases/support-agent) | | Backoffice workflow with writes, approvals, audit, and idempotency | [Backoffice Agent](/docs/best-practices/real-cases/backoffice-agent) | | Research workflow with many read-only tools and extraction | [Research Agent](/docs/best-practices/real-cases/research-agent) | | Coding assistant over a codebase with file, command, patch, and git boundaries | [Coding Agent](/docs/best-practices/real-cases/coding-agent) | ## Quality and Observability [#quality-and-observability] | Need | Pattern | | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | | Build regression checks for prompts, tools, and workflows | [Eval Strategy](/docs/best-practices/quality-observability/eval-strategy) | | Connect traces, tool calls, retrieval evidence, logs, and eval scores | [Tracing and Debugging](/docs/best-practices/quality-observability/tracing-and-debugging) | ## Operations [#operations] | Need | Pattern | | ----------------------------------------------- | ------------------------------------------------------------------------------------------------ | | Check whether a harness is ready for production | [Production Readiness Checklist](/docs/best-practices/operations/production-readiness-checklist) | ## Baseline Shape [#baseline-shape] Most production agents follow the same boundaries: * stable provider clients, models, reusable agents, static tool catalogs, observers, and default limits are created at startup * request-local user, tenant, conversation, permission, feature flag, MCP availability, and caller timeout policy are resolved at the application boundary * tools wrap product services and enforce permissions before reading or changing data * context is assembled from instructions, static facts, request facts, retrieval, history, and memory * persistence, retries, idempotency, audit logs, and product response shapes stay in application code ```ts import { AgentBuilder, Message } from "@anvia/core"; import { model } from "./ai/model"; import { createSupportTools } from "./ai/support-tools"; export async function runSupportTurn(input: SupportTurnInput) { const user = await input.auth.requireUser(); const conversation = await input.conversations.load(input.conversationId); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly. Use tools for account data.") .tools( createSupportTools({ userId: user.id, tenantId: user.tenantId, orders: input.services.orders, tickets: input.services.tickets, }), ) .context(`Current plan: ${user.plan}`, "current-plan") .defaultMaxTurns(3) .build(); const response = await agent .prompt([...conversation.history, Message.user(input.message)]) .withTrace({ name: "support-chat", userId: user.id, metadata: { tenantId: user.tenantId, conversationId: input.conversationId, }, }) .send(); await input.conversations.append(input.conversationId, response.messages); return { output: response.output, messages: response.messages, }; } ``` The exact route, worker, queue, or UI framework can vary. The ownership boundary should not: Anvia owns the model-tool loop, and your application owns product state and side effects. # RAG Agent Context (/docs/best-practices/knowledge-patterns/rag-agent-context) RAG context is the runtime side of retrieval. The harness decides whether retrieval should happen automatically with `.dynamicContext(...)`, through a search tool the model can call, or through preloaded context from application code. Retrieval is input, not permission. Filter documents before they can enter the model request. ## Scenario [#scenario] A support agent needs policy docs on every answer, while a research agent should decide when to search. Both use the same indexed knowledge, but retrieval enters the run differently. ## When to Use It [#when-to-use-it] Use this pattern when: * the agent needs facts that are too large or change too often for static context * answers should include source-backed evidence * retrieved documents need formatting, thresholds, or metadata filters * you need evals for retrieval quality and answer quality separately ## Architecture Shape [#architecture-shape] | Pattern | Use when | Tradeoff | | ------------------------ | ---------------------------------------------- | ------------------------------------------ | | `.dynamicContext(...)` | every prompt should receive relevant knowledge | automatic, but search happens every run | | search tool | model should decide whether and when to search | more flexible, but uses a tool turn | | runner-preloaded context | app already knows required facts | deterministic, but less adaptive | | `.dynamicTools(...)` | the retrieved object is a tool definition | solves capability selection, not knowledge | ## Automatic Dynamic Context [#automatic-dynamic-context] ```ts const agent = new AgentBuilder("support", model) .instructions("Use retrieved support docs when answering policy questions.") .dynamicContext(supportDocsIndex, { topK: 4, threshold: 0.72, format: (result) => ({ id: result.id, text: [ `Source: ${result.document.title}`, `Product: ${result.metadata?.product ?? "unknown"}`, result.document.body, ].join("\n"), }), }) .defaultMaxTurns(3) .build(); ``` Use `format(...)` to control exactly what the model sees. Include source ids or titles when answers should be traceable. ## Search Tool [#search-tool] Use a tool when retrieval should be optional or iterative. ```ts const searchSupportDocs = supportDocsIndex.asTool({ name: "search_support_docs", description: "Search support documentation by query.", topK: 5, threshold: 0.7, }); const agent = new AgentBuilder("support-research", model) .instructions("Search docs when the answer depends on current support policy.") .tool(searchSupportDocs) .defaultMaxTurns(4) .build(); ``` ## Tenant and Access Filtering [#tenant-and-access-filtering] Filter before retrieval results become model input. Use separate indexes when strict isolation is simpler than filtering. ```ts const tenantDocsIndex = docsStore.indexForTenant(user.tenantId); const agent = new AgentBuilder("tenant-support", model) .dynamicContext(tenantDocsIndex, { topK: 4, threshold: 0.75, }) .build(); ``` If a shared index is used, make the index handle enforce metadata filters so another tenant's document cannot be returned. ## Tuning TopK and Threshold [#tuning-topk-and-threshold] | Symptom | Adjustment | | ----------------------------- | -------------------------------------------------------------- | | answers miss obvious docs | lower `threshold`, raise `topK`, improve titles/chunks | | irrelevant docs enter prompts | raise `threshold`, lower `topK`, improve chunking | | prompts are too large | lower `topK` or shorten formatted text | | model ignores citations | include source ids in text and instruction | | retrieval is slow | use a production vector store and avoid request-time ingestion | ## Test Queries [#test-queries] Test retrieval separately from answer generation. ```ts const matches = await supportDocsIndex.searchIds({ query: "How long does a password reset link last?", topK: 3, threshold: 0.72, }); expect(matches.map((match) => match.id)).toContain("password-reset-policy"); ``` Then test the full agent answer with evals. ## Failure Modes [#failure-modes] | Failure | Fix | | ---------------------------- | -------------------------------------------------------------- | | source is stale | fix ingestion refresh policy | | answer lacks evidence | include source ids in formatted context and traces | | tenant leak | use tenant-specific index handles or enforced metadata filters | | model overuses search tool | use `.dynamicContext(...)` for always-needed knowledge | | retrieval hides missing data | return no context and make the model state uncertainty | ## Test Checklist [#test-checklist] * Search representative queries and assert expected document ids. * Test threshold behavior for weak matches. * Test tenant and visibility filters. * Inspect traces or Studio knowledge view for retrieved documents. * Add evals that fail when required facts are missing. ## Related Docs [#related-docs] * [RAG Ingestion](/docs/best-practices/knowledge-patterns/rag-ingestion) * [RAG Context](/docs/guides/retrieval/rag-context) * [Eval Strategy](/docs/best-practices/quality-observability/eval-strategy) # RAG Ingestion (/docs/best-practices/knowledge-patterns/rag-ingestion) RAG quality starts before the agent runs. Ingestion is the application-owned workflow that loads source material, normalizes it, chooses ids and metadata, embeds it, writes it to a vector store, and decides when indexes are refreshed. Do not rebuild a retrieval index inside a hot prompt path. Build it during deploy, startup, admin ingestion, or background work. ## Scenario [#scenario] A support agent answers product questions from Markdown docs, PDFs, release notes, and internal policy pages. The application needs a repeatable ingestion job so every agent run sees fresh, filtered, traceable context. ## When to Use It [#when-to-use-it] Use this pattern when: * knowledge is larger than static `.context(...)` * documents change outside the request path * documents need metadata filters such as tenant, product, visibility, version, or locale * retrieval evidence should be debugged in traces and evals * multiple agents share the same knowledge source ## Architecture Shape [#architecture-shape] | Layer | Responsibility | | -------------- | ----------------------------------------------------------------------- | | loader | read files, directories, PDFs, bytes, or application records | | normalizer | clean text, split sections, remove boilerplate, attach source ids | | metadata | tenant, product, visibility, version, locale, source path, updated time | | embedding job | call `embedDocuments(...)` with bounded concurrency | | vector store | store embedded documents and expose an index | | refresh policy | rebuild or incrementally update at an explicit boundary | ## Code Example [#code-example] ```ts import { InMemoryVectorStore, embedDocuments } from "@anvia/core"; import { FileLoader, fileLoaderToDocuments } from "@anvia/core/loaders"; import { embeddings } from "./models"; const loaded = await fileLoaderToDocuments( FileLoader.withGlob("content/support/**/*.md") .readWithPath() .ignoreErrors(), ); const documents = loaded.map((doc) => ({ id: doc.id, title: doc.metadata.path, body: doc.text, product: "support", visibility: "public", })); const embedded = await embedDocuments(embeddings, documents, { id: (doc) => doc.id, content: (doc) => `${doc.title}\n${doc.body}`, metadata: (doc) => ({ product: doc.product, visibility: doc.visibility, title: doc.title, }), concurrency: 2, }); export const supportDocsIndex = InMemoryVectorStore.fromDocuments(embedded).index(embeddings); ``` For production storage, build the same embedded document shape and write it to the vector store your application owns. ## Document Ids and Metadata [#document-ids-and-metadata] Stable ids make refreshes and debugging easier. ```ts const document = { id: `support:${locale}:${slug}:${sectionId}`, title, body, locale, product, visibility, sourceUrl, updatedAt, }; ``` Metadata should support the filters and trace evidence you need later. Do not rely on prompt instructions to prevent a tenant or visibility leak. ## Chunking Ownership [#chunking-ownership] Anvia loaders convert source files into documents. Your application owns semantic chunking policy. | Source | Chunk default | | ------------------ | ---------------------------------------------------- | | Markdown docs | one document per page or heading section | | PDFs | one document per page, then merge or split if needed | | tickets or records | one document per record or summary | | code files | one document per file or symbol-level section | | long policies | one document per rule group with source id | ## Refresh Strategy [#refresh-strategy] | Strategy | Use when | | ------------------------------ | ----------------------------------------------- | | deploy-time rebuild | docs change with code deploys | | startup build | small indexes and local development | | background ingestion | docs change frequently or source data is remote | | tenant-specific index | strict tenant isolation is required | | metadata-filtered shared index | source store supports reliable filtering | ## Failure Modes [#failure-modes] | Failure | Fix | | --------------------- | ----------------------------------------------------------- | | stale answers | define refresh triggers and store `updatedAt` metadata | | weak retrieval | improve chunking, titles, content selector, and source text | | tenant leakage | separate indexes or enforce metadata filters before search | | slow ingestion | bound `concurrency` and move ingestion off request paths | | hard-to-debug context | include stable ids, source paths, titles, and versions | ## Test Checklist [#test-checklist] * Test loaders against representative files and PDFs. * Snapshot normalized documents before embedding. * Verify ids are stable across repeated ingestion. * Verify metadata includes tenant, visibility, product, and source fields where needed. * Run search probes and assert expected document ids appear. * Add eval cases for known questions that depend on retrieved facts. ## Related Docs [#related-docs] * [Embed Documents](/docs/guides/retrieval/embed-documents) * [Loaders](/docs/guides/retrieval/loaders) * [RAG Agent Context](/docs/best-practices/knowledge-patterns/rag-agent-context) # Enqueue Pipeline Jobs (/docs/best-practices/long-process-pipelines/enqueue-pipeline-jobs) The enqueue boundary is product code. It should authorize the caller, validate input, create a durable pending record, and then call `queue.add(...)` with a stable job id. ## Job Data [#job-data] ```ts import { z } from "zod"; export const ticketInputSchema = z.object({ tenantId: z.string(), ticketId: z.string(), summary: z.string().min(1), }); export type TicketJobData = z.infer; ``` Keep job data scoped and serializable. Do not enqueue request objects, database clients, provider clients, or unbounded conversation history. ## Queue Setup [#queue-setup] ```ts import { Queue, type JobsOptions } from "bullmq"; import IORedis from "ioredis"; import type { TicketJobData } from "./schema"; export const queueName = "ticket-triage"; const producerConnection = new IORedis(process.env.REDIS_URL); export const triageQueue = new Queue(queueName, { connection: producerConnection, }); ``` Use separate producer and worker connections when API and worker processes have different runtime constraints. ## Job Options [#job-options] ```ts export const jobOptions: JobsOptions = { attempts: 3, backoff: { type: "exponential", delay: 5_000, }, removeOnComplete: { age: 60 * 60 * 24, count: 1_000, }, removeOnFail: { age: 60 * 60 * 24 * 7, }, }; ``` Retries are useful for transient provider, network, and database failures. They are dangerous when the worker performs non-idempotent side effects. ## Enqueue Function [#enqueue-function] ```ts import { ticketInputSchema } from "./schema"; import { jobOptions, triageQueue } from "./queue"; import { ticketJobs } from "./storage"; export async function enqueueTicketTriage(input: unknown) { const data = ticketInputSchema.parse(input); const jobId = `ticket-triage:${data.tenantId}:${data.ticketId}`; await ticketJobs.createPending({ jobId, tenantId: data.tenantId, ticketId: data.ticketId, }); const job = await triageQueue.add("triage", data, { ...jobOptions, jobId, }); return { jobId: job.id, status: "queued", }; } ``` The stable `jobId` prevents duplicate queue jobs for the same product operation. Keep a matching idempotency constraint in app storage because the database is the source of product truth. ## API Boundary [#api-boundary] ```ts import { enqueueTicketTriage } from "./enqueue"; export async function POST(request: Request) { const body = await request.json(); const result = await enqueueTicketTriage(body); return Response.json(result, { status: 202, }); } ``` Do authentication, tenant scoping, quotas, and validation before `queue.add(...)`. The worker should still trust only scoped job data, but the API is the first product boundary. ## Related Docs [#related-docs] * [Request Runners](/docs/best-practices/common-patterns/request-runners) * [Tool Validation and Contracts](/docs/best-practices/tool-patterns/tool-validation-and-contracts) * [Run Pipeline Workers](/docs/best-practices/long-process-pipelines/run-pipeline-workers) # Long Process Pipelines (/docs/best-practices/long-process-pipelines) Use a long process pipeline when the work should not finish inside one HTTP request. The request validates input and creates a job. A worker process runs the Anvia pipeline, persists the result, and reports progress through app-owned status records. ## What is BullMQ? [#what-is-bullmq] [BullMQ](https://docs.bullmq.io/) is a Node.js queue library built on Redis. A [`Queue`](https://docs.bullmq.io/guide/queues) adds jobs, a [`Worker`](https://docs.bullmq.io/guide/workers) processes jobs, and [`QueueEvents`](https://docs.bullmq.io/guide/events) can observe queue lifecycle events. BullMQ uses Redis [connections](https://docs.bullmq.io/guide/connections) so API producers and worker processes can run separately and scale independently. Install BullMQ in the application that owns the queue and worker: ```sh pnpm add bullmq ioredis ``` The docs app does not need these dependencies unless it is also running a queue worker. ## When to Use It [#when-to-use-it] Use this pattern when: * a pipeline can run longer than the caller should wait * work should survive API process restarts * jobs need retry, backoff, delayed execution, or worker concurrency * pipeline results should be persisted for later polling or review * API and worker processes scale independently ## Architecture Shape [#architecture-shape] | Layer | Responsibility | | -------------- | ------------------------------------------------------------------ | | API route | authorize caller, validate input, create app record, enqueue job | | BullMQ queue | store pending jobs, retry policy, backoff, job lifecycle | | worker process | consume jobs, run the Anvia pipeline, update progress | | pipeline | own typed stages, agents, extractors, and deterministic transforms | | app storage | durable status, result, errors, audit metadata | | observability | connect queue job ids, pipeline stage events, traces, and logs | ## Baseline Flow [#baseline-flow] Keep the API response small. Return a job id and store user-facing status in your own database. BullMQ return values are useful for worker internals and queue inspection, but product status pages should read durable app records. ## Pages [#pages] * [Enqueue Pipeline Jobs](/docs/best-practices/long-process-pipelines/enqueue-pipeline-jobs) shows the API producer boundary. * [Run Pipeline Workers](/docs/best-practices/long-process-pipelines/run-pipeline-workers) shows the worker and `pipeline.run(...)` boundary. * [Status, Retries, and Testing](/docs/best-practices/long-process-pipelines/status-retries-testing) covers production behavior and tests. ## Related Docs [#related-docs] * [Pipeline](/docs/best-practices/common-patterns/pipeline) * [Request Runners](/docs/best-practices/common-patterns/request-runners) * [Production Guardrails](/docs/best-practices/common-patterns/production-guardrails) * [Testing and Observability](/docs/best-practices/common-patterns/testing-and-observability) # Run Pipeline Workers (/docs/best-practices/long-process-pipelines/run-pipeline-workers) The worker owns execution. It should mark jobs running, call `pipeline.run(...)`, update progress from pipeline stage events, persist the final result, and let BullMQ retry failures that should be retried. ## Build the Pipeline [#build-the-pipeline] ```ts import { PipelineBuilder } from "@anvia/core"; import { ticketAgent, ticketExtractor } from "./ai"; import type { TicketJobData } from "./schema"; export const ticketPipeline = new PipelineBuilder({ id: "ticket-triage-pipeline", name: "Ticket triage pipeline", }) .step((input) => ({ ...input, summary: input.summary.trim(), })) .prompt(ticketAgent, { name: "Draft triage", }) .extract(ticketExtractor, { name: "Extract ticket result", }) .build(); ``` Build provider clients, agents, extractors, and reusable pipelines at worker startup. Keep request-local user and tenant data inside the job payload. ## Worker Connection [#worker-connection] ```ts import IORedis from "ioredis"; export const workerConnection = new IORedis(process.env.REDIS_URL, { maxRetriesPerRequest: null, }); ``` BullMQ workers need a Redis connection suitable for blocking commands. Keep this separate from short-lived HTTP request behavior. ## Worker Handler [#worker-handler] ```ts import { type PipelineRunEvent } from "@anvia/core"; import { Worker } from "bullmq"; import { queueName } from "./queue"; import { ticketPipeline } from "./pipeline"; import type { TicketJobData } from "./schema"; import { ticketJobs } from "./storage"; import { workerConnection } from "./worker-connection"; export const triageWorker = new Worker( queueName, async (job) => { await ticketJobs.markRunning(job.id!, { startedAt: new Date(), attempt: job.attemptsMade + 1, }); const output = await ticketPipeline.run(job.data, { observer: { async onEvent(event: PipelineRunEvent) { await job.updateProgress({ type: event.type, stage: event.node.label, }); }, }, }); await ticketJobs.markCompleted(job.id!, { completedAt: new Date(), output, }); return { ticketId: job.data.ticketId, status: "completed", }; }, { connection: workerConnection, concurrency: 4, }, ); ``` Use the pipeline observer for metadata-only progress. Do not put sensitive model output, retrieved documents, or tool results into queue progress if those values should only live in app storage or traces. ## Worker Events [#worker-events] ```ts triageWorker.on("failed", async (job, error) => { if (!job) return; await ticketJobs.markFailed(job.id!, { failedAt: new Date(), message: error.message, attempt: job.attemptsMade, }); }); triageWorker.on("error", (error) => { console.error("ticket triage worker error", error); }); ``` The `failed` event is job-level state. The `error` event is worker-level runtime state. Handle both so job status and worker health are visible. ## Related Docs [#related-docs] * [Pipeline](/docs/best-practices/common-patterns/pipeline) * [Tracing and Debugging](/docs/best-practices/quality-observability/tracing-and-debugging) * [Inspect Pipelines](/docs/studio/pipelines/inspect-pipelines) # Status, Retries, and Testing (/docs/best-practices/long-process-pipelines/status-retries-testing) BullMQ owns queue mechanics. Your application owns product status, permissions, idempotency, audit records, and user-facing results. ## Durable Status [#durable-status] ```ts export type TicketJobRecord = { jobId: string; tenantId: string; ticketId: string; status: "queued" | "running" | "completed" | "failed"; output?: unknown; errorMessage?: string; updatedAt: Date; }; ``` Store status in app storage before enqueueing. Update the same record from the worker. Use BullMQ job state for operations and debugging, not as the only user-facing product record. ## Queue Events [#queue-events] ```ts import { QueueEvents } from "bullmq"; import { queueName } from "./queue"; import { workerConnection } from "./worker-connection"; export const triageQueueEvents = new QueueEvents(queueName, { connection: workerConnection, }); triageQueueEvents.on("completed", ({ jobId }) => { console.info("ticket triage job completed", { jobId }); }); triageQueueEvents.on("failed", ({ jobId, failedReason }) => { console.warn("ticket triage job failed", { jobId, failedReason }); }); ``` Queue events are useful for operational logs and metrics. Product state should still be written by the API and worker paths that understand tenant, ticket, and result data. ## Retry Policy [#retry-policy] | Case | Suggested behavior | | --------------------------------- | ----------------------------------------- | | provider timeout | retry with backoff | | temporary Redis or database error | retry with backoff | | invalid job data | fail without re-enqueueing | | permission or tenant mismatch | fail and alert | | non-idempotent side effect | protect with operation id before retrying | If the pipeline writes data, sends messages, or calls external write APIs, use service-layer idempotency keys. A retried worker must not create duplicate side effects. ## Failure Modes [#failure-modes] | Failure | Fix | | ------------------------------------------------- | ------------------------------------------------------------------------ | | duplicate jobs for one ticket | stable `jobId` plus app-level idempotency | | Redis outage blocks API too long | tune producer connection retry behavior and return a retryable API error | | worker retries repeat side effects | keep side effects idempotent and persist operation ids | | user cannot see progress after completion cleanup | store status and result in app storage | | worker process hides emitted errors | attach a worker `error` listener | | pipeline stage is hard to debug | pass a pipeline observer and include job id in logs and traces | ## Test Checklist [#test-checklist] * Test input validation before enqueueing. * Test stable job ids for duplicate enqueue attempts. * Test pending records are created before `queue.add(...)`. * Test worker success writes completed status and output. * Test worker failure writes failed status and leaves retry behavior to BullMQ. * Test idempotent side effects when BullMQ retries the same job. * Test progress updates from pipeline stage events. * Run the pipeline directly in unit tests before testing BullMQ wiring. ## Related Docs [#related-docs] * [Production Guardrails](/docs/best-practices/common-patterns/production-guardrails) * [Testing and Observability](/docs/best-practices/common-patterns/testing-and-observability) * [Side Effect Tools](/docs/best-practices/tool-patterns/side-effect-tools) # MCP Agent Harness (/docs/best-practices/mcp-patterns/mcp-agent-harness) An MCP agent harness is a normal request runner with one extra dependency: connected MCP servers. Keep MCP connection ownership outside prompt logic, then pass validated servers or filtered tools into the scoped agent. ## Scenario [#scenario] A support agent uses local account tools plus a remote docs MCP server. If the docs server is unavailable, the agent can still answer from account tools and say it cannot search docs right now. ## When to Use It [#when-to-use-it] Use this pattern when an agent combines application-owned tools with MCP tools, or when MCP availability changes independently from the rest of the app. ## Architecture Shape [#architecture-shape] | Layer | Responsibility | | ------------ | ------------------------------------------------------ | | registry | owns connected MCP servers and health state | | startup | connects required servers and optional servers | | runner | selects servers or filtered MCP tools for this request | | scoped agent | registers local tools plus MCP tools | | tools | enforce local permissions and side-effect policy | | Studio | inspects registered MCP metadata during development | ## Code Example [#code-example] ```ts import { AgentBuilder, type McpServer } from "@anvia/core"; type McpRegistry = { get(name: string): McpServer | undefined; }; export function createSupportAgent(input: { docsServer?: McpServer; scope: SupportToolScope; }) { const builder = new AgentBuilder("support", model) .instructions(` Answer support questions clearly. Use account tools for customer-specific data. Use docs tools when policy or product documentation is needed. `) .tools(createSupportTools(input.scope)) .defaultMaxTurns(4); if (input.docsServer) { builder.mcp([input.docsServer]); } return builder.build(); } ``` The runner chooses the MCP capability for the current request. ```ts export async function runSupportWithMcp(input: SupportRunnerInput) { const user = await input.auth.requireUser(); const history = await input.conversations.loadMessages(input.conversationId); const docsServer = input.mcp.get("docs"); const agent = createSupportAgent({ docsServer, scope: { userId: user.id, tenantId: user.tenantId, orders: input.services.orders, tickets: input.services.tickets, }, }); const response = await agent .prompt([...history, Message.user(input.message)]) .withTrace({ name: "support-chat", userId: user.id, metadata: { tenantId: user.tenantId, docsMcpAvailable: docsServer !== undefined, }, }) .send(); await input.conversations.append(input.conversationId, response.messages); return response.output; } ``` ## Filter MCP Tools in the Harness [#filter-mcp-tools-in-the-harness] If the docs server exposes more tools than support should use, filter before building the agent. ```ts const docsTools = docsServer?.tools.filter((tool) => ["search_docs", "read_doc"].includes(tool.name), ); const agent = new AgentBuilder("support", model) .tools(createSupportTools(scope)) .tools(docsTools ?? []) .defaultMaxTurns(4) .build(); ``` ## Error Handling [#error-handling] MCP call errors are tool errors during an agent run. Keep the agent bounded and make server health visible in trace metadata. ```ts const response = await agent .prompt(message) .withTrace({ name: "support-chat", metadata: { docsMcpAvailable: docsServer !== undefined, }, }) .maxTurns(3) .send(); ``` Use registry health checks and reconnects outside the prompt run. The model should not be responsible for repairing infrastructure. ## Failure Modes [#failure-modes] | Failure | Fix | | --------------------------------------- | --------------------------------------------------------------------------- | | optional MCP unavailable | build agent without it and include trace metadata | | wrong MCP tools exposed | filter or wrap tools before registration | | remote tool loops after errors | keep max turns low | | credentials are request-scoped | connect in `try/finally` for that request | | Studio does not show expected MCP tools | verify `.mcp([server])` or filtered tools are registered on the built agent | ## Test Checklist [#test-checklist] * Test runner behavior with MCP available and unavailable. * Test filtered MCP tool list includes only allowed names. * Test required server validation during startup. * Test traces include MCP availability metadata. * Use Studio MCP inspection for local verification. ## Related Docs [#related-docs] * [MCP Server Lifecycle](/docs/best-practices/mcp-patterns/mcp-server-lifecycle) * [MCP Tool Inspection](/docs/best-practices/mcp-patterns/mcp-tool-inspection) * [Support Agent](/docs/best-practices/real-cases/support-agent) # MCP Server Lifecycle (/docs/best-practices/mcp-patterns/mcp-server-lifecycle) MCP servers expose external tools to an agent. Treat each MCP connection as infrastructure: decide whether it is required, when it connects, how it is closed, and what happens when it fails. ## Scenario [#scenario] An agent needs tools from a filesystem MCP server, an internal docs MCP server, and a remote CRM MCP server. Some are required for startup. Others are optional and should degrade gracefully. ## When to Use It [#when-to-use-it] Use this pattern whenever an agent registers tools with `.mcp(...)` or wraps MCP tools with local Anvia tools. ## Architecture Shape [#architecture-shape] | Boundary | Pattern | | --------------------------- | ------------------------------------------------------------------------------ | | required startup dependency | connect once during application boot and fail fast on error | | optional dependency | attempt connection, log failure, run agent without that server | | request-scoped server | connect inside `try/finally` and always close | | long-lived process | close all servers during shutdown | | reconnect | close previous server, connect next server, rebuild agents or update tool sets | ## Connect at Startup [#connect-at-startup] ```ts import { connectMcp, mcp, type McpConnection, type McpServer } from "@anvia/core"; export async function connectMcpServers() { const filesystem = await connectMcp( mcp.stdio({ name: "filesystem", command: "npx", args: ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"], }), ); const docs = await connectMcp( mcp.http({ name: "docs", url: "https://mcp.example.com/mcp", }), ); return { filesystem, docs }; } ``` Register connected servers on an agent. ```ts const { filesystem, docs } = await connectMcpServers(); const agent = new AgentBuilder("research", model) .instructions("Use MCP tools when external context is needed.") .mcp([filesystem, docs]) .defaultMaxTurns(4) .build(); ``` ## Optional Servers [#optional-servers] Optional servers should not block the whole application. ```ts async function connectOptional(connection: McpConnection) { try { return await connectMcp(connection); } catch (error) { logger.warn({ error, server: connection.name }, "optional MCP server unavailable"); return undefined; } } const docs = await connectOptional( mcp.http({ name: "docs", url: "https://mcp.example.com/mcp" }), ); const mcpServers = [docs].filter((server): server is McpServer => server !== undefined); ``` ## Request-Scoped Connections [#request-scoped-connections] Use request scope only when the connection depends on request-local credentials or a short-lived resource. ```ts const server = await connectMcp(connection); try { const agent = new AgentBuilder("tenant-docs", model) .mcp([server]) .defaultMaxTurns(3) .build(); return await agent.prompt(message).send(); } finally { await server.close(); } ``` Most application servers should prefer startup connections over request-scoped connections. ## Registry and Reconnect [#registry-and-reconnect] ```ts type McpRegistry = { get(name: string): McpServer | undefined; set(server: McpServer): void; closeAll(): Promise; }; async function reconnect(connection: McpConnection, registry: McpRegistry) { const previous = registry.get(connection.name); await previous?.close(); const next = await connectMcp(connection); registry.set(next); return next; } ``` After reconnecting, create new agents or update the shared tool catalog used by future runs. A built agent keeps the tools it was built with unless it uses a shared mutable `ToolSet`. ## Failure Modes [#failure-modes] | Failure | Fix | | ------------------------------------- | ----------------------------------------------------- | | app starts without required MCP tools | fail fast during startup | | optional MCP outage breaks all agents | omit optional server and log degraded capability | | server process leaks | close on shutdown or in `finally` | | reconnected tools not visible | rebuild agents or update the shared tool set | | tool call loops on remote errors | keep turn limits low and expose clear tool error text | ## Test Checklist [#test-checklist] * Test required server connection failure at startup. * Test optional server failure produces a reduced agent capability set. * Test request-scoped connections close in `finally`. * Test reconnect closes the previous server before replacing it. * Use Studio or `/agents/:agentId/mcps` to inspect registered MCP servers. ## Related Docs [#related-docs] * [MCP Connections](/docs/guides/mcp/connections) * [Errors and Reconnects](/docs/guides/mcp/errors-and-reconnects) * [MCP Agent Harness](/docs/best-practices/mcp-patterns/mcp-agent-harness) # MCP Tool Inspection (/docs/best-practices/mcp-patterns/mcp-tool-inspection) MCP tools are adapted automatically, but a production harness should still inspect what the server exposed. Tool names, descriptions, input schemas, and permission expectations decide whether a tool is safe to show to a model. ## Scenario [#scenario] A remote MCP server exposes 40 tools. Your support agent should use only read-only documentation tools, while an internal admin agent can use a smaller set of write tools behind approval. ## When to Use It [#when-to-use-it] Use this pattern when: * a server exposes more tools than one agent should see * tool names are unclear or conflict with local tools * a tool needs product permissions or audit records * you need to verify required tools exist before startup succeeds ## Architecture Shape [#architecture-shape] | Step | Purpose | | ----------------------- | ---------------------------------------------------------------- | | connect server | get adapted Anvia tools | | inspect definitions | verify name, description, and schema | | validate required tools | fail fast for missing required capabilities | | filter safe tools | expose only the tools this agent should use | | wrap tools | rename, audit, permission-check, or normalize output when needed | ## Inspect Definitions [#inspect-definitions] ```ts import { connectMcp, mcp, type McpServer } from "@anvia/core"; const docsServer = await connectMcp( mcp.http({ name: "docs", url: "https://mcp.example.com/mcp", }), ); for (const tool of docsServer.tools) { const definition = await tool.definition(""); logger.info({ server: docsServer.name, tool: definition.name, description: definition.description, parameters: definition.parameters, }); } ``` ## Validate Required Tools [#validate-required-tools] ```ts async function requireMcpTools(server: McpServer, names: string[]) { const available = new Set(server.tools.map((tool) => tool.name)); const missing = names.filter((name) => !available.has(name)); if (missing.length > 0) { throw new Error(`MCP server ${server.name} is missing tools: ${missing.join(", ")}`); } } await requireMcpTools(docsServer, ["search_docs", "read_doc"]); ``` ## Filter Tools [#filter-tools] ```ts function pickMcpTools(server: McpServer, allowedNames: string[]) { const allowed = new Set(allowedNames); return server.tools.filter((tool) => allowed.has(tool.name)); } const safeDocsTools = pickMcpTools(docsServer, ["search_docs", "read_doc"]); const agent = new AgentBuilder("support", model) .instructions("Use docs tools for public support documentation.") .tools(safeDocsTools) .defaultMaxTurns(3) .build(); ``` Use `.mcp([server])` when the agent should receive every tool from that server. Use `.tools(filteredTools)` when the agent should receive only selected tools. ## Wrap Tools [#wrap-tools] Wrap an MCP tool when you need product-specific names, permissions, audit records, or normalized output. ```ts import { createTool } from "@anvia/core"; import { z } from "zod"; const searchDocs = docsServer.tools.find((tool) => tool.name === "search_docs"); if (!searchDocs) { throw new Error("docs MCP server is missing search_docs"); } const searchInternalDocs = createTool({ name: "search_internal_docs", description: "Search internal support documentation for the current tenant.", input: z.object({ query: z.string().min(1), }), async execute({ query }) { await permissions.require(scope.userId, "docs:read"); const result = await searchDocs.call({ query: `${query}\ntenant:${scope.tenantId}`, }); await audit.write({ actorId: scope.userId, action: "docs.search", tenantId: scope.tenantId, }); return result; }, }); ``` ## Failure Modes [#failure-modes] | Failure | Fix | | -------------------------------- | ---------------------------------- | | server exposes unsafe tools | filter or wrap before registering | | tool names conflict | wrap with product-specific names | | schema is too loose | wrap with a stricter Zod schema | | server changes tool names | validate required tools at startup | | remote tool returns noisy output | wrap and normalize result shape | ## Test Checklist [#test-checklist] * Assert required tools exist after connection. * Snapshot provider-facing tool definitions for critical MCP tools. * Test wrapped tools with fake permission and audit services. * Verify agents receive only allowed MCP tools. * Inspect MCP metadata in Studio when available. ## Related Docs [#related-docs] * [MCP Tool Adapters](/docs/guides/mcp/tool-adapters) * [MCP Result Handling](/docs/guides/mcp/result-handling) * [Tool Validation and Contracts](/docs/best-practices/tool-patterns/tool-validation-and-contracts) # Build Streaming Specialists (/docs/best-practices/multi-agent-patterns/build-streaming-specialists) Build multi-agent streaming around one coordinator and a small set of narrow specialists. Each specialist should have a clear role, stable agent id, and focused output. ## Build Specialist Agents [#build-specialist-agents] ```ts import { AgentBuilder } from "@anvia/core"; import { model } from "./model"; export const supportAgent = new AgentBuilder("support", model) .name("Support Specialist") .description("Summarize customer impact and support next steps.") .instructions("Return compact support triage bullets using only the provided facts.") .build(); export const engineeringAgent = new AgentBuilder("engineering", model) .name("Engineering Specialist") .description("Summarize diagnostics and engineering next steps.") .instructions("Return compact engineering triage bullets without unverified root-cause claims.") .build(); export const commsAgent = new AgentBuilder("communications", model) .name("Communications Specialist") .description("Draft customer-facing incident communication guidance.") .instructions("Return concise customer-safe communication notes.") .build(); ``` Keep specialist instructions narrow. Each child agent should own one role and return a focused result that the coordinator can use. ## Expose Specialists With Streaming [#expose-specialists-with-streaming] ```ts import { AgentBuilder } from "@anvia/core"; import { commsAgent, engineeringAgent, supportAgent } from "./specialists"; import { model } from "./model"; export const coordinator = new AgentBuilder("incident-coordinator", model) .name("Incident Coordinator") .instructions( [ "Coordinate specialist agents through tools.", "Call specialists only when their expertise is useful.", "Combine specialist findings into one concise incident brief.", "Do not expose internal-only notes in the final customer-facing summary.", ].join("\n"), ) .tools([ supportAgent.asTool({ name: "ask_support_agent", stream: true }), engineeringAgent.asTool({ name: "ask_engineering_agent", stream: true }), commsAgent.asTool({ name: "ask_comms_agent", stream: true }), ]) .defaultMaxTurns(4) .build(); ``` The coordinator should decide which specialists to call. Do not attach every possible specialist if the model cannot choose reliably. ## Run the Coordinator Stream [#run-the-coordinator-stream] ```ts const prompt = [ "Acme Co. reports webhook retries fail for payloads larger than 512 KB.", "They have missed several order updates in the last hour.", "Prepare an incident brief for support, engineering, and communications.", ].join(" "); for await (const event of coordinator .prompt(prompt) .withToolConcurrency(3) .withTrace({ name: "incident-multi-agent-stream", metadata: { incidentId: "inc_123", tenantId: "tenant_123", }, }) .stream()) { renderEvent(event); } ``` Use `.withToolConcurrency(...)` only when specialist calls are independent. If one specialist must wait for another specialist's finding, keep concurrency at `1` or split the workflow into explicit stages. ## Related Docs [#related-docs] * [Multi-Agent Workflows](/docs/guides/agents/multi-agent-workflows) * [Agent Structure](/docs/best-practices/common-patterns/agent-structure) * [Consume Nested Events](/docs/best-practices/multi-agent-patterns/consume-nested-events) # Consume Nested Events (/docs/best-practices/multi-agent-patterns/consume-nested-events) When a coordinator runs with `.stream()`, normal parent events and nested child-agent events arrive in the same stream. Render coordinator text directly and group child progress by `internalCallId`. ## Render Parent and Child Events [#render-parent-and-child-events] ```ts import type { AgentStreamEvent } from "@anvia/core"; function renderEvent(event: AgentStreamEvent): void { switch (event.type) { case "text_delta": renderCoordinatorText(event.delta); break; case "tool_call": renderDelegation(event.toolCall.function.name); break; case "agent_tool_event": renderSpecialistEvent({ groupId: event.internalCallId, agentLabel: event.agentName ?? event.agentId, toolName: event.toolName, event: event.event, }); break; case "final": saveFinalResponse(event); break; case "error": renderStreamError(event.error); break; } } ``` Group nested progress by `internalCallId` when the same specialist can be called more than once. Use `agentId`, `agentName`, and `toolName` for labels. ## Render Specialist Progress [#render-specialist-progress] ```ts import type { AgentStreamEvent } from "@anvia/core"; type SpecialistEvent = Extract; function renderSpecialistEvent(input: { groupId: string; agentLabel: string; toolName: string; event: SpecialistEvent["event"]; }) { if (input.event.type === "text_delta") { appendSpecialistText(input.groupId, input.agentLabel, input.event.delta); } if (input.event.type === "tool_call") { showSpecialistToolCall(input.groupId, input.event.toolCall.function.name); } if (input.event.type === "tool_result") { markSpecialistToolDone(input.groupId, input.event.toolName); } if (input.event.type === "final") { markSpecialistComplete(input.groupId, input.event.output); } } ``` Do not render raw tool results, sensitive retrieved context, or internal reasoning in a user-facing UI unless the product intentionally exposes those details. Prefer status labels and safe summaries for customer-facing screens. ## Event Mapping [#event-mapping] | Event | UI use | | ------------------------------ | --------------------------------------------------- | | `tool_call` | coordinator is delegating to a specialist | | `agent_tool_event.text_delta` | specialist is streaming visible progress | | `agent_tool_event.tool_call` | specialist is using one of its tools | | `agent_tool_event.tool_result` | specialist tool completed | | `agent_tool_event.final` | specialist returned final output to the coordinator | | `tool_result` | coordinator received the specialist's final output | | `text_delta` | coordinator is streaming the final answer | | `final` | persist final messages, usage, trace, and run id | ## Related Docs [#related-docs] * [Streaming Events](/docs/guides/streaming/streaming-events) * [Readable Streams](/docs/guides/streaming/readable-streams) * [Build Streaming Specialists](/docs/best-practices/multi-agent-patterns/build-streaming-specialists) # Multi-agent Streaming (/docs/best-practices/multi-agent-patterns) Use multi-agent streaming when one coordinator agent delegates work to specialist agents and the caller should see progress while the specialists run. The coordinator still owns the final answer, but the UI can show nested child-agent activity through `agent_tool_event`. ## Scenario [#scenario] An incident coordinator receives a customer-impact report. It delegates to support, engineering, and communications specialists. The user sees the coordinator's final brief stream in normally, while each specialist's progress appears in grouped status panels. ## When to Use It [#when-to-use-it] Use this pattern when: * one coordinator should decide which specialists to call * specialist work may take long enough that hidden progress feels stalled * the UI should group nested progress by specialist or tool call * the final answer should still come from one coordinator * child-agent progress should be replayable or inspectable after the run Do not use this pattern just to run known independent work. Use parallel pipelines when every branch should run and the coordinator does not need to decide. ## Architecture Shape [#architecture-shape] | Layer | Responsibility | | --------------------- | -------------------------------------------------------------------- | | coordinator agent | decides which specialists to call and writes the final response | | specialist agents | own narrow role instructions and return focused findings | | streaming agent tools | expose specialists with `asTool({ stream: true })` | | request runner | applies trace metadata, tool concurrency, and stream response shape | | UI stream consumer | renders parent events and groups nested `agent_tool_event` values | | event store | optionally persists parent and child events for replay and debugging | ## Requirements [#requirements] Nested streaming requires: * the parent run uses `.stream()` * specialist tools are created with `stream: true` * the child model supports streaming If nested streaming is unavailable, the specialist still behaves like a normal opaque `asTool(...)` call. The parent model receives only the final specialist output as the normal `tool_result`; intermediate child events are for the caller, trace, or event store. ## Pages [#pages] * [Build Streaming Specialists](/docs/best-practices/multi-agent-patterns/build-streaming-specialists) shows how to build specialist agents, expose them with `asTool({ stream: true })`, and run the coordinator stream. * [Consume Nested Events](/docs/best-practices/multi-agent-patterns/consume-nested-events) shows how to group and render parent and child events safely. * [Persistence and Testing](/docs/best-practices/multi-agent-patterns/persistence-and-testing) covers event-store replay, failure modes, and test coverage. ## Choose the Right Shape [#choose-the-right-shape] | Shape | Use it when | | -------------------------------- | -------------------------------------------------------------------------------------- | | `agent.asTool(...)` | child progress can stay hidden | | `agent.asTool({ stream: true })` | coordinator decides which specialists to call and the caller should see child progress | | parallel pipelines | every branch should always run and return typed outputs | | separate Studio agents | operators should run specialists manually during development | ## Related Docs [#related-docs] * [Multi-Agent Workflows](/docs/guides/agents/multi-agent-workflows) * [Streaming Events](/docs/guides/streaming/streaming-events) * [Readable Streams](/docs/guides/streaming/readable-streams) * [Pipeline](/docs/best-practices/common-patterns/pipeline) # Persistence and Testing (/docs/best-practices/multi-agent-patterns/persistence-and-testing) Multi-agent streams are useful live, but production systems often need replay, support inspection, and regression coverage. Persist events when users may refresh, support teams need a run history, or child-agent progress is part of the product record. ## Persist Events for Replay [#persist-events-for-replay] Add an event store when nested progress must be replayed after the run, inspected in support tools, or attached to a product audit trail. ```ts import { AgentBuilder, type AgentEventStore } from "@anvia/core"; import { supportAgent } from "./specialists"; import { model } from "./model"; declare const eventStore: AgentEventStore; export const coordinatorWithEvents = new AgentBuilder("incident-coordinator", model) .instructions("Delegate support triage, then produce a short final brief.") .tool(supportAgent.asTool({ name: "ask_support_agent", stream: true })) .eventStore(eventStore, { include: "all" }) .defaultMaxTurns(3) .build(); ``` The final stream event includes `runId`. Use that id to load saved parent and child events from the event store. ## Failure Modes [#failure-modes] | Failure | Fix | | -------------------------------------- | --------------------------------------------------------------------------------- | | child progress never appears | check parent `.stream()`, `stream: true`, and child model streaming support | | nested events are hard to group | group by `internalCallId`, then label with `agentName` or `toolName` | | coordinator calls too many specialists | tighten instructions, lower max turns, or reduce available specialist tools | | tool results leak sensitive data | render safe status labels and persist sensitive details only in protected storage | | parallel calls overload services | lower `.withToolConcurrency(...)` or add service-level limits | | progress disappears after refresh | attach an event store and replay by `runId` | ## Test Checklist [#test-checklist] * Test specialist agents directly with representative prompts. * Test coordinator tool registration includes `stream: true` for visible specialists. * Test stream handling for `agent_tool_event`, `tool_result`, `text_delta`, `final`, and `error`. * Test UI grouping when the same specialist is called more than once. * Test event-store replay with saved parent and child events. * Inspect a provider-backed run in Studio or traces before exposing nested progress to users. ## Related Docs [#related-docs] * [Event Store](/docs/guides/agents/event-store) * [Testing and Observability](/docs/best-practices/common-patterns/testing-and-observability) * [Tracing and Debugging](/docs/best-practices/quality-observability/tracing-and-debugging) * [Consume Nested Events](/docs/best-practices/multi-agent-patterns/consume-nested-events) # Production Readiness Checklist (/docs/best-practices/operations/production-readiness-checklist) Use this checklist before deploying an agent harness to production or expanding it from read-only behavior into product actions. ## Runtime Boundaries [#runtime-boundaries] | Check | Ready when | | --------------- | ------------------------------------------------------------------------------ | | stable agent id | traces, sessions, Studio, and logs use a stable id | | runner boundary | route or job calls a named runner instead of embedding all prompt logic inline | | scoped tools | request-local user, tenant, services, and transactions are passed explicitly | | turn limits | every agent has `.defaultMaxTurns(...)`, and high-risk runs override lower | | timeouts | HTTP routes, jobs, and approval waits have app-owned timeout policy | ## Tools [#tools] | Check | Ready when | | ----------------- | ---------------------------------------------------------------------- | | input schemas | every tool validates model arguments | | output contracts | important tool results are typed states or validated output schemas | | permission checks | every data-bearing tool checks actor and tenant scope | | side effects | writes use service-layer transactions, idempotency, and audit records | | direct tests | high-risk tools are tested through direct calls or `ToolSet.call(...)` | ## Dynamic Tools [#dynamic-tools] | Check | Ready when | | ----------------- | ------------------------------------------------------------------------- | | catalog ownership | a `ToolSet` owns the full catalog | | index lifecycle | tool index is built at startup, deploy time, or another explicit boundary | | search tests | representative prompts select expected tool ids | | critical tools | required tools are either static or reliably selected | | trace review | traces show which dynamic tools were available during runs | ## MCP [#mcp] | Check | Ready when | | --------------- | --------------------------------------------------------------------------- | | lifecycle | required servers fail fast, optional servers degrade gracefully | | tool inspection | required MCP tools are validated after connection | | filtering | agents receive only the MCP tools they should use | | reconnect | reconnect closes the previous server and updates future agents | | cleanup | long-lived servers close during shutdown, scoped servers close in `finally` | ## Context and Memory [#context-and-memory] | Check | Ready when | | ----------------- | --------------------------------------------------------------------------- | | instructions | durable behavior is in instructions | | request facts | current user, tenant, plan, locale, or flags are loaded by the runner | | retrieval filters | indexes or searches enforce tenant and access boundaries | | history | conversation history is stored and replayed deliberately | | session policy | workflows use explicit history or sessions deliberately, not casually mixed | ## Observability [#observability] | Check | Ready when | | ------------------ | ------------------------------------------------------------------------------ | | trace names | every workflow uses stable names such as `support-chat` | | metadata | traces include safe ids such as tenant, conversation, ticket, or channel | | usage | usage is available for cost and regression checks | | external telemetry | observers or integrations are attached where needed | | Studio | local Studio can inspect tools, MCPs, context, sessions, approvals, and traces | ## Testing and Evals [#testing-and-evals] | Check | Ready when | | ----------------- | ------------------------------------------------------------------------- | | unit tests | tools, services, runners, filters, and known errors are tested with fakes | | integration tests | narrow provider-backed tests cover model-dependent behavior | | evals | known prompts or regression cases have repeatable evals | | approval tests | approval accepted, rejected, and timeout paths are covered | | MCP tests | missing, unavailable, and filtered MCP tools are covered | ## Deployment [#deployment] | Check | Ready when | | --------- | ---------------------------------------------------------------------------- | | secrets | provider and MCP credentials are server-only | | storage | history, memory, traces, audit, approval, and idempotency stores are durable | | retries | job retries are safe for side effects or disabled for non-idempotent runs | | streaming | proxies and platforms do not buffer streams that need live events | | rollback | agents can be rebuilt with a previous prompt, tool catalog, or MCP config | ## Smoke Test Shape [#smoke-test-shape] ```ts const result = await runSupportTurn({ conversationId: "smoke_test", message: "Say hello and do not call tools.", auth: smokeAuth, conversations: smokeConversations, services: smokeServices, }); expect(result.ok).toBe(true); ``` For tool workflows, add a smoke test that calls one read-only tool and verifies trace metadata. For side-effect workflows, run against sandbox services with explicit idempotency records. ## Related Docs [#related-docs] * [Testing and Observability](/docs/best-practices/common-patterns/testing-and-observability) * [Eval Strategy](/docs/best-practices/quality-observability/eval-strategy) * [Tracing and Debugging](/docs/best-practices/quality-observability/tracing-and-debugging) * [RAG Ingestion](/docs/best-practices/knowledge-patterns/rag-ingestion) * [Production Guardrails](/docs/best-practices/common-patterns/production-guardrails) * [MCP Server Lifecycle](/docs/best-practices/mcp-patterns/mcp-server-lifecycle) # Eval Strategy (/docs/best-practices/quality-observability/eval-strategy) Evals are for behavior that cannot be fully proven with unit tests. Use unit tests for deterministic boundaries, then use eval suites for model-dependent behavior, retrieval quality, answer quality, tool choice, and regressions. Do not make every route test call a provider. Keep provider-backed evals focused and repeatable. ## Scenario [#scenario] A support agent should answer refund-policy questions correctly, use account tools only when needed, and avoid claiming a refund happened unless the refund tool confirms it. Unit tests cover tools and services. Evals cover model behavior. ## When to Use It [#when-to-use-it] Use evals when: * prompts, instructions, tools, or retrieval change often * failures are quality regressions rather than type errors * you need a repeatable signal before deploying prompt changes * outputs are judged by required facts, schema, similarity, or a rubric * traces or Langfuse scores should connect eval outcomes to runs ## Architecture Shape [#architecture-shape] | Layer | Test with | | ----------------------------- | ------------------------------------------------------- | | service and permission policy | unit tests with fakes | | tool contracts | direct `tool.call(...)` or `ToolSet.call(...)` | | retrieval selection | search probes against indexes | | runner response shape | unit tests with fake agents or sandbox agents | | agent answer quality | eval suite with `agentEvalTarget(...)` or custom target | | traced workflow | eval reporter plus trace metadata | ## Golden Cases [#golden-cases] Store cases close to the workflow they protect. Keep them small and named by the behavior they assert. ```ts const supportCases = [ { id: "refund-window", input: "How long do I have to request a refund?", expected: "30 days", }, { id: "password-reset-expiry", input: "Can I use yesterday's password reset link?", expected: "30 minutes", }, ]; ``` ## Agent Eval [#agent-eval] ```ts import { agentEvalTarget, contains, runEvalSuite } from "@anvia/core/evals"; const result = await runEvalSuite({ name: "support-agent-regression", cases: supportCases, target: agentEvalTarget(supportAgent), metrics: [ contains(), ], }); console.log(result.passed, result.failed, result.invalid); ``` Use `agentEvalTarget(...)` when the target is exactly `agent.prompt(input).send()`. ## Custom Harness Eval [#custom-harness-eval] Use a custom target when the real behavior lives in your runner. ```ts const result = await runEvalSuite({ name: "support-runner-regression", cases: supportCases, target: async (input) => { const response = await runSupportTurn({ conversationId: `eval:${input}`, message: input, auth: evalAuth, conversations: evalConversations, services: evalServices, }); return response.output; }, metrics: [contains()], }); ``` This covers the harness boundary: history, tools, retrieval, trace metadata, and final output. ## Metric Choice [#metric-choice] | Metric | Use it for | | ------------------------- | -------------------------------------------------- | | `exactMatch(...)` | deterministic booleans, labels, JSON-shaped values | | `contains(...)` | required phrases, facts, or regex matches | | `semanticSimilarity(...)` | answer similarity when wording can vary | | `llmJudge(...)` | schema-shaped rubric checks | | `llmScore(...)` | scored quality feedback with a threshold | Start with deterministic metrics. Add LLM judges when the behavior cannot be checked with simple selectors. ## Report to Langfuse [#report-to-langfuse] ```ts import { createLangfuseEvalReporter, langfuse } from "@anvia/langfuse"; const tracing = langfuse.create({ publicKey, secretKey, baseUrl }); await runEvalSuite({ name: "support-agent-regression", cases: supportCases, target: agentEvalTarget(supportAgent), metrics: [contains()], reporters: [createLangfuseEvalReporter(tracing)], }); ``` When your target returns a prompt response with trace ids, reporters can connect eval scores back to traces. ## Failure Modes [#failure-modes] | Failure | Fix | | -------------------------- | ---------------------------------------------------------- | | evals are flaky | prefer deterministic metrics and sandbox services | | cases are too broad | split into one behavior per case | | evals duplicate unit tests | move deterministic policy checks back to unit tests | | judges hide regressions | store judge feedback and add required fact metrics | | no trace linkage | include trace metadata or return prompt response trace ids | ## Test Checklist [#test-checklist] * Cover tool contracts with unit tests before provider evals. * Add retrieval probe tests before answer-quality evals. * Keep golden cases small, named, and versioned. * Run evals on prompt, retrieval, tool, and model changes. * Report eval scores to the same observability system used for traces when useful. ## Related Docs [#related-docs] * [Evals](/docs/guides/testing/evals) * [Testing and Observability](/docs/best-practices/common-patterns/testing-and-observability) * [Tracing and Debugging](/docs/best-practices/quality-observability/tracing-and-debugging) # Tracing and Debugging (/docs/best-practices/quality-observability/tracing-and-debugging) Tracing gives you the evidence needed to debug an agent harness: prompt runs, model generations, tool calls, errors, usage, retrieval evidence, trace metadata, and eval scores. Use traces for runtime debugging. Use product logs and audit records for product accountability. ## Scenario [#scenario] A support answer was wrong. You need to know which prompt ran, which user and conversation it belonged to, which tools were available, which retrieval documents were injected, what the model returned, and whether the same case fails in evals. ## When to Use It [#when-to-use-it] Use this pattern for every production harness that handles real users, private data, side effects, or recurring quality checks. ## Architecture Shape [#architecture-shape] | Layer | Responsibility | | ----------------- | ----------------------------------------------------------------------------- | | observer | records runs, tool calls, model calls, errors, and usage | | `.withTrace(...)` | attaches workflow name, user id, session id, tags, version, and safe metadata | | product logs | store trace ids next to product events | | Studio | inspect local runs, tools, MCPs, context, approvals, and traces | | external tracing | long-term search, metrics, dashboards, eval scores | | shutdown | flush and close buffered telemetry | ## Code Example [#code-example] ```ts import { AgentBuilder } from "@anvia/core"; import { langfuse } from "@anvia/langfuse"; const tracing = langfuse.create({ publicKey, secretKey, baseUrl, }); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .observe(tracing) .defaultMaxTurns(3) .build(); const response = await agent .prompt(message) .withTrace({ name: "support-chat", userId: user.id, sessionId: conversationId, tags: ["support", channel], version: "2026-05-11", metadata: { tenantId: user.tenantId, conversationId, plan: user.plan, }, }) .send(); logger.info({ traceId: response.trace?.traceId, observationId: response.trace?.observationId, conversationId, }, "support agent completed"); ``` ## Trace Metadata Rules [#trace-metadata-rules] | Include | Avoid | | ------------------------------- | ----------------------------------- | | stable workflow name | raw secrets or API keys | | user id when safe | full prompts in app logs | | tenant id | large records | | conversation, ticket, or job id | private document bodies in metadata | | model or prompt version | unbounded objects | | channel, route, or feature flag | values your policy forbids storing | Trace metadata should connect systems. It should not become a second database. ## Debugging Flow [#debugging-flow] 1. Find the product event, ticket, conversation, or eval case. 2. Open the linked trace id. 3. Check prompt input, instructions, retrieved context, and available tools. 4. Check tool calls, tool outputs, errors, and turn count. 5. Reproduce with the same case in Studio or an eval suite. 6. Fix the deterministic boundary first: tool, retrieval, runner, or prompt. 7. Add an eval case when the issue is model-dependent. ## Local Studio vs External Tracing [#local-studio-vs-external-tracing] | Use Studio | Use external tracing | | -------------------------------- | ------------------------ | | local development | production traffic | | inspect agent registration | long-term trace search | | exercise approvals and questions | aggregate cost and usage | | debug context and MCP visibility | dashboards and alerting | | iterate before product UI exists | eval score reporting | Studio and tracing are complementary. Studio shortens local iteration; tracing preserves production evidence. ## Flush and Shutdown [#flush-and-shutdown] ```ts await tracing.flush?.(); await tracing.shutdown?.(); ``` Call `flush()` before process exit when pending events matter. Call `shutdown()` during application shutdown for long-lived integrations. ## Failure Modes [#failure-modes] | Failure | Fix | | --------------------------------------- | ------------------------------------------------- | | traces cannot be tied to product events | log `traceId` and product ids together | | trace metadata leaks sensitive data | restrict metadata to ids and small safe fields | | eval scores are disconnected | report evals with trace ids or case metadata | | tool failures are invisible | attach observer before building production agents | | buffered traces are missing | flush or shutdown the observer on exit | ## Test Checklist [#test-checklist] * Assert runners attach stable trace names. * Assert safe metadata includes product correlation ids. * Verify `response.trace` is logged or returned where needed. * Inspect one local Studio run for context, tools, and trace evidence. * Verify external observer flushes during shutdown. ## Related Docs [#related-docs] * [Tracing](/docs/guides/observability/tracing) * [Langfuse](/docs/guides/observability/langfuse) * [Eval Strategy](/docs/best-practices/quality-observability/eval-strategy) # Backoffice Agent (/docs/best-practices/real-cases/backoffice-agent) This pattern is for internal operations agents that can change product state. The harness must treat every write as a product operation with permissions, approvals, idempotency, and audit records. ## Scenario [#scenario] A support lead asks an agent to resolve a billing ticket, issue a refund, and notify the customer. The model can help coordinate the work, but application code owns every restricted operation. ## When to Use It [#when-to-use-it] Use this pattern when the agent can: * issue refunds * close or transition tickets * update accounts * send customer messages * trigger jobs or webhooks * access admin-only data ## Architecture Shape [#architecture-shape] | Layer | Responsibility | | ---------------- | ----------------------------------------------------------------- | | runner | resolve admin actor, ticket, tenant, approval run, trace metadata | | agent | coordinate the workflow and call tools | | write tools | validate inputs and call product services | | approval runtime | store decisions, notify reviewers, wait or resume | | service layer | permissions, transactions, idempotency, audit | | storage | ticket state, operation records, audit records, traces | ## Code Example [#code-example] ```ts import { AgentBuilder, Message, createHook } from "@anvia/core"; import { model } from "./model"; import { createBackofficeTools } from "./tools"; export async function runBackofficeResolution(input: BackofficeInput) { const actor = await input.auth.requireAdmin(); const ticket = await input.tickets.getForTenant(input.ticketId, actor.tenantId); const approvalHook = createHook({ async onToolCall({ toolName, tool }) { if (!["issue_refund", "send_customer_email", "close_account"].includes(toolName)) { return tool.run(); } const approved = await input.approvals.waitForDecision({ actorId: actor.id, tenantId: actor.tenantId, ticketId: ticket.id, toolName, }); return approved ? tool.run() : tool.cancel("Operation was not approved."); }, }); const agent = new AgentBuilder("backoffice", model) .instructions(` Help resolve backoffice tickets. Use tools for account data and product changes. Never claim that a side effect happened unless the tool result confirms it. `) .tools( createBackofficeTools({ actorId: actor.id, tenantId: actor.tenantId, ticketId: ticket.id, billing: input.services.billing, tickets: input.services.tickets, messages: input.services.messages, audit: input.audit, }), ) .hook(approvalHook) .defaultMaxTurns(5) .build(); const response = await agent .prompt([ Message.user(`Ticket ${ticket.id}: ${ticket.summary}`), Message.user(input.instruction), ]) .withTrace({ name: "backoffice-resolution", userId: actor.id, metadata: { tenantId: actor.tenantId, ticketId: ticket.id, }, }) .send(); await input.audit.write({ actorId: actor.id, tenantId: actor.tenantId, action: "backoffice.agent_run", targetId: ticket.id, }); return response.output; } ``` ## Write Tool Shape [#write-tool-shape] ```ts async execute({ orderId, amount, reason }) { const operationId = `refund:${scope.tenantId}:${orderId}:${amount}`; const result = await scope.billing.issueRefund({ actorId: scope.actorId, tenantId: scope.tenantId, orderId, amount, reason, operationId, }); await scope.audit.write({ actorId: scope.actorId, tenantId: scope.tenantId, action: "refund.issued", targetId: orderId, operationId, }); return result; } ``` ## Failure Modes [#failure-modes] | Failure | Fix | | ------------------------------------- | ------------------------------------------------ | | duplicate refund | idempotency key in service layer | | model performs write without approval | tool approval metadata or `onToolCall` hook | | approval blocks HTTP request too long | store approval and resume asynchronously | | audit only appears in traces | write product audit records | | admin data leaks | enforce actor and tenant permissions in services | ## Test Checklist [#test-checklist] * Test admin auth and tenant scoping. * Test approval approved, rejected, and timed out paths. * Test write tools call services with idempotency keys. * Test audit records for each successful side effect. * Test runner maps cancellation to a product-safe response. * Use Studio streaming runs to inspect approval behavior locally. ## Related Docs [#related-docs] * [Side Effect Tools](/docs/best-practices/tool-patterns/side-effect-tools) * [Production Guardrails](/docs/best-practices/common-patterns/production-guardrails) * [Approval Handlers](/docs/guides/human-in-the-loop/approval-handlers) # Coding Agent (/docs/best-practices/real-cases/coding-agent) A coding agent is a high-risk harness because it can inspect source code, run commands, and propose or apply changes. Treat it as an application over a workspace: your app owns the repository boundary, allowed paths, command policy, patch approval, git behavior, audit records, and trace metadata. This pattern does not require new SDK primitives. It composes agents, tools, MCP, approvals, tracing, and evals around a codebase workflow. ## Scenario [#scenario] A user asks, "Find why the checkout test fails and propose a fix." The agent should search files, read relevant code, optionally run allowed test commands, propose a patch, and wait for approval before any write. ## When to Use It [#when-to-use-it] Use this pattern when: * an agent assists with code review, debugging, test triage, migration, or docs changes * workspace access must be scoped to allowed repositories and paths * command execution must be allow-listed * writes require preview, approval, idempotency, and git diff inspection * behavior should be checked with coding-task evals ## Architecture Shape [#architecture-shape] | Layer | Responsibility | | ------------ | ----------------------------------------------------------------------------- | | runner | resolve user, repo, branch, task, allowed paths, trace metadata | | read tools | search files, read files, inspect git diff, list tests | | command tool | run only allow-listed commands with timeouts and sandbox policy | | patch tool | propose or apply patches behind approval | | MCP tools | optional filesystem or git server tools, filtered before registration | | audit | record command runs, patch proposals, approvals, and applied changes | | evals | regression tasks for search, diagnosis, patch proposal, and no-write behavior | ## Code Example [#code-example] ```ts import { AgentBuilder, createHook } from "@anvia/core"; import { model } from "./model"; import { createCodebaseTools } from "./tools"; export async function runCodingAgent(input: CodingAgentInput) { const user = await input.auth.requireUser(); const workspace = await input.workspaces.open({ repoId: input.repoId, userId: user.id, }); const approvalHook = createHook({ async onToolCall({ toolName, tool }) { if (!["apply_patch", "run_command"].includes(toolName)) { return tool.run(); } const approved = await input.approvals.waitForDecision({ actorId: user.id, repoId: input.repoId, toolName, reason: "Codebase mutation or command execution requires approval.", }); return approved ? tool.run() : tool.cancel("Operation was not approved."); }, }); const agent = new AgentBuilder("coding", model) .instructions(` Help with codebase tasks. Search and read files before proposing changes. Prefer minimal patches. Do not run commands unless a tool allows them. Do not claim a patch was applied unless the tool confirms it. `) .tools( createCodebaseTools({ workspace, allowedPaths: input.allowedPaths, allowedCommands: ["pnpm test", "pnpm lint", "pnpm typecheck"], audit: input.audit, }), ) .hook(approvalHook) .defaultMaxTurns(8) .build(); const response = await agent .prompt(input.task) .withTrace({ name: "coding-agent-task", userId: user.id, metadata: { repoId: input.repoId, branch: workspace.branch, taskId: input.taskId, }, }) .send(); return { output: response.output, trace: response.trace, }; } ``` ## Tool Boundaries [#tool-boundaries] Keep read tools separate from mutation tools. ```ts export function createCodebaseTools(scope: CodebaseToolScope) { return [ createSearchFilesTool(scope), createReadFileTool(scope), createGitDiffTool(scope), createRunCommandTool(scope), createApplyPatchTool(scope), ]; } ``` Read tools should enforce allowed paths. ```ts async execute({ path }) { scope.workspace.requireAllowedPath(path, scope.allowedPaths); return scope.workspace.readFile(path); } ``` Command tools should enforce exact allow lists, timeouts, and working directory policy. ```ts async execute({ command }) { if (!scope.allowedCommands.includes(command)) { return { status: "blocked" as const, reason: "command_not_allowed" }; } return scope.workspace.run(command, { timeoutMs: 60_000, audit: scope.audit, }); } ``` Patch tools should support preview-first behavior. ```ts async execute({ patch, mode }) { if (mode === "preview") { return scope.workspace.previewPatch(patch); } return scope.workspace.applyPatch({ patch, operationId: `patch:${scope.workspace.id}:${hashPatch(patch)}`, }); } ``` ## MCP Filesystem Tools [#mcp-filesystem-tools] If a filesystem MCP server is used, filter or wrap its tools before the coding agent sees them. Prefer local wrapper tools when you need path allow lists, audit records, command policy, or patch approval. ```ts const filesystem = await connectMcp( mcp.stdio({ name: "filesystem", command: "npx", args: ["-y", "@modelcontextprotocol/server-filesystem", workspace.root], }), ); const readOnlyTools = filesystem.tools.filter((tool) => ["read_file", "list_directory"].includes(tool.name), ); ``` ## Failure Modes [#failure-modes] | Failure | Fix | | ------------------------------ | ---------------------------------------------------------------- | | agent reads outside repo | enforce allowed paths in every file tool | | command is too broad | exact allow list and timeout in command tool | | patch applies twice | idempotency key from patch hash | | write happens without approval | hook or approval metadata on mutation tools | | traces leak source code | keep trace metadata to ids, paths, and summaries | | evals only check final prose | add task cases for tool choice, no-write mode, and patch preview | ## Test Checklist [#test-checklist] * Test file reads inside and outside allowed paths. * Test blocked commands and approved commands. * Test patch preview without mutation. * Test approved and rejected patch application. * Test git diff inspection after a patch. * Add eval cases for diagnosis, minimal patch proposal, and refusal to run disallowed commands. ## Related Docs [#related-docs] * [MCP Tool Inspection](/docs/best-practices/mcp-patterns/mcp-tool-inspection) * [Side Effect Tools](/docs/best-practices/tool-patterns/side-effect-tools) * [Eval Strategy](/docs/best-practices/quality-observability/eval-strategy) * [Tracing and Debugging](/docs/best-practices/quality-observability/tracing-and-debugging) # Research Agent (/docs/best-practices/real-cases/research-agent) This pattern is for research, analysis, due diligence, competitive intelligence, and internal knowledge workflows. The agent often has many read-only tools, uses retrieval heavily, and returns structured output for downstream processing. ## Scenario [#scenario] A research agent answers a market question by searching internal docs, querying external MCP tools, reading product notes, and extracting a structured brief. ## When to Use It [#when-to-use-it] Use this pattern when: * the workflow is read-heavy * the tool catalog is large * sources need to be inspected in traces * the final output should match a schema * deterministic post-processing is useful ## Architecture Shape [#architecture-shape] | Layer | Responsibility | | -------------------------- | -------------------------------------------------------- | | dynamic tool catalog | selects relevant read-only tools | | retrieval | provides internal knowledge context | | MCP tools | expose external research systems | | agent | gathers and synthesizes evidence | | extractor or output schema | produces typed downstream output | | pipeline | separates normalize, research, extract, and enrich steps | ## Code Example [#code-example] ```ts import { AgentBuilder, ExtractorBuilder, PipelineBuilder, ToolSet, createToolIndex } from "@anvia/core"; import { z } from "zod"; import { embeddingModel, model } from "./models"; import { internalDocsIndex } from "./retrieval"; import { createResearchTools } from "./tools"; const researchTools = ToolSet.fromTools(createResearchTools()); const researchToolIndex = await createToolIndex(embeddingModel, researchTools, { metadata: (tool) => ({ name: tool.name, domain: "research" }), }); const researchAgent = new AgentBuilder("research", model) .instructions(` Research the question using available tools and retrieved context. Prefer cited facts over guesses. Call tools only when they are relevant to the question. `) .dynamicContext(internalDocsIndex, { topK: 6, threshold: 0.72, }) .dynamicTools(researchToolIndex, { topK: 8, threshold: 0.68, }) .defaultMaxTurns(6) .build(); const briefSchema = z.object({ summary: z.string(), confidence: z.enum(["low", "medium", "high"]), keyFindings: z.array(z.string()), openQuestions: z.array(z.string()), }); const briefExtractor = new ExtractorBuilder(model, briefSchema).build(); export const researchWorkflow = new PipelineBuilder() .step((question) => question.trim()) .prompt(researchAgent) .extract(briefExtractor) .build(); ``` ## Add MCP Research Tools [#add-mcp-research-tools] If external research systems are MCP servers, inspect and filter their tools before adding them to the catalog. ```ts import { connectMcp, mcp } from "@anvia/core"; const docsServer = await connectMcp(mcp.http({ name: "external-docs", url: "https://mcp.example.com/mcp", })); const allowedMcpTools = docsServer.tools.filter((tool) => ["search_docs", "read_doc"].includes(tool.name), ); researchTools.addTools(allowedMcpTools); ``` Rebuild the dynamic tool index after changing the catalog. ## Failure Modes [#failure-modes] | Failure | Fix | | -------------------------------- | ---------------------------------------------------------- | | agent picks irrelevant tools | improve tool descriptions, lower `topK`, raise `threshold` | | important tools are not selected | improve embedding text or lower `threshold` | | final output is hard to consume | use an extractor or agent output schema | | citations are weak | include source ids in retrieved context and tool outputs | | tool catalog changes | rebuild the index before future runs | ## Test Checklist [#test-checklist] * Test dynamic tool search for representative research prompts. * Test retrieval filters and source formatting. * Test pipeline output schema with known questions. * Inspect traces for selected tools and retrieved context. * Add evals for answer quality and missing-evidence behavior. ## Related Docs [#related-docs] * [Dynamic Tool Catalogs](/docs/best-practices/tool-patterns/dynamic-tool-catalogs) * [RAG Agent Context](/docs/best-practices/knowledge-patterns/rag-agent-context) * [Eval Strategy](/docs/best-practices/quality-observability/eval-strategy) * [MCP Tool Inspection](/docs/best-practices/mcp-patterns/mcp-tool-inspection) * [Pipeline](/docs/best-practices/common-patterns/pipeline) # Support Agent (/docs/best-practices/real-cases/support-agent) This pattern is for customer support chat, help center assistants, and account-aware product support. The agent answers user questions, retrieves support knowledge, reads customer state through tools, persists conversation history, and emits trace metadata. ## Scenario [#scenario] A signed-in customer asks, "Where is my order A-100 and can I change the address?" The agent needs previous conversation history, support documentation, order tools, and tenant-safe account context. ## When to Use It [#when-to-use-it] Use this pattern when: * the user is authenticated * the agent needs conversation history * account-specific data must be fetched through tools * documentation or policy should come from retrieval * the response is user-facing and should be traceable ## Architecture Shape [#architecture-shape] | Layer | Responsibility | | ------------- | ----------------------------------------------------------------- | | route | parse request and return product response | | runner | resolve user, load history, create scoped tools, persist messages | | support agent | instructions, dynamic context, scoped account tools, turn limit | | tools | order, ticket, account, and subscription service calls | | retrieval | tenant-safe or public support docs | | storage | conversation history and trace correlation ids | ## Code Example [#code-example] ```ts import { AgentBuilder, Message } from "@anvia/core"; import { model } from "./model"; import { supportDocsIndex } from "./support-docs"; import { createSupportTools } from "./support-tools"; export async function runSupportTurn(input: SupportTurnInput) { const user = await input.auth.requireUser(); const history = await input.conversations.loadMessages(input.conversationId); const agent = new AgentBuilder("support", model) .instructions(` Answer support questions clearly. Use account tools for customer-specific data. Use retrieved support docs for policy and product behavior. Ask for missing details before guessing. `) .dynamicContext(supportDocsIndex, { topK: 4, threshold: 0.72, }) .tools( createSupportTools({ userId: user.id, tenantId: user.tenantId, orders: input.services.orders, tickets: input.services.tickets, subscriptions: input.services.subscriptions, }), ) .context(`Current customer plan: ${user.plan}`, "customer-plan") .defaultMaxTurns(4) .build(); const response = await agent .prompt([...history, Message.user(input.message)]) .withTrace({ name: "support-chat", userId: user.id, metadata: { tenantId: user.tenantId, conversationId: input.conversationId, channel: input.channel, }, }) .send(); await input.conversations.append(input.conversationId, response.messages); return { output: response.output, usage: response.usage, }; } ``` ## Tool Scope [#tool-scope] Support tools should be scoped to the current user and tenant. ```ts export function createSupportTools(scope: SupportToolScope) { return [ createLookupOrderTool(scope), createCreateTicketTool(scope), createSubscriptionStatusTool(scope), ]; } ``` Do not give the model raw database clients. Give it narrow tools that call permission-aware services. ## Failure Modes [#failure-modes] | Failure | Fix | | --------------------------------- | --------------------------------------------------------- | | agent answers from stale docs | rebuild or refresh the retrieval index | | account data leaks across tenants | enforce tenant filters in every tool and retrieval source | | route tests call live providers | test runner and tools with fakes first | | history grows too large | summarize, window, or use session memory policy | | model calls too many tools | lower `defaultMaxTurns` or split tools by workflow | ## Test Checklist [#test-checklist] * Test empty and malformed messages in the runner. * Test order lookup allowed, denied, and not found paths. * Test conversation history is appended with `response.messages`. * Test retrieval filters only return allowed documents. * Use Studio to inspect tool calls and retrieved context. * Add evals for common support questions and known policy answers. ## Related Docs [#related-docs] * [Request Runners](/docs/best-practices/common-patterns/request-runners) * [Context and Memory](/docs/best-practices/common-patterns/context-and-memory) * [Tools and Services](/docs/best-practices/common-patterns/tools-and-services) # Dynamic Tool Catalogs (/docs/best-practices/tool-patterns/dynamic-tool-catalogs) Use dynamic tools when an agent has a large catalog of possible actions but only a small subset should be sent to the model on each turn. This is common for internal platforms, operations agents, research agents, and backoffice assistants with many read-only service tools. Do not use dynamic tools to hide permission checks. The selected tool still needs to enforce permissions in code. ## Scenario [#scenario] A support operations agent can inspect orders, refunds, tickets, subscriptions, feature flags, warehouse records, fraud signals, and policy documents. Sending every tool definition on every turn is noisy and expensive. Dynamic tools let Anvia search a tool index with the current prompt and send only the matching definitions. ## When to Use It [#when-to-use-it] | Use dynamic tools | Prefer static tools | | ------------------------------------------------------- | ------------------------------------------------------------ | | many tools compete for the same agent | the agent has a small stable tool set | | tools are discoverable by name, description, and schema | every tool is needed in almost every run | | the prompt usually needs only a few tools | missing one tool would be more costly than sending all tools | | tool catalog can be indexed at startup or deploy time | tools are created entirely per request | ## Architecture Shape [#architecture-shape] | Layer | Responsibility | | -------------------------------- | ---------------------------------------------------------------- | | `ToolSet` | owns the full catalog and direct test calls | | embedding model | embeds provider-facing tool definitions | | `createToolIndex(...)` | builds a searchable dynamic tool index | | `AgentBuilder.dynamicTools(...)` | selects top matching tools per prompt | | tool code | still enforces permissions, tenant scope, and side-effect policy | ## Code Example [#code-example] ```ts import { AgentBuilder, ToolSet, createToolIndex } from "@anvia/core"; import { embeddingModel, model } from "./models"; import { createAdminTools, createBillingTools, createSupportTools } from "./tools"; const toolCatalog = ToolSet.fromTools([ ...createSupportTools(), ...createBillingTools(), ...createAdminTools(), ]); const toolIndex = await createToolIndex(embeddingModel, toolCatalog, { content: (tool, definition) => [ definition.name, definition.description, JSON.stringify(definition.parameters), ], metadata: (tool) => ({ name: tool.name, domain: tool.name.split("_")[0], }), }); export const operationsAgent = new AgentBuilder("operations", model) .instructions("Use the most relevant tool for the user's operational request.") .dynamicTools(toolIndex, { topK: 6, threshold: 0.7, }) .defaultMaxTurns(4) .build(); ``` Static tools can still be registered when they should always be available. ```ts const agent = new AgentBuilder("operations", model) .tool(createThinkTool()) .dynamicTools(toolIndex, { topK: 6, threshold: 0.7 }) .build(); ``` ## Validate Tool Selection [#validate-tool-selection] Test the catalog before testing model behavior. Search the index with representative prompts and assert that the expected tool ids appear. ```ts const matches = await toolIndex.searchIds({ query: "refund order A-100 because it was duplicated", topK: 6, threshold: 0.7, }); expect(matches.map((match) => match.id)).toContain("issue_refund"); ``` Then test the selected tool directly through the catalog. ```ts const result = await toolCatalog.call( "issue_refund", JSON.stringify({ orderId: "A-100", amount: 25, reason: "duplicate_charge", }), ); expect(JSON.parse(result).status).toBe("refunded"); ``` ## Failure Modes [#failure-modes] | Failure | Fix | | --------------------------------- | ---------------------------------------------------------------------------------- | | model cannot find the right tool | improve tool name, description, schema text, or `content` embedding text | | too many unrelated tools are sent | raise `threshold`, lower `topK`, or split catalogs by domain | | needed tools are missing | lower `threshold`, add domain words to descriptions, or keep critical tools static | | permission leak | enforce user and tenant checks inside every tool | | catalog changes at runtime | rebuild the index or use a new indexed catalog for future agents | ## Test Checklist [#test-checklist] * Search the index with common prompts and assert expected tool ids. * Call high-risk tools directly through `ToolSet.call(...)`. * Verify each tool still enforces permissions with fake users and tenants. * Use Studio traces to inspect which dynamic tools were sent during real runs. * Add evals for prompts where tool selection is part of the expected behavior. ## Related Docs [#related-docs] * [Tool Sets](/docs/guides/tools/tool-sets) * [Tools and Services](/docs/best-practices/common-patterns/tools-and-services) * [Research Agent](/docs/best-practices/real-cases/research-agent) # Side Effect Tools (/docs/best-practices/tool-patterns/side-effect-tools) Side-effect tools change product state: refunds, deletes, emails, status transitions, webhooks, exports, or external API calls. Treat them as product operations first and model-callable tools second. ## Scenario [#scenario] A backoffice agent can resolve tickets and issue refunds. The model can decide that a refund is appropriate, but product code must enforce permissions, approval policy, idempotency, transactions, and audit records. ## When to Use It [#when-to-use-it] Use this pattern for any tool that: * writes to your database * calls an external write API * sends a message or notification * changes a workflow state * triggers a job * exposes restricted data as a side effect ## Architecture Shape [#architecture-shape] | Layer | Responsibility | | ------------------------- | -------------------------------------------------------- | | tool schema | require bounded, explicit operation inputs | | approval metadata or hook | decide whether a human must approve | | product service | enforce permissions, transaction, idempotency, and audit | | runner | attach actor, tenant, trace, and operation context | | storage | persist operation status and audit evidence | ## Code Example [#code-example] ```ts import { createTool } from "@anvia/core"; import { z } from "zod"; export function createRefundTool(scope: RefundToolScope) { return createTool({ name: "issue_refund", description: "Issue an approved refund for a paid order.", input: z.object({ orderId: z.string().min(1), amount: z.number().positive(), reason: z.string().min(1), }), output: z.object({ status: z.enum(["refunded", "blocked"]), operationId: z.string().optional(), reason: z.string().optional(), }), approval: { when: ({ args }) => args.amount >= 100, reason: "Refunds of 100 or more require reviewer approval.", rejectMessage: "Refund was not approved.", }, async execute({ orderId, amount, reason }) { const operationId = `refund:${scope.tenantId}:${orderId}:${amount}`; return scope.billing.issueRefund({ actorId: scope.userId, tenantId: scope.tenantId, orderId, amount, reason, operationId, }); }, }); } ``` Approval metadata is useful for Studio and approval-capable runtimes. Use a hook when approval depends on request-local state or spans multiple tools. ```ts import { createHook } from "@anvia/core"; const approvalHook = createHook({ async onToolCall({ toolName, tool }) { if (!["issue_refund", "close_account"].includes(toolName)) { return tool.run(); } const approved = await approvals.waitForDecision({ actorId: scope.userId, tenantId: scope.tenantId, toolName, }); return approved ? tool.run() : tool.cancel("Operation was not approved."); }, }); ``` ## Idempotency and Audit [#idempotency-and-audit] The model should not invent idempotency keys. Generate them from product state or pass an operation id from the caller. ```ts await audit.write({ actorId: scope.userId, tenantId: scope.tenantId, action: "refund.issued", targetId: orderId, operationId, traceName: "backoffice-resolution", }); ``` Use traces to debug agent behavior. Use audit records for product accountability. ## Failure Modes [#failure-modes] | Failure | Fix | | --------------------------- | ----------------------------------------------------------- | | duplicate write after retry | idempotency key or durable operation record | | model bypasses policy | enforce permissions in service code | | approval waits forever | app-owned approval timeout or asynchronous workflow | | audit only exists in trace | write product audit records in service code | | side effect is hard to test | split tool adapter from service method and fake the service | ## Test Checklist [#test-checklist] * Test permission denied before the write executes. * Test approval required and rejected paths. * Test idempotency by calling the service twice with the same operation id. * Test audit records are written for successful side effects. * Test runner behavior when the tool throws a transient service error. ## Related Docs [#related-docs] * [Production Guardrails](/docs/best-practices/common-patterns/production-guardrails) * [Human in the Loop](/docs/guides/human-in-the-loop) * [Backoffice Agent](/docs/best-practices/real-cases/backoffice-agent) # Tool Validation and Contracts (/docs/best-practices/tool-patterns/tool-validation-and-contracts) Tool contracts are the strongest deterministic boundary in an agent harness. Use schemas to validate arguments and outputs, return typed product states for expected outcomes, and test tools directly before model runs. ## Scenario [#scenario] A model can choose when to call a tool, but the tool owns the contract. The model should not be able to pass arbitrary unvalidated data into your service layer, and downstream code should not need to parse vague natural-language tool results. ## When to Use It [#when-to-use-it] Use this pattern for every tool that reads product state, writes product state, or returns values used by downstream code. ## Architecture Shape [#architecture-shape] | Layer | Responsibility | | ----------------- | ---------------------------------------------------------- | | Zod input schema | validate model arguments before execution | | Zod output schema | validate tool result before it is serialized | | tool `execute` | call product services and return typed states | | runner | map expected states and thrown errors to product responses | | tests | call tools directly with valid and invalid arguments | ## Code Example [#code-example] ```ts import { createTool } from "@anvia/core"; import { z } from "zod"; const lookupOrderOutput = z.discriminatedUnion("status", [ z.object({ status: z.literal("found"), orderId: z.string(), fulfillmentStatus: z.enum(["processing", "shipped", "delivered"]), }), z.object({ status: z.literal("not_found"), orderId: z.string(), }), z.object({ status: z.literal("blocked"), reason: z.literal("access_denied"), }), ]); export function createLookupOrderTool(scope: OrderToolScope) { return createTool({ name: "lookup_order", description: "Look up one order owned by the current customer.", input: z.object({ orderId: z.string().min(1), }), output: lookupOrderOutput, async execute({ orderId }) { const allowed = await scope.orders.canRead({ userId: scope.userId, tenantId: scope.tenantId, orderId, }); if (!allowed) { return { status: "blocked" as const, reason: "access_denied" as const }; } const order = await scope.orders.find(orderId); if (!order) { return { status: "not_found" as const, orderId }; } return { status: "found" as const, orderId, fulfillmentStatus: order.fulfillmentStatus, }; }, }); } ``` ## Expected States vs Errors [#expected-states-vs-errors] | Situation | Return state | Throw | | --------------------------------------------------- | ----------------- | ----- | | record not found | yes | no | | user lacks access and the model can continue safely | yes | no | | malformed model arguments | schema handles it | no | | database unavailable | no | yes | | invariant violated | no | yes | | downstream service timeout | no | yes | Expected states are useful model input. Unexpected failures belong to the runner, logs, retries, or product error boundary. ## Direct Tool Tests [#direct-tool-tests] ```ts const tool = createLookupOrderTool({ userId: "user_123", tenantId: "tenant_123", orders: fakeOrders, }); const result = await tool.call({ orderId: "A-100" }); expect(result).toEqual({ status: "found", orderId: "A-100", fulfillmentStatus: "shipped", }); ``` Use `ToolSet.call(...)` when you want to exercise JSON parsing and serialized output. ```ts const tools = ToolSet.fromTools([tool]); await expect( tools.call("lookup_order", JSON.stringify({ orderId: "" })), ).rejects.toThrow(); ``` ## Runner Error Mapping [#runner-error-mapping] The runner should decide which failures become user-facing product errors. ```ts try { const response = await agent.prompt(message).send(); return { ok: true as const, output: response.output }; } catch (error) { if (isTemporaryStorageError(error)) { return { ok: false as const, error: "temporarily_unavailable" }; } throw error; } ``` ## Failure Modes [#failure-modes] | Failure | Fix | | ------------------------------------- | ----------------------------------------------------------------------- | | model keeps passing invalid arguments | tighten description, schema descriptions, or ask for missing data first | | downstream code parses prose | return structured tool output or agent output schema | | permission errors leak details | return a compact `blocked` state or generic product error | | tests only cover provider runs | add direct tool and runner tests with fakes | ## Test Checklist [#test-checklist] * Test valid inputs, invalid inputs, and missing required fields. * Test permission allowed and denied paths. * Test expected states such as `not_found` and `blocked`. * Test unexpected service failures at the runner boundary. * Inspect tool result text in traces for readability. ## Related Docs [#related-docs] * [Creating Tools](/docs/guides/tools/creating-tools) * [Tool Schemas](/docs/guides/tools/tool-schemas) * [Testing and Observability](/docs/best-practices/common-patterns/testing-and-observability) # @anvia/anthropic (/docs/changelog/anthropic) # `@anvia/anthropic` [#anviaanthropic] Anthropic provider adapter for Anvia. Source: [`packages/providers/anthropic/CHANGELOG.md`](https://github.com/anvia-hq/anvia/blob/main/packages/providers/anthropic/CHANGELOG.md) ## 0.1.10 [#0110] ### Patch Changes [#patch-changes] * 49e43a3: Update upstream runtime dependencies for Anthropic, Gemini, OpenAI, and Studio. ## 0.1.9 [#019] ### Patch Changes [#patch-changes-1] * 896ae21: Update upstream provider and runtime dependencies. ## 0.1.8 [#018] ### Patch Changes [#patch-changes-2] * 1ad360d: Fix Anthropic-compatible streaming tool inputs and update provider dependencies. # @anvia/chroma (/docs/changelog/chroma) # `@anvia/chroma` [#anviachroma] ChromaDB vector store adapter for Anvia. Source: [`packages/vector-stores/chroma/CHANGELOG.md`](https://github.com/anvia-hq/anvia/blob/main/packages/vector-stores/chroma/CHANGELOG.md) ## 0.1.2 [#012] ### Patch Changes [#patch-changes] * Updated dependencies \[a0a5def] * @anvia/core\@0.2.4 # @anvia/core (/docs/changelog/core) # `@anvia/core` [#anviacore] Core runtime primitives for context-aware Anvia agents. Source: [`packages/core/CHANGELOG.md`](https://github.com/anvia-hq/anvia/blob/main/packages/core/CHANGELOG.md) ## 0.2.4 [#024] ### Patch Changes [#patch-changes] * a0a5def: Preserve accumulated streamed tool arguments when a provider final response contains an empty tool input. # @anvia/fastembed (/docs/changelog/fastembed) # `@anvia/fastembed` [#anviafastembed] FastEmbed embedding model adapter for Anvia. Source: [`packages/embeddings/fastembed/CHANGELOG.md`](https://github.com/anvia-hq/anvia/blob/main/packages/embeddings/fastembed/CHANGELOG.md) ## 0.1.2 [#012] ### Patch Changes [#patch-changes] * Updated dependencies \[a0a5def] * @anvia/core\@0.2.4 # @anvia/gemini (/docs/changelog/gemini) # `@anvia/gemini` [#anviagemini] Gemini provider adapter for Anvia. Source: [`packages/providers/gemini/CHANGELOG.md`](https://github.com/anvia-hq/anvia/blob/main/packages/providers/gemini/CHANGELOG.md) ## 0.1.9 [#019] ### Patch Changes [#patch-changes] * 49e43a3: Update upstream runtime dependencies for Anthropic, Gemini, OpenAI, and Studio. ## 0.1.8 [#018] ### Patch Changes [#patch-changes-1] * 896ae21: Update upstream provider and runtime dependencies. ## 0.1.7 [#017] ### Patch Changes [#patch-changes-2] * 1ad360d: Fix Anthropic-compatible streaming tool inputs and update provider dependencies. # Package Changelog (/docs/changelog) Anvia package release notes are generated from the package changelog files maintained by Changesets. Developers should keep writing release notes with `pnpm changeset`. The docs pages in this section are generated from `packages/**/CHANGELOG.md`. ## Core [#core] | Package | Current version | Release notes | | ------------- | --------------- | -------------------------------------- | | `@anvia/core` | `0.2.4` | [View changelog](/docs/changelog/core) | ## Providers [#providers] | Package | Current version | Release notes | | ------------------ | --------------- | ------------------------------------------- | | `@anvia/anthropic` | `0.1.10` | [View changelog](/docs/changelog/anthropic) | | `@anvia/gemini` | `0.1.9` | [View changelog](/docs/changelog/gemini) | | `@anvia/mistral` | `0.1.4` | [View changelog](/docs/changelog/mistral) | | `@anvia/openai` | `0.1.11` | [View changelog](/docs/changelog/openai) | ## Embeddings [#embeddings] | Package | Current version | Release notes | | --------------------- | --------------- | ---------------------------------------------- | | `@anvia/fastembed` | `0.1.2` | [View changelog](/docs/changelog/fastembed) | | `@anvia/transformers` | `0.1.2` | [View changelog](/docs/changelog/transformers) | ## Vector Stores [#vector-stores] | Package | Current version | Release notes | | ----------------- | --------------- | ------------------------------------------ | | `@anvia/chroma` | `0.1.2` | [View changelog](/docs/changelog/chroma) | | `@anvia/pgvector` | `0.1.4` | [View changelog](/docs/changelog/pgvector) | | `@anvia/qdrant` | `0.1.2` | [View changelog](/docs/changelog/qdrant) | ## Observability [#observability] | Package | Current version | Release notes | | ----------------- | --------------- | ------------------------------------------ | | `@anvia/langfuse` | `0.1.5` | [View changelog](/docs/changelog/langfuse) | | `@anvia/otel` | `0.1.3` | [View changelog](/docs/changelog/otel) | ## Tools [#tools] | Package | Current version | Release notes | | --------------- | --------------- | ---------------------------------------- | | `@anvia/studio` | `0.2.9` | [View changelog](/docs/changelog/studio) | # @anvia/langfuse (/docs/changelog/langfuse) # `@anvia/langfuse` [#anvialangfuse] Langfuse tracing adapter for Anvia. Source: [`packages/observability/langfuse/CHANGELOG.md`](https://github.com/anvia-hq/anvia/blob/main/packages/observability/langfuse/CHANGELOG.md) ## No release notes yet [#no-release-notes-yet] `@anvia/langfuse` is public, but `packages/observability/langfuse/CHANGELOG.md` does not exist yet. Release notes will appear here after the next Changesets version bump creates the package changelog. # @anvia/mistral (/docs/changelog/mistral) # `@anvia/mistral` [#anviamistral] Mistral provider adapter for Anvia. Source: [`packages/providers/mistral/CHANGELOG.md`](https://github.com/anvia-hq/anvia/blob/main/packages/providers/mistral/CHANGELOG.md) ## 0.1.4 [#014] ### Patch Changes [#patch-changes] * Updated dependencies \[a0a5def] * @anvia/core\@0.2.4 # @anvia/openai (/docs/changelog/openai) # `@anvia/openai` [#anviaopenai] OpenAI provider adapter for Anvia. Source: [`packages/providers/openai/CHANGELOG.md`](https://github.com/anvia-hq/anvia/blob/main/packages/providers/openai/CHANGELOG.md) ## 0.1.11 [#0111] ### Patch Changes [#patch-changes] * 49e43a3: Update upstream runtime dependencies for Anthropic, Gemini, OpenAI, and Studio. ## 0.1.10 [#0110] ### Patch Changes [#patch-changes-1] * Updated dependencies \[a0a5def] * @anvia/core\@0.2.4 ## 0.1.9 [#019] ### Patch Changes [#patch-changes-2] * 1f7d3aa: Republish packages with registry-safe dependency metadata. ## 0.1.8 [#018] ### Patch Changes [#patch-changes-3] * 1ad360d: Fix Anthropic-compatible streaming tool inputs and update provider dependencies. # @anvia/otel (/docs/changelog/otel) # `@anvia/otel` [#anviaotel] OpenTelemetry tracing adapter for Anvia. Source: [`packages/observability/otel/CHANGELOG.md`](https://github.com/anvia-hq/anvia/blob/main/packages/observability/otel/CHANGELOG.md) ## 0.1.3 [#013] ### Patch Changes [#patch-changes] * Updated dependencies \[a0a5def] * @anvia/core\@0.2.4 # @anvia/pgvector (/docs/changelog/pgvector) # `@anvia/pgvector` [#anviapgvector] Postgres pgvector store adapter for Anvia. Source: [`packages/vector-stores/pgvector/CHANGELOG.md`](https://github.com/anvia-hq/anvia/blob/main/packages/vector-stores/pgvector/CHANGELOG.md) ## 0.1.4 [#014] ### Patch Changes [#patch-changes] * Updated dependencies \[a0a5def] * @anvia/core\@0.2.4 ## 0.1.3 [#013] ### Patch Changes [#patch-changes-1] * 1f7d3aa: Republish packages with registry-safe dependency metadata. ## 0.1.2 [#012] ### Patch Changes [#patch-changes-2] * 1ad360d: Fix Anthropic-compatible streaming tool inputs and update provider dependencies. # @anvia/qdrant (/docs/changelog/qdrant) # `@anvia/qdrant` [#anviaqdrant] Qdrant vector store adapter for Anvia. Source: [`packages/vector-stores/qdrant/CHANGELOG.md`](https://github.com/anvia-hq/anvia/blob/main/packages/vector-stores/qdrant/CHANGELOG.md) ## 0.1.2 [#012] ### Patch Changes [#patch-changes] * Updated dependencies \[a0a5def] * @anvia/core\@0.2.4 # @anvia/studio (/docs/changelog/studio) # `@anvia/studio` [#anviastudio] Studio UI and HTTP runtime for Anvia agents. Source: [`packages/tools/studio/CHANGELOG.md`](https://github.com/anvia-hq/anvia/blob/main/packages/tools/studio/CHANGELOG.md) ## 0.2.9 [#029] ### Patch Changes [#patch-changes] * 49e43a3: Update upstream runtime dependencies for Anthropic, Gemini, OpenAI, and Studio. ## 0.2.8 [#028] ### Patch Changes [#patch-changes-1] * 896ae21: Update upstream provider and runtime dependencies. ## 0.2.7 [#027] ### Patch Changes [#patch-changes-2] * a0a5def: Lazy-load the default SQLite store so importing Studio does not require `node:sqlite` in Bun-compatible runtimes. * Updated dependencies \[a0a5def] * @anvia/core\@0.2.4 ## 0.2.6 [#026] ### Patch Changes [#patch-changes-3] * 1f7d3aa: Republish packages with registry-safe dependency metadata. ## 0.2.5 [#025] ### Patch Changes [#patch-changes-4] * 1ad360d: Fix Anthropic-compatible streaming tool inputs and update provider dependencies. ## 0.2.4 [#024] ### Patch Changes [#patch-changes-5] * 1e5b78d: Polish the Studio UI with updated sidebar, page surfaces, tracing views, playground logs, transcript auto-scroll, and full-width markdown tables. # @anvia/transformers (/docs/changelog/transformers) # `@anvia/transformers` [#anviatransformers] Transformers.js embedding model adapter for Anvia. Source: [`packages/embeddings/transformers/CHANGELOG.md`](https://github.com/anvia-hq/anvia/blob/main/packages/embeddings/transformers/CHANGELOG.md) ## 0.1.2 [#012] ### Patch Changes [#patch-changes] * Updated dependencies \[a0a5def] * @anvia/core\@0.2.4 # 01 Prep (/docs/frameworks/express/01-prep) Use this path when Anvia runs inside an existing Express server or a new Node API. ## 1. Create An Express Project [#1-create-an-express-project] ```sh mkdir anvia-express cd anvia-express pnpm init pnpm add express zod pnpm add -D tsx typescript @types/node @types/express ``` ## 2. Install Anvia [#2-install-anvia] ```sh pnpm add @anvia/core @anvia/openai ``` Install other provider packages when needed: ```sh pnpm add @anvia/anthropic @anvia/gemini @anvia/mistral ``` ## 3. Add Environment Variables [#3-add-environment-variables] ```txt OPENAI_API_KEY=sk_... ``` Read the value in server code: ```ts const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) { throw new Error("OPENAI_API_KEY is required"); } ``` ## 4. Choose File Boundaries [#4-choose-file-boundaries] | File | Purpose | | ------------------------- | ------------------------------------------------------- | | `src/ai/support-agent.ts` | Provider client, model, tools, and reusable agent | | `src/routes/support.ts` | Express router for prompt and stream endpoints | | `src/middleware/auth.ts` | Request auth and `req.user` enrichment | | `src/app.ts` | Express app, JSON parser, routers, and error middleware | ## Next [#next] Build the reusable agent in [Setup Anvia](/docs/frameworks/express/02-setup-anvia). Read [Runtime Boundaries](/docs/guides/sdk-fundamentals/runtime-boundaries) for where application code should own auth, storage, and side effects. # 02 Setup Anvia (/docs/frameworks/express/02-setup-anvia) Create provider clients and shared tools outside route handlers. Express routes should call an already configured agent. ## 1. Create `src/ai/support-agent.ts` [#1-create-srcaisupport-agentts] ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) { throw new Error("OPENAI_API_KEY is required"); } const client = new OpenAIClient({ apiKey }); export const model = client.completionModel("gpt-5.5"); const lookupPolicy = createTool({ name: "lookup_policy", description: "Look up a support policy by key.", input: z.object({ key: z.enum(["password_reset", "priority_support"]), }), output: z.object({ text: z.string(), }), async execute({ key }) { const policies = { password_reset: "Password reset links expire after 30 minutes.", priority_support: "Enterprise customers receive priority support.", }; return { text: policies[key] }; }, }); export const supportAgent = new AgentBuilder("support", model) .name("Support Agent") .instructions("Answer clearly. Use tools when policy detail is needed.") .tool(lookupPolicy) .defaultMaxTurns(3) .build(); ``` ## 2. Keep Route State Out Of The Agent [#2-keep-route-state-out-of-the-agent] The shared agent can hold provider configuration and static tools. Request-local auth, database records, and retrieval results should be attached inside routes or route-specific tool factories. ## 3. Swap Providers Later [#3-swap-providers-later] ```ts import { MistralClient } from "@anvia/mistral"; const client = new MistralClient({ apiKey: process.env.MISTRAL_API_KEY }); const model = client.completionModel("mistral-large-latest"); ``` ## Next [#next] Expose the agent through an Express router in [Route Handler](/docs/frameworks/express/03-route-handler). Related guides: [Creating Agents](/docs/guides/agents/creating-agents) and [Tools](/docs/guides/tools/creating-tools). # 03 Route Handler (/docs/frameworks/express/03-route-handler) Use Express middleware for JSON parsing and route handlers for application-owned validation and error shapes. ## 1. Create `src/routes/support.ts` [#1-create-srcroutessupportts] ```ts import { Router } from "express"; import { z } from "zod"; import { supportAgent } from "../ai/support-agent"; const SupportRequest = z.object({ message: z.string().trim().min(1, "message is required"), }); export const supportRouter = Router(); supportRouter.post("/support", async (req, res, next) => { try { const parsed = SupportRequest.safeParse(req.body); if (!parsed.success) { return res.status(400).json({ error: { code: "bad_request", message: parsed.error.issues[0]?.message }, }); } const response = await supportAgent.prompt(parsed.data.message).send(); return res.json({ output: response.output, usage: response.usage, messages: response.messages, }); } catch (error) { return next(error); } }); ``` ## 2. Mount The Router [#2-mount-the-router] ```ts import express from "express"; import { supportRouter } from "./routes/support"; export const app = express(); app.use(express.json({ limit: "1mb" })); app.use("/api", supportRouter); app.use((error: unknown, _req, res, _next) => { console.error(error); res.status(500).json({ error: { code: "agent_failed", message: "The agent run failed." }, }); }); ``` ## 3. Call The Route [#3-call-the-route] ```ts const response = await fetch("http://localhost:3000/api/support", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ message: "How long does a reset link last?" }), }); const data = await response.json(); console.log(data.output); ``` ## Next [#next] Return live run events in [Streaming](/docs/frameworks/express/04-streaming). For response fields, read [Prompt Responses](/docs/guides/sdk-fundamentals/prompt-responses). # 04 Streaming (/docs/frameworks/express/04-streaming) Express uses Node response objects. Read from Anvia's Web `ReadableStream` and write NDJSON chunks to `res`. ## 1. Add `/api/support/stream` [#1-add-apisupportstream] ```ts import { Router } from "express"; import { z } from "zod"; import { supportAgent } from "../ai/support-agent"; const SupportStreamRequest = z.object({ message: z.string().trim().min(1, "message is required"), }); export const supportRouter = Router(); supportRouter.post("/support/stream", async (req, res, next) => { try { const parsed = SupportStreamRequest.safeParse(req.body); if (!parsed.success) { return res.status(400).json({ error: { code: "bad_request", message: parsed.error.issues[0]?.message }, }); } res.setHeader("Content-Type", "application/x-ndjson"); res.setHeader("Cache-Control", "no-cache"); const reader = supportAgent.prompt(parsed.data.message).readableStream().getReader(); const decoder = new TextDecoder(); while (true) { const next = await reader.read(); if (next.done) break; res.write(decoder.decode(next.value)); } res.end(); } catch (error) { next(error); } }); ``` ## 2. Consume The Stream [#2-consume-the-stream] ```ts const response = await fetch("http://localhost:3000/api/support/stream", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ message: "Draft a support reply." }), }); const reader = response.body?.getReader(); const decoder = new TextDecoder(); while (reader) { const next = await reader.read(); if (next.done) break; for (const line of decoder.decode(next.value).split("\n")) { if (line.trim()) console.log(JSON.parse(line)); } } ``` ## 3. Operational Notes [#3-operational-notes] Disable proxy buffering for this route and keep request timeouts long enough for model calls. Clients should handle both `final` and `error` stream events. ## Next [#next] Add auth, request-local tools, and retrieval in [Tools and Context](/docs/frameworks/express/05-tools-and-context). Related guides: [Readable Streams](/docs/guides/streaming/readable-streams) and [Streaming Events](/docs/guides/streaming/streaming-events). # 05 Tools and Context (/docs/frameworks/express/05-tools-and-context) Express middleware should authenticate the request. Anvia tools should receive the smallest request-local context they need. ## 1. Add Auth Middleware [#1-add-auth-middleware] ```ts import type { NextFunction, Request, Response } from "express"; declare global { namespace Express { interface Request { user?: { id: string }; } } } export async function requireUser(req: Request, res: Response, next: NextFunction) { const user = await auth.userFromRequest(req); if (!user) { return res.status(401).json({ error: { code: "unauthorized" } }); } req.user = { id: user.id }; return next(); } ``` ## 2. Build Request-Local Tools [#2-build-request-local-tools] ```ts import { createTool } from "@anvia/core"; import { z } from "zod"; export function createAccountTool(input: { userId: string }) { return createTool({ name: "get_account_status", description: "Read the authenticated user's account status.", input: z.object({}), output: z.object({ plan: z.string(), openTickets: z.number() }), async execute() { return db.account.findStatus({ userId: input.userId }); }, }); } ``` ## 3. Attach Context In The Route [#3-attach-context-in-the-route] ```ts supportRouter.post("/support", requireUser, async (req, res, next) => { try { const { message } = SupportRequest.parse(req.body); const userId = req.user?.id; if (!userId) { return res.status(401).json({ error: { code: "unauthorized" } }); } const response = await supportAgent .prompt(message) .tool(createAccountTool({ userId })) .context({ userId }) .send(); return res.json({ output: response.output }); } catch (error) { return next(error); } }); ``` ## 4. Add Retrieval Context [#4-add-retrieval-context] ```ts const documents = await knowledge.search({ query: message, filter: { userId: req.user.id }, limit: 5, }); const response = await supportAgent.prompt(message).documents(documents).send(); ``` ## Next [#next] Persist history in [Persistence](/docs/frameworks/express/06-persistence). Related guides: [Runtime Context](/docs/guides/agents/runtime-context), [Tool Handlers](/docs/guides/tools/tool-handlers), and [RAG Context](/docs/guides/retrieval/rag-context). # 06 Persistence (/docs/frameworks/express/06-persistence) Express does not prescribe storage. Load existing messages before the run and store `response.messages` after success. ## 1. Load The Session [#1-load-the-session] ```ts const session = await db.chatSession.findUnique({ where: { id: req.params.sessionId, userId: req.user.id }, include: { messages: { orderBy: { createdAt: "asc" } } }, }); if (!session) { return res.status(404).json({ error: { code: "not_found" } }); } ``` ## 2. Send With History [#2-send-with-history] ```ts const response = await supportAgent .prompt(message) .messages(session.messages.map((item) => item.message)) .send(); ``` ## 3. Store New Messages [#3-store-new-messages] ```ts await db.chatMessage.createMany({ data: response.messages.map((message) => ({ sessionId: session.id, message, })), }); ``` Keep persistence in your transaction boundary when the app needs message history and related business records to commit together. ## 4. Use Memory Deliberately [#4-use-memory-deliberately] Use chat history for turn-by-turn continuity. Use Anvia memory when the model should recall durable facts across future sessions. ## Next [#next] Prepare runtime constraints in [Deploy](/docs/frameworks/express/07-deploy). Related guides: [Memory](/docs/guides/memory), [Memory and Sessions](/docs/guides/sdk-fundamentals/memory-and-sessions), and [Agent History](/docs/guides/agents/agent-history). # 07 Deploy (/docs/frameworks/express/07-deploy) Express runs in Node. Size timeouts, body limits, and proxy buffering for model calls and streams. ## 1. Start The Server [#1-start-the-server] ```ts import { app } from "./app"; const port = Number(process.env.PORT ?? 3000); app.listen(port, () => { console.log(`listening on :${port}`); }); ``` ## 2. Configure Environment Variables [#2-configure-environment-variables] ```txt OPENAI_API_KEY=sk_... DATABASE_URL=... ANVIA_STUDIO_TOKEN=... ``` Keep provider keys server-side and inject them through your deployment platform. ## 3. Streaming Checks [#3-streaming-checks] Disable buffering in reverse proxies for `/api/support/stream`. Keep Node and proxy timeouts longer than expected agent runs. ## 4. Production Checklist [#4-production-checklist] | Check | Why | | -------------------------- | --------------------------------------- | | `express.json` limit set | Avoid unbounded body parsing | | Error middleware installed | Avoid leaking provider stack traces | | Proxy buffering disabled | NDJSON streams must flush incrementally | | Observability enabled | Tool and provider failures need traces | ## Next [#next] Debug common failures in [Troubleshooting](/docs/frameworks/express/08-troubleshooting). Add telemetry with [Observability](/docs/guides/observability/tracing). # 08 Troubleshooting (/docs/frameworks/express/08-troubleshooting) Most Express failures come from missing middleware, untyped bodies, stream buffering, or errors bypassing `next(error)`. ## `req.body` Is Undefined [#reqbody-is-undefined] Mount JSON parsing before the router: ```ts app.use(express.json({ limit: "1mb" })); app.use("/api", supportRouter); ``` ## Validation Returns 500 [#validation-returns-500] Use `safeParse` for request validation. Reserve error middleware for unexpected failures. ```ts const parsed = SupportRequest.safeParse(req.body); if (!parsed.success) { return res.status(400).json({ error: { code: "bad_request" } }); } ``` ## Stream Does Not Flush [#stream-does-not-flush] Set `application/x-ndjson`, call `res.write(...)`, and check proxy buffering. ```ts res.setHeader("Content-Type", "application/x-ndjson"); res.write(chunk); ``` ## Provider Failures Leak Details [#provider-failures-leak-details] Route handlers should call `next(error)`, and centralized error middleware should return a stable error shape. ## Request Times Out [#request-times-out] Raise Node, proxy, and platform timeouts for agent endpoints. For long approvals, use human-in-the-loop storage instead of holding an HTTP request open indefinitely. ## Next [#next] Add reviewer workflows in [Human in the Loop](/docs/frameworks/express/09-human-in-the-loop). Related guides: [Tool Errors](/docs/guides/tools/tool-errors), [Readable Streams](/docs/guides/streaming/readable-streams), and [Tracing](/docs/guides/observability/tracing). # 09 Human in the Loop (/docs/frameworks/express/09-human-in-the-loop) Express can expose agent routes and reviewer routes from the same server. Anvia provides hooks; your app provides approval storage and reviewer workflows. ## 1. Use Studio During Development [#1-use-studio-during-development] ```ts import { Studio } from "@anvia/studio"; import { supportAgent } from "../ai/support-agent"; new Studio([supportAgent]).start({ port: 4021 }); ``` Studio helps inspect pending approvals locally. Production reviewer permissions and notifications belong to your Express app. ## 2. Create A Hook [#2-create-a-hook] ```ts import { createHook } from "@anvia/core"; import { approvalRuntime } from "../approvals/runtime"; export function createApprovalHook(input: { userId: string; approvalRunId: string }) { return createHook({ async onToolCall({ toolName, args, tool }) { if (toolName !== "refund_order") { return tool.run(); } const approved = await approvalRuntime.waitForDecision({ userId: input.userId, approvalRunId: input.approvalRunId, toolName, args, }); return approved ? tool.run() : tool.skip("Refund was not approved."); }, }); } ``` `approvalRuntime` is not imported from Anvia. It is your module for database records, notifications, reviewer UI, and waiter resolution. ## 3. Create The Approval Runtime [#3-create-the-approval-runtime] ```ts type ApprovalRequest = { userId: string; approvalRunId: string; toolName: string; args: string; }; type ApprovalDecision = { approved: boolean; reason?: string; }; export function createApprovalRuntime() { const waiters = new Map void>(); return { async waitForDecision(request: ApprovalRequest): Promise { const approval = await db.approval.create({ data: { ...request, status: "pending" }, }); await notifyReviewers({ approvalId: approval.id }); const decision = await new Promise((resolve) => { waiters.set(approval.id, resolve); }); waiters.delete(approval.id); return decision.approved; }, async listPendingForReviewer(reviewerId: string) { return db.approval.findMany({ where: { reviewerId, status: "pending" }, orderBy: { createdAt: "asc" }, }); }, async decide(input: { approvalId: string; approved: boolean; reason?: string }) { await db.approval.update({ where: { id: input.approvalId }, data: { status: input.approved ? "approved" : "rejected", decisionReason: input.reason, resolvedAt: new Date(), }, }); waiters.get(input.approvalId)?.({ approved: input.approved, reason: input.reason, }); }, }; } export const approvalRuntime = createApprovalRuntime(); ``` Use durable storage plus queue, pub/sub, websocket, or polling workers for production. The `Map` only works inside one process. ## 4. Add Reviewer Routes [#4-add-reviewer-routes] ```ts const DecisionRequest = z.object({ approved: z.boolean(), reason: z.string().optional(), }); supportRouter.get("/approvals", requireUser, async (req, res, next) => { try { res.json(await approvalRuntime.listPendingForReviewer(req.user.id)); } catch (error) { next(error); } }); supportRouter.post("/approvals/:id/decision", requireUser, async (req, res, next) => { try { const decision = DecisionRequest.parse(req.body); await approvalRuntime.decide({ approvalId: req.params.id, ...decision }); res.json({ ok: true }); } catch (error) { next(error); } }); ``` ## Next [#next] Add route tests in [Setup Tests](/docs/frameworks/express/10-setup-tests). Core concepts: [Human in the Loop](/docs/guides/human-in-the-loop), [Approval by Hooks](/docs/guides/human-in-the-loop/tool-approval), and [Approval Runtimes](/docs/guides/human-in-the-loop/approval-handlers). # 10 Setup Tests (/docs/frameworks/express/10-setup-tests) Use route tests with mocked agents. Keep provider calls behind explicit integration tests. ## 1. Install Test Tools [#1-install-test-tools] ```sh pnpm add -D vitest supertest @types/supertest ``` ## 2. Test The JSON Route [#2-test-the-json-route] ```ts import request from "supertest"; import { describe, expect, it, vi } from "vitest"; import { app } from "../src/app"; vi.mock("../src/ai/support-agent", () => ({ supportAgent: { prompt: () => ({ send: async () => ({ output: "Reset links expire after 30 minutes.", usage: { totalTokens: 12 }, messages: [], }), }), }, })); describe("POST /api/support", () => { it("returns the agent output", async () => { const response = await request(app) .post("/api/support") .send({ message: "How long does a reset link last?" }) .expect(200); expect(response.body.output).toBe("Reset links expire after 30 minutes."); }); }); ``` ## 3. Test The Stream Route [#3-test-the-stream-route] ```ts const response = await request(app) .post("/api/support/stream") .send({ message: "Hello" }) .expect(200); expect(response.headers["content-type"]).toContain("application/x-ndjson"); ``` Mock `readableStream()` with a small `ReadableStream` that emits one `final` event. ## 4. Test Studio Without A Port [#4-test-studio-without-a-port] ```ts import { Studio } from "@anvia/studio"; import { supportAgent } from "../src/ai/support-agent"; const studio = new Studio([supportAgent]); const response = await studio.fetch( new Request("http://studio.test/agents/support/runs", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ message: "Hello" }), }), ); expect(response.status).toBe(200); ``` ## Next [#next] Related guides: [Testing](/docs/guides/testing), [Tools and Pipelines](/docs/guides/testing/tools-and-pipelines), and [Studio and Providers](/docs/guides/testing/studio-and-providers). # 01 Prep (/docs/frameworks/fastify/01-prep) Use this path when Anvia runs inside a Fastify API or plugin-based Node service. ## 1. Create A Fastify Project [#1-create-a-fastify-project] ```sh mkdir anvia-fastify cd anvia-fastify pnpm init pnpm add fastify fastify-plugin zod pnpm add -D tsx typescript @types/node ``` ## 2. Install Anvia [#2-install-anvia] ```sh pnpm add @anvia/core @anvia/openai ``` Install other provider packages when needed: ```sh pnpm add @anvia/anthropic @anvia/gemini @anvia/mistral ``` ## 3. Add Environment Variables [#3-add-environment-variables] ```txt OPENAI_API_KEY=sk_... ``` Read the value in server code: ```ts const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) { throw new Error("OPENAI_API_KEY is required"); } ``` ## 4. Choose File Boundaries [#4-choose-file-boundaries] | File | Purpose | | ------------------------- | ------------------------------------------------- | | `src/ai/support-agent.ts` | Provider client, model, tools, and reusable agent | | `src/routes/support.ts` | Fastify plugin with prompt and stream routes | | `src/plugins/auth.ts` | Auth decoration and hooks | | `src/app.ts` | Fastify instance and plugin registration | ## Next [#next] Build the reusable agent in [Setup Anvia](/docs/frameworks/fastify/02-setup-anvia). Read [Runtime Boundaries](/docs/guides/sdk-fundamentals/runtime-boundaries) for the SDK and application boundaries. # 02 Setup Anvia (/docs/frameworks/fastify/02-setup-anvia) Create provider clients and shared agents outside Fastify route handlers. Register request-local tools inside routes. ## 1. Create `src/ai/support-agent.ts` [#1-create-srcaisupport-agentts] ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) { throw new Error("OPENAI_API_KEY is required"); } const client = new OpenAIClient({ apiKey }); export const model = client.completionModel("gpt-5.5"); const lookupPolicy = createTool({ name: "lookup_policy", description: "Look up a support policy by key.", input: z.object({ key: z.enum(["password_reset", "priority_support"]), }), output: z.object({ text: z.string(), }), async execute({ key }) { const policies = { password_reset: "Password reset links expire after 30 minutes.", priority_support: "Enterprise customers receive priority support.", }; return { text: policies[key] }; }, }); export const supportAgent = new AgentBuilder("support", model) .name("Support Agent") .instructions("Answer clearly. Use tools when policy detail is needed.") .tool(lookupPolicy) .defaultMaxTurns(3) .build(); ``` ## 2. Keep The Fastify Instance Separate [#2-keep-the-fastify-instance-separate] The agent module should not import `FastifyInstance`. This keeps it reusable from routes, jobs, Studio, and tests. ## 3. Swap Providers Later [#3-swap-providers-later] ```ts import { GeminiClient } from "@anvia/gemini"; const client = new GeminiClient({ apiKey: process.env.GEMINI_API_KEY }); const model = client.completionModel("gemini-2.5-pro"); ``` ## Next [#next] Expose the agent through a Fastify plugin in [Route Handler](/docs/frameworks/fastify/03-route-handler). Related guides: [Creating Agents](/docs/guides/agents/creating-agents) and [Tools](/docs/guides/tools/creating-tools). # 03 Route Handler (/docs/frameworks/fastify/03-route-handler) Fastify routes can live in plugins. Validate unknown bodies with `zod` before calling the agent. ## 1. Create `src/routes/support.ts` [#1-create-srcroutessupportts] ```ts import type { FastifyInstance } from "fastify"; import { z } from "zod"; import { supportAgent } from "../ai/support-agent"; const SupportRequest = z.object({ message: z.string().trim().min(1, "message is required"), }); export async function supportRoutes(app: FastifyInstance) { app.post("/support", async (request, reply) => { const parsed = SupportRequest.safeParse(request.body); if (!parsed.success) { return reply.status(400).send({ error: { code: "bad_request", message: parsed.error.issues[0]?.message }, }); } const response = await supportAgent.prompt(parsed.data.message).send(); return reply.send({ output: response.output, usage: response.usage, messages: response.messages, }); }); } ``` ## 2. Register The Plugin [#2-register-the-plugin] ```ts import Fastify from "fastify"; import { supportRoutes } from "./routes/support"; export const app = Fastify({ logger: true }); await app.register(supportRoutes, { prefix: "/api" }); ``` ## 3. Call The Route [#3-call-the-route] ```ts const response = await fetch("http://localhost:3000/api/support", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ message: "How long does a reset link last?" }), }); const data = await response.json(); console.log(data.output); ``` ## Next [#next] Return live run events in [Streaming](/docs/frameworks/fastify/04-streaming). For response fields, read [Prompt Responses](/docs/guides/sdk-fundamentals/prompt-responses). # 04 Streaming (/docs/frameworks/fastify/04-streaming) Fastify replies can send stream-like payloads. Set NDJSON headers and return Anvia's `readableStream()`. ## 1. Add `/api/support/stream` [#1-add-apisupportstream] ```ts import type { FastifyInstance } from "fastify"; import { z } from "zod"; import { supportAgent } from "../ai/support-agent"; const SupportStreamRequest = z.object({ message: z.string().trim().min(1, "message is required"), }); export async function supportRoutes(app: FastifyInstance) { app.post("/support/stream", async (request, reply) => { const parsed = SupportStreamRequest.safeParse(request.body); if (!parsed.success) { return reply.status(400).send({ error: { code: "bad_request", message: parsed.error.issues[0]?.message }, }); } return reply .header("Content-Type", "application/x-ndjson") .header("Cache-Control", "no-cache") .send(supportAgent.prompt(parsed.data.message).readableStream()); }); } ``` ## 2. Consume The Stream [#2-consume-the-stream] ```ts const response = await fetch("http://localhost:3000/api/support/stream", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ message: "Draft a support reply." }), }); const reader = response.body?.getReader(); const decoder = new TextDecoder(); while (reader) { const next = await reader.read(); if (next.done) break; for (const line of decoder.decode(next.value).split("\n")) { if (line.trim()) console.log(JSON.parse(line)); } } ``` ## 3. Operational Notes [#3-operational-notes] Keep reverse proxies from buffering NDJSON responses. Clients should handle both `final` and `error` events. ## Next [#next] Add auth, request-local tools, and retrieval in [Tools and Context](/docs/frameworks/fastify/05-tools-and-context). Related guides: [Readable Streams](/docs/guides/streaming/readable-streams) and [Streaming Events](/docs/guides/streaming/streaming-events). # 05 Tools and Context (/docs/frameworks/fastify/05-tools-and-context) Use Fastify decorators or hooks for auth. Pass request-local data into Anvia at the route boundary. ## 1. Add A User Decoration [#1-add-a-user-decoration] ```ts import fp from "fastify-plugin"; declare module "fastify" { interface FastifyRequest { user?: { id: string }; } } export const authPlugin = fp(async (app) => { app.addHook("preHandler", async (request, reply) => { const user = await auth.userFromRequest(request); if (!user) { return reply.status(401).send({ error: { code: "unauthorized" } }); } request.user = { id: user.id }; }); }); ``` ## 2. Build Request-Local Tools [#2-build-request-local-tools] ```ts import { createTool } from "@anvia/core"; import { z } from "zod"; export function createAccountTool(input: { userId: string }) { return createTool({ name: "get_account_status", description: "Read the authenticated user's account status.", input: z.object({}), output: z.object({ plan: z.string(), openTickets: z.number() }), async execute() { return db.account.findStatus({ userId: input.userId }); }, }); } ``` ## 3. Attach Context In The Route [#3-attach-context-in-the-route] ```ts app.post("/support", async (request, reply) => { const { message } = SupportRequest.parse(request.body); const userId = request.user?.id; if (!userId) { return reply.status(401).send({ error: { code: "unauthorized" } }); } const response = await supportAgent .prompt(message) .tool(createAccountTool({ userId })) .context({ userId }) .send(); return reply.send({ output: response.output }); }); ``` ## 4. Add Retrieval Context [#4-add-retrieval-context] ```ts const documents = await knowledge.search({ query: message, filter: { userId: request.user.id }, limit: 5, }); const response = await supportAgent.prompt(message).documents(documents).send(); ``` ## Next [#next] Persist history in [Persistence](/docs/frameworks/fastify/06-persistence). Related guides: [Runtime Context](/docs/guides/agents/runtime-context), [Tool Handlers](/docs/guides/tools/tool-handlers), and [RAG Context](/docs/guides/retrieval/rag-context). # 06 Persistence (/docs/frameworks/fastify/06-persistence) Fastify gives you routing and lifecycle hooks. Your app owns session storage and message persistence. ## 1. Load Existing Messages [#1-load-existing-messages] ```ts const session = await db.chatSession.findUnique({ where: { id: request.params.sessionId, userId: request.user.id }, include: { messages: { orderBy: { createdAt: "asc" } } }, }); if (!session) { return reply.status(404).send({ error: { code: "not_found" } }); } ``` ## 2. Send With History [#2-send-with-history] ```ts const response = await supportAgent .prompt(message) .messages(session.messages.map((item) => item.message)) .send(); ``` ## 3. Store New Messages [#3-store-new-messages] ```ts await db.chatMessage.createMany({ data: response.messages.map((message) => ({ sessionId: session.id, message, })), }); ``` Store messages only after the run succeeds. Use a transaction when message writes must commit with application state changes. ## 4. Use Memory When The Model Should Remember [#4-use-memory-when-the-model-should-remember] Use chat history for conversation continuity. Use Anvia memory for durable facts that should be recalled across future runs. ## Next [#next] Prepare runtime constraints in [Deploy](/docs/frameworks/fastify/07-deploy). Related guides: [Memory](/docs/guides/memory), [Memory and Sessions](/docs/guides/sdk-fundamentals/memory-and-sessions), and [Agent History](/docs/guides/agents/agent-history). # 07 Deploy (/docs/frameworks/fastify/07-deploy) Fastify is a good fit for long-lived Node services. Configure timeouts and stream behavior explicitly. ## 1. Start The Server [#1-start-the-server] ```ts import { app } from "./app"; const port = Number(process.env.PORT ?? 3000); await app.listen({ host: "0.0.0.0", port }); ``` ## 2. Configure Environment Variables [#2-configure-environment-variables] ```txt OPENAI_API_KEY=sk_... DATABASE_URL=... ANVIA_STUDIO_TOKEN=... ``` ## 3. Streaming Checks [#3-streaming-checks] Confirm `application/x-ndjson` responses flush through your proxy and hosting layer. ## 4. Production Checklist [#4-production-checklist] | Check | Why | | ---------------------------- | ------------------------------------------- | | Body limits configured | Avoid unbounded JSON requests | | Error handler installed | Keep provider errors out of response bodies | | Stream proxy behavior tested | NDJSON needs incremental flushing | | Tracing connected | Tool and provider runs need observability | ## Next [#next] Debug common failures in [Troubleshooting](/docs/frameworks/fastify/08-troubleshooting). Add telemetry with [Observability](/docs/guides/observability/tracing). # 08 Troubleshooting (/docs/frameworks/fastify/08-troubleshooting) Most Fastify issues come from missing body validation, plugin ordering, or streams being buffered by infrastructure. ## Auth Is Not Available In Routes [#auth-is-not-available-in-routes] Register auth plugins before support routes: ```ts await app.register(authPlugin); await app.register(supportRoutes, { prefix: "/api" }); ``` ## Validation Returns 500 [#validation-returns-500] Validate unknown bodies with `safeParse` and return a 400 from the route. ```ts const parsed = SupportRequest.safeParse(request.body); if (!parsed.success) { return reply.status(400).send({ error: { code: "bad_request" } }); } ``` ## Stream Does Not Flush [#stream-does-not-flush] Set the NDJSON content type and check proxy buffering. ```ts return reply .header("Content-Type", "application/x-ndjson") .send(agent.prompt(message).readableStream()); ``` ## Provider Failures Leak Details [#provider-failures-leak-details] Install a Fastify error handler that logs internal details and returns a stable application error shape. ## Long Runs Time Out [#long-runs-time-out] Increase application, proxy, and platform timeouts. For reviewer waits, store approvals and resume through a decision endpoint. ## Next [#next] Add reviewer workflows in [Human in the Loop](/docs/frameworks/fastify/09-human-in-the-loop). Related guides: [Tool Errors](/docs/guides/tools/tool-errors), [Readable Streams](/docs/guides/streaming/readable-streams), and [Tracing](/docs/guides/observability/tracing). # 09 Human in the Loop (/docs/frameworks/fastify/09-human-in-the-loop) Fastify plugins can expose agent routes and approval routes together. Anvia supplies hooks; your app supplies approval storage and reviewer permissions. ## 1. Use Studio During Development [#1-use-studio-during-development] ```ts import { Studio } from "@anvia/studio"; import { supportAgent } from "../ai/support-agent"; new Studio([supportAgent]).start({ port: 4021 }); ``` Studio helps inspect approvals locally. Production approval storage and reviewer workflow belong to your app. ## 2. Create A Hook [#2-create-a-hook] ```ts import { createHook } from "@anvia/core"; import { approvalRuntime } from "../approvals/runtime"; export function createApprovalHook(input: { userId: string; approvalRunId: string }) { return createHook({ async onToolCall({ toolName, args, tool }) { if (toolName !== "refund_order") { return tool.run(); } const approved = await approvalRuntime.waitForDecision({ userId: input.userId, approvalRunId: input.approvalRunId, toolName, args, }); return approved ? tool.run() : tool.skip("Refund was not approved."); }, }); } ``` `approvalRuntime` is user code. It is not exported by Anvia. ## 3. Create The Approval Runtime [#3-create-the-approval-runtime] ```ts type ApprovalRequest = { userId: string; approvalRunId: string; toolName: string; args: string; }; type ApprovalDecision = { approved: boolean; reason?: string; }; export function createApprovalRuntime() { const waiters = new Map void>(); return { async waitForDecision(request: ApprovalRequest): Promise { const approval = await db.approval.create({ data: { ...request, status: "pending" }, }); await notifyReviewers({ approvalId: approval.id }); const decision = await new Promise((resolve) => { waiters.set(approval.id, resolve); }); waiters.delete(approval.id); return decision.approved; }, async decide(input: { approvalId: string; approved: boolean; reason?: string }) { await db.approval.update({ where: { id: input.approvalId }, data: { status: input.approved ? "approved" : "rejected", decisionReason: input.reason, resolvedAt: new Date(), }, }); waiters.get(input.approvalId)?.({ approved: input.approved, reason: input.reason, }); }, }; } export const approvalRuntime = createApprovalRuntime(); ``` Use durable storage and an external wakeup mechanism in production. ## 4. Add Reviewer Routes [#4-add-reviewer-routes] ```ts const DecisionRequest = z.object({ approved: z.boolean(), reason: z.string().optional(), }); app.post("/approvals/:id/decision", async (request, reply) => { const decision = DecisionRequest.parse(request.body); const params = request.params as { id: string }; await approvalRuntime.decide({ approvalId: params.id, ...decision, }); return reply.send({ ok: true }); }); ``` ## Next [#next] Add route tests in [Setup Tests](/docs/frameworks/fastify/10-setup-tests). Core concepts: [Human in the Loop](/docs/guides/human-in-the-loop), [Approval by Hooks](/docs/guides/human-in-the-loop/tool-approval), and [Approval Runtimes](/docs/guides/human-in-the-loop/approval-handlers). # 10 Setup Tests (/docs/frameworks/fastify/10-setup-tests) Use Fastify `inject` for route tests. Mock the agent so unit tests do not call providers. ## 1. Install Test Tools [#1-install-test-tools] ```sh pnpm add -D vitest ``` ## 2. Test The JSON Route [#2-test-the-json-route] ```ts import { describe, expect, it, vi } from "vitest"; import { app } from "../src/app"; vi.mock("../src/ai/support-agent", () => ({ supportAgent: { prompt: () => ({ send: async () => ({ output: "Reset links expire after 30 minutes.", usage: { totalTokens: 12 }, messages: [], }), }), }, })); describe("POST /api/support", () => { it("returns the agent output", async () => { const response = await app.inject({ method: "POST", url: "/api/support", payload: { message: "How long does a reset link last?" }, }); expect(response.statusCode).toBe(200); expect(response.json().output).toBe("Reset links expire after 30 minutes."); }); }); ``` ## 3. Test The Stream Route [#3-test-the-stream-route] ```ts const response = await app.inject({ method: "POST", url: "/api/support/stream", payload: { message: "Hello" }, }); expect(response.headers["content-type"]).toContain("application/x-ndjson"); ``` Mock `readableStream()` with a small `ReadableStream` that emits one `final` event. ## 4. Test Studio Without A Port [#4-test-studio-without-a-port] ```ts import { Studio } from "@anvia/studio"; import { supportAgent } from "../src/ai/support-agent"; const studio = new Studio([supportAgent]); const response = await studio.fetch( new Request("http://studio.test/agents/support/runs", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ message: "Hello" }), }), ); expect(response.status).toBe(200); ``` ## Next [#next] Related guides: [Testing](/docs/guides/testing), [Tools and Pipelines](/docs/guides/testing/tools-and-pipelines), and [Studio and Providers](/docs/guides/testing/studio-and-providers). # 01 Prep (/docs/frameworks/hono/01-prep) Use this path when Anvia will run behind a plain Hono server or a framework that exposes Hono routes. ## 1. Create a Hono Project [#1-create-a-hono-project] ```sh mkdir anvia-hono cd anvia-hono pnpm init pnpm add hono @hono/node-server pnpm add -D tsx typescript @types/node ``` ## 2. Install Anvia [#2-install-anvia] ```sh pnpm add @anvia/core @anvia/openai @hono/zod-validator zod ``` Install other provider packages when you need them: ```sh pnpm add @anvia/anthropic @anvia/gemini @anvia/mistral ``` ## 3. Add Environment Variables [#3-add-environment-variables] Anvia clients use explicit constructor options. ```txt OPENAI_API_KEY=sk_... ``` Read the value in server code: ```ts const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) { throw new Error("OPENAI_API_KEY is required"); } ``` ## 4. Choose File Boundaries [#4-choose-file-boundaries] | File | Purpose | | ------------------------- | ------------------------------------------------- | | `src/ai/support-agent.ts` | Provider client, model, tools, and reusable agent | | `src/app.ts` | Hono app and routes | | `src/server.ts` | Node server entry point | Hono handlers receive `c.req`, but they can also return standard Web `Response` objects. That makes Anvia streaming direct. ## Next [#next] Build the reusable agent in [Setup Anvia](/docs/frameworks/hono/02-setup-anvia). Read [How Anvia Works](/docs/guides/sdk-fundamentals/runtime-boundaries) for the SDK boundaries. # 02 Setup Anvia (/docs/frameworks/hono/02-setup-anvia) Create provider clients, models, and shared tools outside the Hono handler when their configuration is the same for every request. ## 1. Create `src/ai/support-agent.ts` [#1-create-srcaisupport-agentts] ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) { throw new Error("OPENAI_API_KEY is required"); } const client = new OpenAIClient({ apiKey }); export const model = client.completionModel("gpt-5.5"); const lookupPolicy = createTool({ name: "lookup_policy", description: "Look up a short support policy by key.", input: z.object({ key: z.enum(["password_reset", "priority_support"]), }), output: z.object({ text: z.string(), }), async execute({ key }) { const policies = { password_reset: "Password reset links expire after 30 minutes.", priority_support: "Enterprise customers receive priority support.", }; return { text: policies[key] }; }, }); export const supportAgent = new AgentBuilder("support", model) .instructions("Answer support questions clearly. Use tools for policy facts.") .tool(lookupPolicy) .defaultMaxTurns(3) .build(); ``` ## 2. Create `src/app.ts` [#2-create-srcappts] ```ts import { Hono } from "hono"; export const app = new Hono(); app.get("/health", (c) => c.json({ ok: true })); ``` ## 3. Create `src/server.ts` [#3-create-srcserverts] ```ts import { serve } from "@hono/node-server"; import { app } from "./app"; serve({ fetch: app.fetch, port: 3000, }); ``` Run it: ```sh pnpm exec tsx src/server.ts ``` ## Next [#next] Expose the agent through a JSON route in [Route Handler](/docs/frameworks/hono/03-route-handler). Related guides: [Creating Agents](/docs/guides/agents/creating-agents), [Tools](/docs/guides/tools/creating-tools), and [Provider Clients](/docs/guides/sdk-fundamentals/clients-and-models). # 03 Route Handler (/docs/frameworks/hono/03-route-handler) Validate JSON at the Hono boundary with `zValidator(...)`, then read the typed body with `c.req.valid("json")`. ## 1. Add `/api/support` [#1-add-apisupport] ```ts import { Hono } from "hono"; import { zValidator } from "@hono/zod-validator"; import { z } from "zod"; import { supportAgent } from "./ai/support-agent"; export const app = new Hono(); const SupportRequest = z.object({ message: z.string().trim().min(1, "message is required"), }); app.post("/api/support", zValidator("json", SupportRequest), async (c) => { const { message } = c.req.valid("json"); const response = await supportAgent.prompt(message).send(); return c.json({ output: response.output, usage: response.usage, messages: response.messages, }); }); ``` ## 2. Call The Route [#2-call-the-route] ```sh curl -X POST http://localhost:3000/api/support \ -H "Content-Type: application/json" \ -d '{"message":"How long does a reset link last?"}' ``` ## 3. Return Structured Failures [#3-return-structured-failures] ```ts app.post("/api/support", zValidator("json", SupportRequest), async (c) => { try { const { message } = c.req.valid("json"); const response = await supportAgent.prompt(message).send(); return c.json({ output: response.output }); } catch (error) { console.error(error); return c.json({ error: "agent_failed" }, 500); } }); ``` ## Next [#next] Return live events in [Streaming](/docs/frameworks/hono/04-streaming). For prompt response fields, read [Prompt Responses](/docs/guides/sdk-fundamentals/prompt-responses). # 04 Streaming (/docs/frameworks/hono/04-streaming) Hono handlers can return a standard `Response`, so Anvia `readableStream()` can be used directly. ## 1. Add `/api/support/stream` [#1-add-apisupportstream] ```ts import { zValidator } from "@hono/zod-validator"; import { z } from "zod"; import { supportAgent } from "./ai/support-agent"; const SupportStreamRequest = z.object({ message: z.string().trim().min(1, "message is required"), }); app.post("/api/support/stream", zValidator("json", SupportStreamRequest), async (c) => { const { message } = c.req.valid("json"); return new Response(supportAgent.prompt(message).readableStream(), { headers: { "Content-Type": "application/x-ndjson", "Cache-Control": "no-cache", }, }); }); ``` ## 2. Consume The Stream [#2-consume-the-stream] ```ts const response = await fetch("http://localhost:3000/api/support/stream", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ message: "Draft a support reply." }), }); const reader = response.body?.getReader(); const decoder = new TextDecoder(); while (reader) { const next = await reader.read(); if (next.done) break; for (const line of decoder.decode(next.value).split("\n")) { if (line.trim()) console.log(JSON.parse(line)); } } ``` ## 3. Handle Stream Errors [#3-handle-stream-errors] Anvia writes a terminal `error` event if iteration fails. Clients should handle both `final` and `error`. ## Next [#next] Add auth, request-local tools, and retrieval in [Tools and Context](/docs/frameworks/hono/05-tools-and-context). Related guides: [Readable Streams](/docs/guides/streaming/readable-streams) and [Streaming Events](/docs/guides/streaming/streaming-events). # 05 Tools and Context (/docs/frameworks/hono/05-tools-and-context) Hono gives you the raw request. Resolve auth in middleware or the handler, then create scoped tools when a tool needs the current user. ## 1. Add Auth Middleware [#1-add-auth-middleware] ```ts type Variables = { userId: string; }; export const app = new Hono<{ Variables: Variables }>(); app.use("/api/*", async (c, next) => { const userId = c.req.header("x-user-id"); if (!userId) { return c.json({ error: "unauthorized" }, 401); } c.set("userId", userId); await next(); }); ``` ## 2. Create a Scoped Agent [#2-create-a-scoped-agent] ```ts import { z } from "zod"; import { AgentBuilder, createTool } from "@anvia/core"; import { model } from "./ai/support-agent"; import { orders } from "./db/orders"; export function createSupportAgent(scope: { userId: string }) { const lookupOrder = createTool({ name: "lookup_order", description: "Look up one order owned by the current user.", input: z.object({ orderId: z.string(), }), output: z.object({ status: z.string(), }), async execute({ orderId }) { return orders.findForUser(scope.userId, orderId); }, }); return new AgentBuilder("support", model) .instructions("Use tools for account-specific data.") .tool(lookupOrder) .defaultMaxTurns(3) .build(); } ``` ## 3. Use Request State In The Handler [#3-use-request-state-in-the-handler] ```ts import { zValidator } from "@hono/zod-validator"; import { z } from "zod"; const SupportRequest = z.object({ message: z.string().trim().min(1, "message is required"), }); app.post("/api/support", zValidator("json", SupportRequest), async (c) => { const userId = c.get("userId"); const { message } = c.req.valid("json"); const agent = createSupportAgent({ userId }); const response = await agent.prompt(message).send(); return c.json({ output: response.output }); }); ``` ## 4. Add Retrieval Context [#4-add-retrieval-context] ```ts const agent = new AgentBuilder("support", model) .instructions("Use retrieved support docs when relevant.") .dynamicContext(supportDocsIndex, { topK: 3, threshold: 0.7, }) .tool(lookupOrder) .build(); ``` Use retrieval for knowledge. Use tools for authorization-sensitive application state. ## Next [#next] Persist conversations in [Persistence](/docs/frameworks/hono/06-persistence). Related guides: [Runtime Context](/docs/guides/agents/runtime-context), [RAG Context](/docs/guides/retrieval/rag-context), and [Tool Handlers](/docs/guides/tools/tool-handlers). # 06 Persistence (/docs/frameworks/hono/06-persistence) Hono does not prescribe persistence. Keep storage in your application layer and pass messages into Anvia. ## 1. Explicit Transcript Storage [#1-explicit-transcript-storage] ```ts import { zValidator } from "@hono/zod-validator"; import { Message } from "@anvia/core"; import { z } from "zod"; import { supportAgent } from "./ai/support-agent"; import { conversations } from "./db/conversations"; const SupportRequest = z.object({ conversationId: z.string().min(1), message: z.string().trim().min(1, "message is required"), }); app.post("/api/support", zValidator("json", SupportRequest), async (c) => { const userId = c.get("userId"); const { conversationId, message } = c.req.valid("json"); const history = await conversations.loadMessages(userId, conversationId); const response = await supportAgent .prompt([...history, Message.user(message)]) .send(); await conversations.saveMessages(userId, conversationId, [ ...history, ...response.messages, ]); return c.json({ output: response.output }); }); ``` ## 2. Agent Memory [#2-agent-memory] ```ts const agent = new AgentBuilder("support", model) .memory(memoryStore, { savePolicy: "message" }) .build(); const response = await agent .session(conversationId, { userId }) .prompt(message) .send(); ``` Use memory when Anvia should load and append transcript messages through your store. ## 3. Studio During Development [#3-studio-during-development] You can inspect the same built agent in Studio without changing the Hono route: ```ts import { Studio } from "@anvia/studio"; import { supportAgent } from "./ai/support-agent"; new Studio([supportAgent]).start({ port: 4021 }); ``` Studio is for local inspection and internal tooling. Your Hono app still owns product auth, routes, and persistence. ## Next [#next] Review deployment checks in [Deploy](/docs/frameworks/hono/07-deploy). Related guides: [Memory](/docs/guides/memory), [Studio](/docs/studio/overview), and [Event Store](/docs/guides/agents/event-store). # 07 Deploy (/docs/frameworks/hono/07-deploy) ## Runtime Checklist [#runtime-checklist] | Area | Check | | --------- | ------------------------------------------------------------------ | | Runtime | Provider SDKs and storage clients work in your chosen Hono adapter | | Secrets | Provider keys are server-only environment variables | | Streaming | Host and proxy do not buffer `application/x-ndjson` responses | | Timeouts | Request timeout covers model latency and tool calls | | Storage | Conversations, memory, retrieval indexes, and traces are durable | ## Node Server Example [#node-server-example] ```ts import { serve } from "@hono/node-server"; import { app } from "./app"; serve({ fetch: app.fetch, hostname: "0.0.0.0", port: Number(process.env.PORT ?? 3000), }); ``` ## Observability [#observability] ```ts const response = await supportAgent .prompt(message) .withTrace({ name: "hono-support-route", userId, sessionId: conversationId, tags: ["hono"], }) .send(); ``` Attach observers for logs, metrics, Langfuse, or OpenTelemetry. ## Deployment Smoke Test [#deployment-smoke-test] ```sh curl -X POST "$APP_URL/api/support" \ -H "Content-Type: application/json" \ -d '{"message":"Say hello"}' ``` ## Next [#next] Use [Troubleshooting](/docs/frameworks/hono/08-troubleshooting) for common failures. Related guides: [Observers](/docs/guides/observability/observers), [Langfuse](/docs/guides/observability/langfuse), and [OpenTelemetry](/docs/guides/observability/otel). # 08 Troubleshooting (/docs/frameworks/hono/08-troubleshooting) ## Validation Returns 400 [#validation-returns-400] The route uses `zValidator("json", schema)`, so the request body must match the Zod schema. ```sh curl -X POST http://localhost:3000/api/support \ -H "Content-Type: application/json" \ -d '{"message":"Hello"}' ``` ## JSON Validation Fails [#json-validation-fails] Set the request header and send valid JSON: ```txt Content-Type: application/json ``` Then read validated data with `c.req.valid("json")` inside the handler: ```ts import { zValidator } from "@hono/zod-validator"; import { z } from "zod"; const SupportRequest = z.object({ message: z.string().trim().min(1, "message is required"), }); app.post("/api/support", zValidator("json", SupportRequest), async (c) => { const { message } = c.req.valid("json"); return c.json({ message }); }); ``` ## Provider Key Is Missing [#provider-key-is-missing] Create the provider client only after checking the environment: ```ts const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) throw new Error("OPENAI_API_KEY is required"); ``` ## Streaming Does Not Flush [#streaming-does-not-flush] Return a Web `Response` with the Anvia stream and NDJSON content type: ```ts return new Response(agent.prompt(message).readableStream(), { headers: { "Content-Type": "application/x-ndjson" }, }); ``` Check adapter support, reverse proxy buffering, and route timeouts. ## Tool Authorization Is Wrong [#tool-authorization-is-wrong] Resolve the current user in middleware or the handler. Build scoped tools that close over that user, and never trust model-supplied identifiers for access control. ## Studio Works But Hono Route Does Not [#studio-works-but-hono-route-does-not] Studio runs the same agent runtime but different HTTP routes. Compare the prompt input, session history, tools, and provider environment used by your Hono handler. ## Next [#next] Revisit [Route Handler](/docs/frameworks/hono/03-route-handler), [Tools and Context](/docs/frameworks/hono/05-tools-and-context), and [Tool Errors](/docs/guides/tools/tool-errors). # 09 Human in the Loop (/docs/frameworks/hono/09-human-in-the-loop) Hono is a good fit for human-in-the-loop routes because the same app can expose agent runs, approval lists, and decision endpoints. ## 1. Use Studio During Development [#1-use-studio-during-development] Add approval metadata to protected tools, then register the built agent in Studio: ```ts import { Studio } from "@anvia/studio"; import { supportAgent } from "./ai/support-agent"; new Studio([supportAgent]).start({ port: 4021 }); ``` Studio gives you a local approval UI. Your Hono app still owns production auth and reviewer permissions. ## 2. Use A Request Hook In Hono [#2-use-a-request-hook-in-hono] ```ts import { createHook } from "@anvia/core"; import { approvalRuntime } from "./approvals/runtime"; function createApprovalHook(input: { userId: string; approvalRunId: string }) { return createHook({ async onToolCall({ toolName, args, tool }) { if (toolName !== "refund_order") { return tool.run(); } const approved = await approvalRuntime.waitForDecision({ userId: input.userId, approvalRunId: input.approvalRunId, toolName, args, }); return approved ? tool.run() : tool.skip("Refund was not approved."); }, }); } ``` `approvalRuntime` is your own module. It is not imported from `@anvia/core` or any Anvia package. Attach the hook inside the route: ```ts import { zValidator } from "@hono/zod-validator"; import { z } from "zod"; const SupportRequest = z.object({ message: z.string().trim().min(1, "message is required"), }); app.post("/api/support", zValidator("json", SupportRequest), async (c) => { const userId = c.get("userId"); const { message } = c.req.valid("json"); const approvalRunId = crypto.randomUUID(); const response = await supportAgent .prompt(message) .requestHook(createApprovalHook({ userId, approvalRunId })) .send(); return c.json({ output: response.output }); }); ``` ## 3. Create The Approval Runtime [#3-create-the-approval-runtime] `approvalRuntime` is not provided by Anvia. It is your Hono application's approval service: create a pending record, notify reviewers, wait for a decision route to resolve the pending promise, then return the boolean to the hook. ```ts type ApprovalRequest = { userId: string; approvalRunId: string; toolName: string; args: string; }; type ApprovalDecision = { approved: boolean; reason?: string; }; export function createApprovalRuntime() { const waiters = new Map void>(); return { async waitForDecision(request: ApprovalRequest): Promise { const approval = await db.approval.create({ data: { userId: request.userId, approvalRunId: request.approvalRunId, toolName: request.toolName, args: request.args, status: "pending", }, }); await notifyReviewers({ approvalId: approval.id }); const decision = await new Promise((resolve) => { waiters.set(approval.id, resolve); }); waiters.delete(approval.id); return decision.approved; }, async listPendingForReviewer(reviewerId: string) { return db.approval.findMany({ where: { reviewerId, status: "pending", }, orderBy: { createdAt: "asc" }, }); }, async decide(input: { approvalId: string; reviewerId: string; approved: boolean; reason?: string; }): Promise { await db.approval.update({ where: { id: input.approvalId }, data: { status: input.approved ? "approved" : "rejected", reviewerId: input.reviewerId, decisionReason: input.reason, resolvedAt: new Date(), }, }); waiters.get(input.approvalId)?.({ approved: input.approved, reason: input.reason, }); }, }; } export const approvalRuntime = createApprovalRuntime(); ``` The `Map` is only a simple waiter for one Node process. In production, keep records in durable storage and resolve waiters through your queue, pub/sub, websocket, or polling worker. ## 4. Add Reviewer Routes [#4-add-reviewer-routes] ```ts const ApprovalDecisionRequest = z.object({ approved: z.boolean(), }); app.get("/api/approvals", async (c) => { const userId = c.get("userId"); return c.json(await approvalRuntime.listPendingForReviewer(userId)); }); app.post( "/api/approvals/:id/decision", zValidator("json", ApprovalDecisionRequest), async (c) => { const userId = c.get("userId"); const { approved } = c.req.valid("json"); await approvalRuntime.decide({ approvalId: c.req.param("id"), reviewerId: userId, approved, }); return c.json({ ok: true }); }, ); ``` Use `zValidator("json", schema)` on the decision route the same way as prompt routes. ## Next [#next] Add Hono route tests in [Setup Tests](/docs/frameworks/hono/10-setup-tests). Core concepts: [Human in the Loop](/docs/guides/human-in-the-loop), [Approval by Hooks](/docs/guides/human-in-the-loop/tool-approval), [Approval Runtimes](/docs/guides/human-in-the-loop/approval-handlers), and [Studio Tool Approvals](/docs/studio/human-in-the-loop/tool-approvals). # 10 Setup Tests (/docs/frameworks/hono/10-setup-tests) Hono apps are straightforward to test because `app.request(...)` exercises the same route stack without starting a server. ## 1. Install Test Tools [#1-install-test-tools] ```sh pnpm add -D vitest ``` ## 2. Test JSON Validation [#2-test-json-validation] ```ts import { describe, expect, it } from "vitest"; import { app } from "../src/app"; describe("POST /api/support", () => { it("rejects invalid JSON bodies", async () => { const response = await app.request("/api/support", { method: "POST", headers: { "content-type": "application/json", "x-user-id": "user_123", }, body: JSON.stringify({ message: "" }), }); expect(response.status).toBe(400); }); }); ``` `zValidator("json", schema)` handles the validation failure before the agent runs. ## 3. Test The Happy Path [#3-test-the-happy-path] Mock the agent module for route tests so provider calls stay out of unit tests. ```ts import { vi } from "vitest"; vi.mock("../src/ai/support-agent", () => ({ supportAgent: { prompt: () => ({ send: async () => ({ output: "Hello", usage: { totalTokens: 4 }, messages: [] }), }), }, })); ``` Then call the route with `app.request(...)` and assert the JSON body. ## 4. Test Streaming Headers [#4-test-streaming-headers] ```ts const response = await app.request("/api/support/stream", { method: "POST", headers: { "content-type": "application/json", "x-user-id": "user_123", }, body: JSON.stringify({ message: "Hello" }), }); expect(response.headers.get("content-type")).toContain("application/x-ndjson"); ``` Mock `readableStream()` with a small `ReadableStream` that emits one final event. ## 5. Test Studio Without a Port [#5-test-studio-without-a-port] ```ts import { Studio } from "@anvia/studio"; import { supportAgent } from "../src/ai/support-agent"; const studio = new Studio([supportAgent]); const response = await studio.fetch( new Request("http://studio.test/agents/support/runs", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ message: "Hello" }), }), ); expect(response.status).toBe(200); ``` ## Next [#next] Related guides: [Testing](/docs/guides/testing), [Tools and Pipelines](/docs/guides/testing/tools-and-pipelines), and [Studio and Providers](/docs/guides/testing/studio-and-providers). # Framework Guides (/docs/frameworks) These guides show how to move Anvia from a local script into real HTTP runtimes. Each framework follows the same path: | Step | Goal | | -------------------- | ------------------------------------------------------------------------ | | 01 Prep | Create the project shape and install Anvia packages | | 02 Setup Anvia | Build provider clients, models, tools, and agents in server-side modules | | 03 Route Handler | Return a normal JSON response from a framework route | | 04 Streaming | Return newline-delimited stream events from the same agent | | 05 Tools and Context | Pass request-local auth, product data, and retrieval context safely | | 06 Persistence | Store conversation history through your application storage | | 07 Deploy | Check runtime, environment, and operational constraints | | 08 Troubleshooting | Fix common framework and provider failures | | 09 Human in the Loop | Add approvals, questions, and reviewer waits to framework routes | | 10 Setup Tests | Test route handlers, streams, Studio wiring, and provider boundaries | Start with the framework that owns your HTTP routes: | Framework | Start here | Route shape | | -------------- | -------------------------------------------------------------- | --------------------------------------- | | Next.js | [Next.js Prep](/docs/frameworks/nextjs/01-prep) | App Router `route.ts` handlers | | TanStack Start | [TanStack Start Prep](/docs/frameworks/tanstack-start/01-prep) | `createServerFn(...)` and server routes | | SvelteKit | [SvelteKit Prep](/docs/frameworks/sveltekit/01-prep) | `+server.ts` endpoint modules | | Hono | [Hono Prep](/docs/frameworks/hono/01-prep) | Plain Hono routes | | Express | [Express Prep](/docs/frameworks/express/01-prep) | Router middleware and handlers | | Fastify | [Fastify Prep](/docs/frameworks/fastify/01-prep) | Plugin routes and replies | | NestJS | [NestJS Prep](/docs/frameworks/nestjs/01-prep) | Modules, services, and controllers | The Anvia runtime shape stays the same across every framework: ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); export const supportAgent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .defaultMaxTurns(3) .build(); ``` For the underlying SDK concepts, read [How Anvia Works](/docs/guides/sdk-fundamentals/runtime-boundaries), [Creating Agents](/docs/guides/agents/creating-agents), [Tools](/docs/guides/tools/creating-tools), [Readable Streams](/docs/guides/streaming/readable-streams), and [Memory](/docs/guides/memory). # 01 Prep (/docs/frameworks/nestjs/01-prep) Use this path when Anvia runs inside a NestJS backend with modules, dependency injection, and controllers. ## 1. Create A NestJS Project [#1-create-a-nestjs-project] ```sh pnpm dlx @nestjs/cli new anvia-nestjs cd anvia-nestjs ``` Use TypeScript and the default Express HTTP adapter unless your app already uses Fastify. ## 2. Install Anvia [#2-install-anvia] ```sh pnpm add @anvia/core @anvia/openai zod pnpm add -D @types/express ``` Install other providers when needed: ```sh pnpm add @anvia/anthropic @anvia/gemini @anvia/mistral ``` ## 3. Add Environment Variables [#3-add-environment-variables] ```txt OPENAI_API_KEY=sk_... ``` Read secrets through your config layer or `process.env`: ```ts const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) { throw new Error("OPENAI_API_KEY is required"); } ``` ## 4. Choose Module Boundaries [#4-choose-module-boundaries] | File | Purpose | | ------------------------------------------- | ------------------------------------------------ | | `src/ai/anvia.module.ts` | Nest module for Anvia providers | | `src/ai/support-agent.service.ts` | Provider client, model, tools, and agent methods | | `src/support/support.controller.ts` | HTTP prompt and stream endpoints | | `src/approvals/approval-runtime.service.ts` | User-owned approval runtime | ## Next [#next] Build the injectable service in [Setup Anvia](/docs/frameworks/nestjs/02-setup-anvia). Read [Runtime Boundaries](/docs/guides/sdk-fundamentals/runtime-boundaries) before wiring Anvia into Nest modules. # 02 Setup Anvia (/docs/frameworks/nestjs/02-setup-anvia) In NestJS, wrap Anvia in services. Controllers should depend on methods like `runSupport(...)`, not construct provider clients directly. ## 1. Create `src/ai/support-agent.service.ts` [#1-create-srcaisupport-agentservicets] ```ts import { Injectable } from "@nestjs/common"; import { AgentBuilder, createTool, type Agent } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; @Injectable() export class SupportAgentService { private readonly agent: Agent; constructor() { const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) { throw new Error("OPENAI_API_KEY is required"); } const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); const lookupPolicy = createTool({ name: "lookup_policy", description: "Look up a support policy by key.", input: z.object({ key: z.enum(["password_reset", "priority_support"]), }), output: z.object({ text: z.string(), }), async execute({ key }) { const policies = { password_reset: "Password reset links expire after 30 minutes.", priority_support: "Enterprise customers receive priority support.", }; return { text: policies[key] }; }, }); this.agent = new AgentBuilder("support", model) .name("Support Agent") .instructions("Answer clearly. Use tools when policy detail is needed.") .tool(lookupPolicy) .defaultMaxTurns(3) .build(); } async runSupport(message: string) { return this.agent.prompt(message).send(); } streamSupport(message: string) { return this.agent.prompt(message).readableStream(); } getAgent() { return this.agent; } } ``` ## 2. Export The Service From A Module [#2-export-the-service-from-a-module] ```ts import { Module } from "@nestjs/common"; import { SupportAgentService } from "./support-agent.service"; @Module({ providers: [SupportAgentService], exports: [SupportAgentService], }) export class AnviaModule {} ``` ## 3. Swap Providers Later [#3-swap-providers-later] ```ts import { OpenAICompatibleClient } from "@anvia/openai"; const client = new OpenAICompatibleClient({ apiKey: process.env.OPENROUTER_API_KEY, baseURL: "https://openrouter.ai/api/v1", }); ``` ## Next [#next] Expose the service through a controller in [Route Handler](/docs/frameworks/nestjs/03-route-handler). Related guides: [Creating Agents](/docs/guides/agents/creating-agents) and [Tools](/docs/guides/tools/creating-tools). # 03 Route Handler (/docs/frameworks/nestjs/03-route-handler) NestJS controllers are the HTTP boundary. Validate request bodies before calling the Anvia service. ## 1. Create `src/support/support.controller.ts` [#1-create-srcsupportsupportcontrollerts] ```ts import { BadRequestException, Body, Controller, Post } from "@nestjs/common"; import { z } from "zod"; import { SupportAgentService } from "../ai/support-agent.service"; const SupportRequest = z.object({ message: z.string().trim().min(1, "message is required"), }); @Controller("api/support") export class SupportController { constructor(private readonly supportAgent: SupportAgentService) {} @Post() async prompt(@Body() body: unknown) { const parsed = SupportRequest.safeParse(body); if (!parsed.success) { throw new BadRequestException(parsed.error.issues[0]?.message); } const response = await this.supportAgent.runSupport(parsed.data.message); return { output: response.output, usage: response.usage, messages: response.messages, }; } } ``` ## 2. Register The Controller [#2-register-the-controller] ```ts import { Module } from "@nestjs/common"; import { AnviaModule } from "../ai/anvia.module"; import { SupportController } from "./support.controller"; @Module({ imports: [AnviaModule], controllers: [SupportController], }) export class SupportModule {} ``` ## 3. Call The Route [#3-call-the-route] ```ts const response = await fetch("http://localhost:3000/api/support", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ message: "How long does a reset link last?" }), }); const data = await response.json(); console.log(data.output); ``` ## 4. Keep Errors Structured [#4-keep-errors-structured] Use Nest exceptions or filters for application error shapes. Do not return raw provider stack traces. ## Next [#next] Return live run events in [Streaming](/docs/frameworks/nestjs/04-streaming). For response fields, read [Prompt Responses](/docs/guides/sdk-fundamentals/prompt-responses). # 04 Streaming (/docs/frameworks/nestjs/04-streaming) With the default Express adapter, inject `@Res()` and pipe Anvia's Web stream into the Node response. ## 1. Add A Streaming Controller Method [#1-add-a-streaming-controller-method] ```ts import { BadRequestException, Body, Controller, Post, Res } from "@nestjs/common"; import type { Response } from "express"; import { Readable } from "node:stream"; import { z } from "zod"; import { SupportAgentService } from "../ai/support-agent.service"; const SupportStreamRequest = z.object({ message: z.string().trim().min(1, "message is required"), }); @Controller("api/support") export class SupportController { constructor(private readonly supportAgent: SupportAgentService) {} @Post("stream") async stream(@Body() body: unknown, @Res() res: Response) { const parsed = SupportStreamRequest.safeParse(body); if (!parsed.success) { throw new BadRequestException(parsed.error.issues[0]?.message); } res.setHeader("Content-Type", "application/x-ndjson"); res.setHeader("Cache-Control", "no-cache"); const stream = Readable.fromWeb(this.supportAgent.streamSupport(parsed.data.message)); stream.pipe(res); } } ``` ## 2. Consume The Stream [#2-consume-the-stream] ```ts const response = await fetch("http://localhost:3000/api/support/stream", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ message: "Draft a support reply." }), }); const reader = response.body?.getReader(); const decoder = new TextDecoder(); while (reader) { const next = await reader.read(); if (next.done) break; for (const line of decoder.decode(next.value).split("\n")) { if (line.trim()) console.log(JSON.parse(line)); } } ``` ## 3. Adapter Notes [#3-adapter-notes] If your Nest app uses the Fastify adapter, use Fastify reply handling instead of Express `Response`. ## Next [#next] Add auth, request-local tools, and retrieval in [Tools and Context](/docs/frameworks/nestjs/05-tools-and-context). Related guides: [Readable Streams](/docs/guides/streaming/readable-streams) and [Streaming Events](/docs/guides/streaming/streaming-events). # 05 Tools and Context (/docs/frameworks/nestjs/05-tools-and-context) In NestJS, guards and services should own auth and data access. Pass request-local values into Anvia from controller or service methods. ## 1. Add A Request User Type [#1-add-a-request-user-type] ```ts import type { Request } from "express"; export type AuthenticatedRequest = Request & { user: { id: string }; }; ``` Use your existing guard to set `request.user`. ## 2. Build Request-Local Tools [#2-build-request-local-tools] ```ts import { createTool } from "@anvia/core"; import { z } from "zod"; export function createAccountTool(input: { userId: string }) { return createTool({ name: "get_account_status", description: "Read the authenticated user's account status.", input: z.object({}), output: z.object({ plan: z.string(), openTickets: z.number() }), async execute() { return db.account.findStatus({ userId: input.userId }); }, }); } ``` ## 3. Add A Request-Local Service Method [#3-add-a-request-local-service-method] ```ts async runSupportForUser(input: { userId: string; message: string }) { return this.agent .prompt(input.message) .tool(createAccountTool({ userId: input.userId })) .context({ userId: input.userId }) .send(); } ``` ## 4. Call It From A Guarded Controller [#4-call-it-from-a-guarded-controller] ```ts import { Controller, Post, Req, UseGuards } from "@nestjs/common"; import type { AuthenticatedRequest } from "../auth/types"; @UseGuards(AuthGuard) @Controller("api/support") export class SupportController { @Post() async prompt(@Req() request: AuthenticatedRequest) { const response = await this.supportAgent.runSupportForUser({ userId: request.user.id, message: request.body.message, }); return { output: response.output }; } } ``` ## 5. Add Retrieval Context [#5-add-retrieval-context] ```ts const documents = await this.knowledge.search({ query: input.message, filter: { userId: input.userId }, limit: 5, }); return this.agent.prompt(input.message).documents(documents).send(); ``` ## Next [#next] Persist history in [Persistence](/docs/frameworks/nestjs/06-persistence). Related guides: [Runtime Context](/docs/guides/agents/runtime-context), [Tool Handlers](/docs/guides/tools/tool-handlers), and [RAG Context](/docs/guides/retrieval/rag-context). # 06 Persistence (/docs/frameworks/nestjs/06-persistence) Use Nest services or repositories for session storage. Anvia should not own your database boundary. ## 1. Load Existing Messages [#1-load-existing-messages] ```ts const session = await this.chatSessions.findForUser({ sessionId: input.sessionId, userId: input.userId, }); if (!session) { throw new NotFoundException("Session not found"); } ``` ## 2. Send With History [#2-send-with-history] ```ts const response = await this.agent .prompt(input.message) .messages(session.messages.map((item) => item.message)) .send(); ``` ## 3. Store New Messages [#3-store-new-messages] ```ts await this.chatMessages.createMany( response.messages.map((message) => ({ sessionId: session.id, message, })), ); ``` Keep transaction ownership in your application services when message writes must commit with business records. ## 4. Use Memory When The Model Should Remember [#4-use-memory-when-the-model-should-remember] Use chat history for the current conversation. Use Anvia memory for durable user or domain facts that should survive across sessions. ## Next [#next] Prepare runtime constraints in [Deploy](/docs/frameworks/nestjs/07-deploy). Related guides: [Memory](/docs/guides/memory), [Memory and Sessions](/docs/guides/sdk-fundamentals/memory-and-sessions), and [Agent History](/docs/guides/agents/agent-history). # 07 Deploy (/docs/frameworks/nestjs/07-deploy) NestJS gives you a long-lived Node server by default. Size request timeouts and observability for model calls. ## 1. Start The App [#1-start-the-app] ```ts import { NestFactory } from "@nestjs/core"; import { AppModule } from "./app.module"; async function bootstrap() { const app = await NestFactory.create(AppModule); await app.listen(process.env.PORT ?? 3000); } void bootstrap(); ``` ## 2. Configure Environment Variables [#2-configure-environment-variables] ```txt OPENAI_API_KEY=sk_... DATABASE_URL=... ANVIA_STUDIO_TOKEN=... ``` ## 3. Streaming Checks [#3-streaming-checks] If you use the Express adapter, verify `res.setHeader(...)` and streaming through any reverse proxy. If you use the Fastify adapter, use Fastify-specific reply handling. ## 4. Production Checklist [#4-production-checklist] | Check | Why | | ------------------------------- | -------------------------------------------- | | Config module validates secrets | Fail early when provider keys are missing | | Exception filters installed | Keep provider stack traces out of responses | | Route timeouts reviewed | Agent runs may take longer than CRUD routes | | Observability module enabled | Tool calls and provider failures need traces | ## Next [#next] Debug common failures in [Troubleshooting](/docs/frameworks/nestjs/08-troubleshooting). Add telemetry with [Observability](/docs/guides/observability/tracing). # 08 Troubleshooting (/docs/frameworks/nestjs/08-troubleshooting) Most NestJS issues come from provider initialization, request validation, adapter-specific streaming, or dependency injection boundaries. ## `OPENAI_API_KEY is required` [#openai_api_key-is-required] Validate configuration before constructing provider clients. Prefer a config module for production apps. ## Service Cannot Be Injected [#service-cannot-be-injected] Export `SupportAgentService` from `AnviaModule` and import that module where your controller lives. ```ts @Module({ providers: [SupportAgentService], exports: [SupportAgentService], }) export class AnviaModule {} ``` ## Validation Returns 500 [#validation-returns-500] Validate body input and throw `BadRequestException` for user errors. ```ts const parsed = SupportRequest.safeParse(body); if (!parsed.success) { throw new BadRequestException(parsed.error.issues[0]?.message); } ``` ## Stream Does Not Flush [#stream-does-not-flush] Check which Nest HTTP adapter you use. Express examples use `@Res() res: Response`; Fastify apps need Fastify reply handling. ## Long Runs Time Out [#long-runs-time-out] Raise server, proxy, and platform timeouts. For reviewer waits, store approvals and resolve them from a decision endpoint. ## Next [#next] Add reviewer workflows in [Human in the Loop](/docs/frameworks/nestjs/09-human-in-the-loop). Related guides: [Tool Errors](/docs/guides/tools/tool-errors), [Readable Streams](/docs/guides/streaming/readable-streams), and [Tracing](/docs/guides/observability/tracing). # 09 Human in the Loop (/docs/frameworks/nestjs/09-human-in-the-loop) In NestJS, place approval storage and decision handling in injectable services. Anvia hooks call those services, but Anvia does not provide your approval runtime. ## 1. Use Studio During Development [#1-use-studio-during-development] ```ts import { Injectable, OnModuleInit } from "@nestjs/common"; import { Studio } from "@anvia/studio"; import { SupportAgentService } from "./support-agent.service"; @Injectable() export class StudioService implements OnModuleInit { constructor(private readonly supportAgent: SupportAgentService) {} onModuleInit() { new Studio([this.supportAgent.getAgent()]).start({ port: 4021 }); } } ``` Studio helps locally. Production reviewer permissions, records, and notifications belong to your Nest app. ## 2. Create An Approval Hook [#2-create-an-approval-hook] ```ts import { createHook } from "@anvia/core"; import { ApprovalRuntimeService } from "../approvals/approval-runtime.service"; export function createApprovalHook(input: { userId: string; approvalRunId: string; approvalRuntime: ApprovalRuntimeService; }) { return createHook({ async onToolCall({ toolName, args, tool }) { if (toolName !== "refund_order") { return tool.run(); } const approved = await input.approvalRuntime.waitForDecision({ userId: input.userId, approvalRunId: input.approvalRunId, toolName, args, }); return approved ? tool.run() : tool.skip("Refund was not approved."); }, }); } ``` `ApprovalRuntimeService` is your own Nest provider. It is not exported by Anvia. ## 3. Create The Approval Runtime Service [#3-create-the-approval-runtime-service] ```ts import { Injectable } from "@nestjs/common"; type ApprovalRequest = { userId: string; approvalRunId: string; toolName: string; args: string; }; type ApprovalDecision = { approved: boolean; reason?: string; }; @Injectable() export class ApprovalRuntimeService { private readonly waiters = new Map void>(); async waitForDecision(request: ApprovalRequest): Promise { const approval = await this.approvals.create({ ...request, status: "pending", }); await this.notifications.notifyReviewers({ approvalId: approval.id }); const decision = await new Promise((resolve) => { this.waiters.set(approval.id, resolve); }); this.waiters.delete(approval.id); return decision.approved; } async decide(input: { approvalId: string; approved: boolean; reason?: string }) { await this.approvals.updateDecision(input); this.waiters.get(input.approvalId)?.({ approved: input.approved, reason: input.reason, }); } } ``` The `Map` is only a single-process waiter. Use durable storage plus queue, pub/sub, websocket, or polling workers for production. ## 4. Add A Decision Controller [#4-add-a-decision-controller] ```ts import { Body, Controller, Param, Post } from "@nestjs/common"; import { z } from "zod"; import { ApprovalRuntimeService } from "./approval-runtime.service"; const DecisionRequest = z.object({ approved: z.boolean(), reason: z.string().optional(), }); @Controller("api/approvals") export class ApprovalsController { constructor(private readonly approvalRuntime: ApprovalRuntimeService) {} @Post(":id/decision") async decide(@Param("id") approvalId: string, @Body() body: unknown) { const decision = DecisionRequest.parse(body); await this.approvalRuntime.decide({ approvalId, ...decision }); return { ok: true }; } } ``` ## Next [#next] Add NestJS tests in [Setup Tests](/docs/frameworks/nestjs/10-setup-tests). Core concepts: [Human in the Loop](/docs/guides/human-in-the-loop), [Approval by Hooks](/docs/guides/human-in-the-loop/tool-approval), and [Approval Runtimes](/docs/guides/human-in-the-loop/approval-handlers). # 10 Setup Tests (/docs/frameworks/nestjs/10-setup-tests) Use Nest testing modules for controllers and services. Mock Anvia services in controller tests and provider clients in integration tests. ## 1. Install Test Tools [#1-install-test-tools] Nest projects usually include Jest. If you use Vitest, install the equivalent Nest test setup for your project. ```sh pnpm add -D @nestjs/testing supertest @types/supertest ``` ## 2. Test The Controller [#2-test-the-controller] ```ts import { Test } from "@nestjs/testing"; import request from "supertest"; import { SupportController } from "../src/support/support.controller"; import { SupportAgentService } from "../src/ai/support-agent.service"; describe("SupportController", () => { it("returns the agent output", async () => { const moduleRef = await Test.createTestingModule({ controllers: [SupportController], providers: [ { provide: SupportAgentService, useValue: { runSupport: async () => ({ output: "Reset links expire after 30 minutes.", usage: { totalTokens: 12 }, messages: [], }), }, }, ], }).compile(); const app = moduleRef.createNestApplication(); await app.init(); await request(app.getHttpServer()) .post("/api/support") .send({ message: "How long does a reset link last?" }) .expect(201) .expect(({ body }) => { expect(body.output).toBe("Reset links expire after 30 minutes."); }); }); }); ``` Nest returns `201` for `POST` by default unless you set `@HttpCode(200)`. ## 3. Test The Stream Method [#3-test-the-stream-method] Mock `streamSupport()` with a small `ReadableStream` that emits one `final` event. Assert the controller sets `application/x-ndjson`. ## 4. Test Studio Without A Port [#4-test-studio-without-a-port] ```ts import { Studio } from "@anvia/studio"; const studio = new Studio([supportAgent]); const response = await studio.fetch( new Request("http://studio.test/agents/support/runs", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ message: "Hello" }), }), ); expect(response.status).toBe(200); ``` ## Next [#next] Related guides: [Testing](/docs/guides/testing), [Tools and Pipelines](/docs/guides/testing/tools-and-pipelines), and [Studio and Providers](/docs/guides/testing/studio-and-providers). # 01 Prep (/docs/frameworks/nextjs/01-prep) Use this path when Anvia will run behind Next.js App Router route handlers. ## 1. Create or Open an App Router Project [#1-create-or-open-an-app-router-project] ```sh pnpm create next-app@latest anvia-next --ts --app cd anvia-next ``` If you already have a Next.js app, use the App Router `app/` directory. The route examples in this guide use `app/api/.../route.ts`. ## 2. Install Anvia [#2-install-anvia] ```sh pnpm add @anvia/core @anvia/openai zod ``` Install other provider packages when you need them: ```sh pnpm add @anvia/anthropic @anvia/gemini @anvia/mistral ``` ## 3. Add Environment Variables [#3-add-environment-variables] Anvia clients use explicit constructor options and do not read environment variables by themselves. ```txt OPENAI_API_KEY=sk_... ``` Read the value only in server-side files: ```ts const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) { throw new Error("OPENAI_API_KEY is required"); } ``` ## 4. Choose Runtime Boundaries [#4-choose-runtime-boundaries] Keep these files server-only: | File | Purpose | | --------------------------------- | ------------------------------------------------- | | `app/ai/support-agent.ts` | Provider client, model, tools, and reusable agent | | `app/api/support/route.ts` | Non-streaming prompt endpoint | | `app/api/support/stream/route.ts` | Streaming prompt endpoint | Next.js route handlers use the Web `Request` and `Response` APIs, so Anvia `readableStream()` can be returned directly. ## Next [#next] Build the reusable agent in [Setup Anvia](/docs/frameworks/nextjs/02-setup-anvia). For the SDK model, read [How Anvia Works](/docs/guides/sdk-fundamentals/runtime-boundaries). # 02 Setup Anvia (/docs/frameworks/nextjs/02-setup-anvia) Create provider clients, models, and ordinary tools outside the route handler when their configuration is shared by every request. ## 1. Create `app/ai/support-agent.ts` [#1-create-appaisupport-agentts] ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) { throw new Error("OPENAI_API_KEY is required"); } const client = new OpenAIClient({ apiKey }); export const model = client.completionModel("gpt-5.5"); const lookupPolicy = createTool({ name: "lookup_policy", description: "Look up a short support policy by key.", input: z.object({ key: z.enum(["password_reset", "priority_support"]), }), output: z.object({ text: z.string(), }), async execute({ key }) { const policies = { password_reset: "Password reset links expire after 30 minutes.", priority_support: "Enterprise customers receive priority support.", }; return { text: policies[key] }; }, }); export const supportAgent = new AgentBuilder("support", model) .name("Support Agent") .description("Answers support questions from the product app.") .instructions("Answer clearly. Use tools when a policy detail is needed.") .tool(lookupPolicy) .defaultMaxTurns(3) .build(); ``` ## 2. Keep The Agent Server-Side [#2-keep-the-agent-server-side] Import `supportAgent` from route handlers, server actions, background jobs, or tests. Do not import it into client components. ## 3. Add More Providers Later [#3-add-more-providers-later] The route code does not change when you swap the provider module: ```ts import { AnthropicClient } from "@anvia/anthropic"; const client = new AnthropicClient({ apiKey }); const model = client.completionModel("claude-opus-4-6"); ``` ## Next [#next] Expose the agent through a JSON endpoint in [Route Handler](/docs/frameworks/nextjs/03-route-handler). For deeper agent configuration, read [Creating Agents](/docs/guides/agents/creating-agents) and [Tools](/docs/guides/tools/creating-tools). # 03 Route Handler (/docs/frameworks/nextjs/03-route-handler) Use a route handler when your UI or another service should call the agent over HTTP. ## 1. Create `app/api/support/route.ts` [#1-create-appapisupportroutets] ```ts import { supportAgent } from "@/app/ai/support-agent"; export const runtime = "nodejs"; type SupportRequest = { message?: string; }; export async function POST(request: Request): Promise { const body = (await request.json()) as SupportRequest; const message = body.message?.trim(); if (!message) { return Response.json( { error: { code: "bad_request", message: "message is required" } }, { status: 400 }, ); } const response = await supportAgent.prompt(message).send(); return Response.json({ output: response.output, usage: response.usage, messages: response.messages, }); } ``` ## 2. Call The Route [#2-call-the-route] ```ts const response = await fetch("/api/support", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ message: "How long does a password reset link last?", }), }); const data = await response.json(); console.log(data.output); ``` ## 3. Keep Errors Structured [#3-keep-errors-structured] Validate the request before calling the model. Provider and tool failures should return an application-owned error shape, not raw stack traces. ```ts try { const response = await supportAgent.prompt(message).send(); return Response.json({ output: response.output }); } catch (error) { console.error(error); return Response.json( { error: { code: "agent_failed", message: "The agent run failed." } }, { status: 500 }, ); } ``` ## Next [#next] Return live run events in [Streaming](/docs/frameworks/nextjs/04-streaming). For response fields, read [Prompt Responses](/docs/guides/sdk-fundamentals/prompt-responses). # 04 Streaming (/docs/frameworks/nextjs/04-streaming) Anvia exposes a Web `ReadableStream`, so Next.js route handlers can return it directly. ## 1. Create `app/api/support/stream/route.ts` [#1-create-appapisupportstreamroutets] ```ts import { supportAgent } from "@/app/ai/support-agent"; export const runtime = "nodejs"; type SupportStreamRequest = { message?: string; }; export async function POST(request: Request): Promise { const body = (await request.json()) as SupportStreamRequest; const message = body.message?.trim(); if (!message) { return Response.json( { error: { code: "bad_request", message: "message is required" } }, { status: 400 }, ); } return new Response(supportAgent.prompt(message).readableStream(), { headers: { "Content-Type": "application/x-ndjson", "Cache-Control": "no-cache", }, }); } ``` ## 2. Consume Events [#2-consume-events] ```ts const response = await fetch("/api/support/stream", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ message: "Draft a refund reply." }), }); const reader = response.body?.getReader(); const decoder = new TextDecoder(); while (reader) { const next = await reader.read(); if (next.done) break; for (const line of decoder.decode(next.value).split("\n")) { if (line.trim()) console.log(JSON.parse(line)); } } ``` ## 3. Handle Terminal Events [#3-handle-terminal-events] The stream ends with either a `final` event or an `error` event. Store final output only after you receive the terminal event. ## Next [#next] Add request-local authorization and retrieval in [Tools and Context](/docs/frameworks/nextjs/05-tools-and-context). For stream details, read [Readable Streams](/docs/guides/streaming/readable-streams) and [Streaming Events](/docs/guides/streaming/streaming-events). # 05 Tools and Context (/docs/frameworks/nextjs/05-tools-and-context) If a tool needs the current user, build a request-scoped tool or agent factory. Keep provider clients and models reusable. ## 1. Create a Scoped Agent Factory [#1-create-a-scoped-agent-factory] ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { z } from "zod"; import { model } from "@/app/ai/support-agent"; import { orders } from "@/app/db/orders"; type SupportScope = { userId: string; }; export function createSupportAgent(scope: SupportScope) { const lookupOrder = createTool({ name: "lookup_order", description: "Look up one order owned by the current user.", input: z.object({ orderId: z.string(), }), output: z.object({ status: z.string(), }), async execute({ orderId }) { return orders.findForUser(scope.userId, orderId); }, }); return new AgentBuilder("support", model) .instructions("Use tools for account-specific data.") .tool(lookupOrder) .defaultMaxTurns(3) .build(); } ``` ## 2. Resolve Auth In The Route [#2-resolve-auth-in-the-route] ```ts import { createSupportAgent } from "@/app/ai/create-support-agent"; import { requireUser } from "@/app/auth"; export async function POST(request: Request): Promise { const user = await requireUser(request); const { message } = (await request.json()) as { message?: string }; if (!message?.trim()) { return Response.json({ error: "message is required" }, { status: 400 }); } const agent = createSupportAgent({ userId: user.id }); const response = await agent.prompt(message).send(); return Response.json({ output: response.output }); } ``` ## 3. Add Retrieval Context [#3-add-retrieval-context] ```ts const agent = new AgentBuilder("support", model) .instructions("Use retrieved support docs when relevant.") .dynamicContext(supportDocsIndex, { topK: 3, threshold: 0.7, }) .tool(lookupOrder) .build(); ``` Build the retrieval index outside hot request paths. Use request-scoped tools for permissions and retrieval context for searchable knowledge. ## Next [#next] Persist conversations in [Persistence](/docs/frameworks/nextjs/06-persistence). Related guides: [Runtime Context](/docs/guides/agents/runtime-context), [RAG Context](/docs/guides/retrieval/rag-context), and [Tool Handlers](/docs/guides/tools/tool-handlers). # 06 Persistence (/docs/frameworks/nextjs/06-persistence) Anvia gives you two practical persistence paths. ## 1. Store Explicit History [#1-store-explicit-history] Use this when your app already owns chat transcripts. ```ts import { Message } from "@anvia/core"; import { supportAgent } from "@/app/ai/support-agent"; import { conversations } from "@/app/db/conversations"; export async function POST(request: Request): Promise { const { conversationId, message } = (await request.json()) as { conversationId: string; message: string; }; const history = await conversations.loadMessages(conversationId); const response = await supportAgent .prompt([...history, Message.user(message)]) .send(); await conversations.saveMessages(conversationId, [ ...history, ...response.messages, ]); return Response.json({ output: response.output }); } ``` ## 2. Use Agent Memory [#2-use-agent-memory] Use this when Anvia should load and append messages through your memory store. ```ts const agent = new AgentBuilder("support", model) .memory(memoryStore, { savePolicy: "message" }) .build(); const response = await agent .session(conversationId, { userId }) .prompt(message) .send(); ``` ## 3. Keep Runtime Events Separate [#3-keep-runtime-events-separate] Memory stores model transcript messages for future prompts. If your UI needs replayable stream events, also use an [Event Store](/docs/guides/agents/event-store). ## Next [#next] Review production constraints in [Deploy](/docs/frameworks/nextjs/07-deploy). For memory adapters, read [Memory](/docs/guides/memory), [Prisma](/docs/guides/memory/prisma), and [Drizzle](/docs/guides/memory/drizzle). # 07 Deploy (/docs/frameworks/nextjs/07-deploy) ## Runtime Checklist [#runtime-checklist] | Area | Check | | --------- | ------------------------------------------------------------------------------------- | | Runtime | Use `export const runtime = "nodejs"` unless every dependency supports edge execution | | Secrets | Set provider keys in the deployment environment, not in client bundles | | Streaming | Disable buffering in proxies that sit in front of streaming routes | | Timeouts | Keep route timeouts above your longest expected model/tool run | | Storage | Use durable storage for history, memory, traces, and retrieval indexes | ## Provider Clients [#provider-clients] Create clients and models in server modules: ```ts const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) { throw new Error("OPENAI_API_KEY is required"); } const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); ``` Do not expose provider keys to browser code. ## Observability [#observability] Attach observers when you need logs, traces, or metrics: ```ts const agent = new AgentBuilder("support", model) .observe(observer) .build(); ``` Use `.withTrace(...)` per request for route, user, and session metadata. ## Deployment Smoke Test [#deployment-smoke-test] ```sh curl -X POST "$APP_URL/api/support" \ -H "Content-Type: application/json" \ -d '{"message":"Say hello"}' ``` ## Next [#next] Use [Troubleshooting](/docs/frameworks/nextjs/08-troubleshooting) when a deployed route behaves differently from local development. Related guides: [Observers](/docs/guides/observability/observers) and [Errors](/docs/guides/sdk-fundamentals/errors). # 08 Troubleshooting (/docs/frameworks/nextjs/08-troubleshooting) ## `OPENAI_API_KEY is required` [#openai_api_key-is-required] The provider client was created before the environment variable was available. ```ts const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) throw new Error("OPENAI_API_KEY is required"); ``` Set the variable in `.env.local` for development and in your deployment environment for production. ## `message is required` [#message-is-required] The route expects JSON with a `message` field. ```sh curl -X POST http://localhost:3000/api/support \ -H "Content-Type: application/json" \ -d '{"message":"Hello"}' ``` ## Client Bundle Contains Server Code [#client-bundle-contains-server-code] The agent module was imported by a client component. Import it only from route handlers, server functions, server actions, or tests. ## Streaming Works Locally But Not In Production [#streaming-works-locally-but-not-in-production] Check proxy buffering, function timeouts, and response headers: ```ts return new Response(agent.prompt(message).readableStream(), { headers: { "Content-Type": "application/x-ndjson", "Cache-Control": "no-cache", }, }); ``` ## Tool Can Read The Wrong User [#tool-can-read-the-wrong-user] Do not rely on model-supplied user ids for authorization. Resolve the user in the route and close over that user in scoped tools. ## Provider Or Tool Fails Mid-Run [#provider-or-tool-fails-mid-run] Wrap non-streaming routes in `try/catch`. For streaming routes, handle terminal `error` events on the client. ## Next [#next] Revisit [Tools and Context](/docs/frameworks/nextjs/05-tools-and-context), [Readable Streams](/docs/guides/streaming/readable-streams), and [Tool Errors](/docs/guides/tools/tool-errors). # 09 Human in the Loop (/docs/frameworks/nextjs/09-human-in-the-loop) Human-in-the-loop work can live in Studio during development or in your own Next.js routes when production users need to approve actions. ## 1. Use Studio For Local Approval UI [#1-use-studio-for-local-approval-ui] Add approval metadata to side-effect tools: ```ts const refundOrder = createTool({ name: "refund_order", description: "Issue a refund.", input: z.object({ orderId: z.string(), amount: z.number().positive(), }), approval: { when: ({ args }) => args.amount > 100, reason: ({ args }) => `Review refund of $${args.amount} for ${args.orderId}.`, rejectMessage: "Refund was not approved.", }, async execute({ orderId, amount }) { return refunds.create({ orderId, amount }); }, }); ``` Run the same built agent in Studio from a server-only script: ```ts import { Studio } from "@anvia/studio"; import { supportAgent } from "@/app/ai/support-agent"; new Studio([supportAgent]).start({ port: 4021 }); ``` Studio reads the approval metadata and waits for a reviewer before running protected tools. ## 2. Use A Request Hook For Product Approval [#2-use-a-request-hook-for-product-approval] Use a hook when your app owns the approval table, reviewer UI, notification, or timeout. ```ts import { createHook } from "@anvia/core"; import { approvalRuntime } from "@/app/approvals/runtime"; function createApprovalHook(input: { userId: string; conversationId: string }) { return createHook({ async onToolCall({ toolName, args, tool }) { if (toolName !== "refund_order") { return tool.run(); } const approved = await approvalRuntime.waitForDecision({ userId: input.userId, conversationId: input.conversationId, toolName, args, }); return approved ? tool.run() : tool.skip("Refund was not approved."); }, }); } ``` `approvalRuntime` is your own module. It is not imported from `@anvia/core` or any Anvia package. Attach the hook in the route: ```ts const response = await supportAgent .prompt(message) .requestHook(createApprovalHook({ userId, conversationId })) .send(); ``` ## 3. Create The Approval Runtime [#3-create-the-approval-runtime] `approvalRuntime` is application code, not an Anvia export. It is the small runtime that creates a pending approval record, notifies reviewers, waits until one of your routes or jobs resolves it, then returns the decision to the hook. ```ts type ApprovalRequest = { userId: string; conversationId: string; toolName: string; args: string; }; type ApprovalDecision = { approved: boolean; reason?: string; }; export function createApprovalRuntime() { const waiters = new Map void>(); return { async waitForDecision(request: ApprovalRequest): Promise { const approval = await db.approval.create({ data: { userId: request.userId, conversationId: request.conversationId, toolName: request.toolName, args: request.args, status: "pending", }, }); await notifyReviewers({ approvalId: approval.id }); const decision = await new Promise((resolve) => { waiters.set(approval.id, resolve); }); waiters.delete(approval.id); return decision.approved; }, async decide(input: { approvalId: string; reviewerId: string; approved: boolean; reason?: string; }): Promise { await db.approval.update({ where: { id: input.approvalId }, data: { status: input.approved ? "approved" : "rejected", reviewerId: input.reviewerId, decisionReason: input.reason, resolvedAt: new Date(), }, }); waiters.get(input.approvalId)?.({ approved: input.approved, reason: input.reason, }); }, }; } export const approvalRuntime = createApprovalRuntime(); ``` The `Map` is only the waiting mechanism for a single Node process. In production, keep approval records in durable storage and use your app's realtime channel, queue, pub/sub system, or polling worker to resolve waiters. ## 4. Keep Approval State In Your App [#4-keep-approval-state-in-your-app] | State | Owner | | ----------------------------- | -------------------------- | | Pending approval record | Your database | | Reviewer authorization | Your app | | Notification or websocket | Your app | | Tool execution after approval | Anvia via `tool.run()` | | Rejection message to model | Anvia via `tool.skip(...)` | ## Next [#next] Add tests in [Setup Tests](/docs/frameworks/nextjs/10-setup-tests). Core concepts: [Human in the Loop](/docs/guides/human-in-the-loop), [Approval by Hooks](/docs/guides/human-in-the-loop/tool-approval), [Approval Runtimes](/docs/guides/human-in-the-loop/approval-handlers), and [Studio Tool Approvals](/docs/studio/human-in-the-loop/tool-approvals). # 10 Setup Tests (/docs/frameworks/nextjs/10-setup-tests) Use tests to cover application routing and validation without turning every branch into a provider call. ## 1. Install Test Tools [#1-install-test-tools] ```sh pnpm add -D vitest ``` ## 2. Test The JSON Route [#2-test-the-json-route] Next.js route handlers are normal exported functions. ```ts import { describe, expect, it, vi } from "vitest"; import { POST } from "@/app/api/support/route"; vi.mock("@/app/ai/support-agent", () => ({ supportAgent: { prompt: () => ({ send: async () => ({ output: "Reset links expire after 30 minutes.", usage: { totalTokens: 12 }, messages: [], }), }), }, })); describe("POST /api/support", () => { it("returns the agent output", async () => { const response = await POST( new Request("http://test.local/api/support", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ message: "How long does a reset link last?" }), }), ); expect(response.status).toBe(200); await expect(response.json()).resolves.toMatchObject({ output: "Reset links expire after 30 minutes.", }); }); }); ``` ## 3. Test The Streaming Route [#3-test-the-streaming-route] ```ts const response = await POST( new Request("http://test.local/api/support/stream", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ message: "Hello" }), }), ); expect(response.headers.get("content-type")).toContain("application/x-ndjson"); ``` Mock `readableStream()` with a small `ReadableStream` that emits one `final` event. ## 4. Test Studio Without a Port [#4-test-studio-without-a-port] ```ts import { Studio } from "@anvia/studio"; import { supportAgent } from "@/app/ai/support-agent"; const studio = new Studio([supportAgent]); const response = await studio.fetch( new Request("http://studio.test/agents/support/runs", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ message: "Hello" }), }), ); expect(response.status).toBe(200); ``` ## Next [#next] Related guides: [Testing](/docs/guides/testing), [Tools and Pipelines](/docs/guides/testing/tools-and-pipelines), and [Studio and Providers](/docs/guides/testing/studio-and-providers). # 01 Prep (/docs/frameworks/sveltekit/01-prep) Use this path when Anvia runs behind SvelteKit endpoints, form actions, or server-only modules. ## 1. Create A SvelteKit Project [#1-create-a-sveltekit-project] ```sh pnpm create svelte@latest anvia-sveltekit cd anvia-sveltekit pnpm install ``` Choose TypeScript. Anvia code belongs in server-only files, not client components. ## 2. Install Anvia [#2-install-anvia] ```sh pnpm add @anvia/core @anvia/openai zod ``` Install other providers when needed: ```sh pnpm add @anvia/anthropic @anvia/gemini @anvia/mistral ``` ## 3. Add Environment Variables [#3-add-environment-variables] ```txt OPENAI_API_KEY=sk_... ``` Read secrets from `$env/static/private` inside server modules: ```ts import { OPENAI_API_KEY } from "$env/static/private"; if (!OPENAI_API_KEY) { throw new Error("OPENAI_API_KEY is required"); } ``` ## 4. Choose File Boundaries [#4-choose-file-boundaries] | File | Purpose | | ------------------------------------------ | ------------------------------------------------- | | `src/lib/server/ai/support-agent.ts` | Provider client, model, tools, and reusable agent | | `src/routes/api/support/+server.ts` | JSON endpoint | | `src/routes/api/support/stream/+server.ts` | NDJSON stream endpoint | | `src/hooks.server.ts` | Auth and request-local `locals` | ## Next [#next] Build the reusable agent in [Setup Anvia](/docs/frameworks/sveltekit/02-setup-anvia). Read [Runtime Boundaries](/docs/guides/sdk-fundamentals/runtime-boundaries) before importing Anvia into Svelte components. # 02 Setup Anvia (/docs/frameworks/sveltekit/02-setup-anvia) Create provider clients and shared agents in `src/lib/server`. SvelteKit keeps this code out of browser bundles. ## 1. Create `src/lib/server/ai/support-agent.ts` [#1-create-srclibserveraisupport-agentts] ```ts import { OPENAI_API_KEY } from "$env/static/private"; import { AgentBuilder, createTool } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; if (!OPENAI_API_KEY) { throw new Error("OPENAI_API_KEY is required"); } const client = new OpenAIClient({ apiKey: OPENAI_API_KEY }); export const model = client.completionModel("gpt-5.5"); const lookupPolicy = createTool({ name: "lookup_policy", description: "Look up a support policy by key.", input: z.object({ key: z.enum(["password_reset", "priority_support"]), }), output: z.object({ text: z.string(), }), async execute({ key }) { const policies = { password_reset: "Password reset links expire after 30 minutes.", priority_support: "Enterprise customers receive priority support.", }; return { text: policies[key] }; }, }); export const supportAgent = new AgentBuilder("support", model) .name("Support Agent") .instructions("Answer clearly. Use tools when policy detail is needed.") .tool(lookupPolicy) .defaultMaxTurns(3) .build(); ``` ## 2. Keep Agents Server-Side [#2-keep-agents-server-side] Import `supportAgent` only from `+server.ts`, server actions, hooks, jobs, or tests. Do not import it from `+page.svelte`. ## 3. Swap Providers Without Rewriting Routes [#3-swap-providers-without-rewriting-routes] ```ts import { AnthropicClient } from "@anvia/anthropic"; const client = new AnthropicClient({ apiKey: process.env.ANTHROPIC_API_KEY }); const model = client.completionModel("claude-opus-4-6"); ``` ## Next [#next] Expose the agent in [Route Handler](/docs/frameworks/sveltekit/03-route-handler). Related guides: [Creating Agents](/docs/guides/agents/creating-agents) and [Tools](/docs/guides/tools/creating-tools). # 03 Route Handler (/docs/frameworks/sveltekit/03-route-handler) SvelteKit endpoint modules export HTTP method functions. Use them to keep validation and agent calls on the server. ## 1. Create `src/routes/api/support/+server.ts` [#1-create-srcroutesapisupportserverts] ```ts import { json, type RequestHandler } from "@sveltejs/kit"; import { z } from "zod"; import { supportAgent } from "$lib/server/ai/support-agent"; const SupportRequest = z.object({ message: z.string().trim().min(1, "message is required"), }); export const POST: RequestHandler = async ({ request }) => { const parsed = SupportRequest.safeParse(await request.json()); if (!parsed.success) { return json( { error: { code: "bad_request", message: parsed.error.issues[0]?.message } }, { status: 400 }, ); } const response = await supportAgent.prompt(parsed.data.message).send(); return json({ output: response.output, usage: response.usage, messages: response.messages, }); }; ``` ## 2. Call The Endpoint [#2-call-the-endpoint] ```ts const response = await fetch("/api/support", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ message: "How long does a reset link last?" }), }); const data = await response.json(); console.log(data.output); ``` ## 3. Keep Errors Application-Owned [#3-keep-errors-application-owned] Provider and tool errors should be logged server-side and returned as a stable app error shape. ```ts try { const response = await supportAgent.prompt(parsed.data.message).send(); return json({ output: response.output }); } catch (error) { console.error(error); return json( { error: { code: "agent_failed", message: "The agent run failed." } }, { status: 500 }, ); } ``` ## Next [#next] Return live run events in [Streaming](/docs/frameworks/sveltekit/04-streaming). For response fields, read [Prompt Responses](/docs/guides/sdk-fundamentals/prompt-responses). # 04 Streaming (/docs/frameworks/sveltekit/04-streaming) SvelteKit endpoints can return standard `Response` objects, so Anvia streams can be returned directly. ## 1. Create `src/routes/api/support/stream/+server.ts` [#1-create-srcroutesapisupportstreamserverts] ```ts import type { RequestHandler } from "@sveltejs/kit"; import { z } from "zod"; import { supportAgent } from "$lib/server/ai/support-agent"; const SupportStreamRequest = z.object({ message: z.string().trim().min(1, "message is required"), }); export const POST: RequestHandler = async ({ request }) => { const parsed = SupportStreamRequest.safeParse(await request.json()); if (!parsed.success) { return Response.json( { error: { code: "bad_request", message: parsed.error.issues[0]?.message } }, { status: 400 }, ); } return new Response(supportAgent.prompt(parsed.data.message).readableStream(), { headers: { "Content-Type": "application/x-ndjson", "Cache-Control": "no-cache", }, }); }; ``` ## 2. Consume The Stream [#2-consume-the-stream] ```ts const response = await fetch("/api/support/stream", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ message: "Draft a reply." }), }); const reader = response.body?.getReader(); const decoder = new TextDecoder(); while (reader) { const next = await reader.read(); if (next.done) break; for (const line of decoder.decode(next.value).split("\n")) { if (line.trim()) console.log(JSON.parse(line)); } } ``` ## 3. Runtime Notes [#3-runtime-notes] Use a Node-compatible adapter when your provider client or retrieval package depends on Node APIs. Edge adapters need separate verification for each provider and integration. ## Next [#next] Add auth, tools, and retrieval in [Tools and Context](/docs/frameworks/sveltekit/05-tools-and-context). Related guides: [Readable Streams](/docs/guides/streaming/readable-streams) and [Streaming Events](/docs/guides/streaming/streaming-events). # 05 Tools and Context (/docs/frameworks/sveltekit/05-tools-and-context) Your app should own auth, database access, and retrieval. Pass only the request-local values a run needs. ## 1. Add `locals` In `hooks.server.ts` [#1-add-locals-in-hooksserverts] ```ts import type { Handle } from "@sveltejs/kit"; export const handle: Handle = async ({ event, resolve }) => { const session = await auth.getSession(event.request); event.locals.userId = session?.user.id; return resolve(event); }; ``` Add local types in `src/app.d.ts`: ```ts declare global { namespace App { interface Locals { userId?: string; } } } ``` ## 2. Build Request-Local Tools [#2-build-request-local-tools] ```ts import { createTool } from "@anvia/core"; import { z } from "zod"; export function createAccountTool(input: { userId: string }) { return createTool({ name: "get_account_status", description: "Read the authenticated user's account status.", input: z.object({}), output: z.object({ plan: z.string(), openTickets: z.number() }), async execute() { return db.account.findStatus({ userId: input.userId }); }, }); } ``` ## 3. Attach Context In The Endpoint [#3-attach-context-in-the-endpoint] ```ts import { json, type RequestHandler } from "@sveltejs/kit"; import { supportAgent } from "$lib/server/ai/support-agent"; import { createAccountTool } from "$lib/server/ai/tools"; export const POST: RequestHandler = async ({ request, locals }) => { if (!locals.userId) { return json({ error: { code: "unauthorized" } }, { status: 401 }); } const { message } = await request.json(); const response = await supportAgent .prompt(message) .tool(createAccountTool({ userId: locals.userId })) .context({ userId: locals.userId }) .send(); return json({ output: response.output }); }; ``` ## 4. Add Retrieval Context [#4-add-retrieval-context] Retrieve documents in application code, then pass them into the prompt or context. Keep vector-store credentials in server modules. ```ts const documents = await knowledge.search({ query: message, filter: { userId: locals.userId }, limit: 5, }); const response = await supportAgent.prompt(message).documents(documents).send(); ``` ## Next [#next] Persist history in [Persistence](/docs/frameworks/sveltekit/06-persistence). Related guides: [Runtime Context](/docs/guides/agents/runtime-context), [Tool Handlers](/docs/guides/tools/tool-handlers), and [RAG Context](/docs/guides/retrieval/rag-context). # 06 Persistence (/docs/frameworks/sveltekit/06-persistence) Anvia returns the new messages from each run. Your SvelteKit app decides where sessions and history live. ## 1. Load Existing Messages [#1-load-existing-messages] ```ts const session = await db.chatSession.findUnique({ where: { id: sessionId, userId: locals.userId }, include: { messages: { orderBy: { createdAt: "asc" } } }, }); if (!session) { return json({ error: { code: "not_found" } }, { status: 404 }); } ``` ## 2. Send With History [#2-send-with-history] ```ts const response = await supportAgent .prompt(message) .messages(session.messages.map((item) => item.message)) .send(); ``` ## 3. Store New Messages [#3-store-new-messages] ```ts await db.chatMessage.createMany({ data: response.messages.map((message) => ({ sessionId, message, })), }); ``` Only store messages after the run succeeds. If the run fails, record an application event instead of adding partial history. ## 4. Use Memory When The Model Should Remember [#4-use-memory-when-the-model-should-remember] Use app storage for conversation history. Use Anvia memory when the model should recall durable facts in future sessions. ## Next [#next] Prepare runtime constraints in [Deploy](/docs/frameworks/sveltekit/07-deploy). Related guides: [Memory](/docs/guides/memory), [Memory and Sessions](/docs/guides/sdk-fundamentals/memory-and-sessions), and [Agent History](/docs/guides/agents/agent-history). # 07 Deploy (/docs/frameworks/sveltekit/07-deploy) Deployment depends on your SvelteKit adapter. Verify provider clients, vector stores, and observability packages against that runtime. ## 1. Prefer Node For First Deploys [#1-prefer-node-for-first-deploys] Use a Node-compatible adapter when you need filesystem loaders, local embeddings, provider SDKs with Node dependencies, or long-running streams. ```sh pnpm add -D @sveltejs/adapter-node ``` ## 2. Configure Environment Variables [#2-configure-environment-variables] Set secrets in your deployment platform: ```txt OPENAI_API_KEY=sk_... ANVIA_STUDIO_TOKEN=... DATABASE_URL=... ``` Do not expose provider keys through public env prefixes. ## 3. Streaming Checks [#3-streaming-checks] Confirm your host supports unbuffered responses for `application/x-ndjson`. Some serverless platforms buffer responses unless streaming is explicitly enabled. ## 4. Production Checklist [#4-production-checklist] | Check | Why | | -------------------------- | ------------------------------------------------------ | | Node runtime verified | Provider and retrieval packages may require Node APIs | | Request timeout configured | Agent runs can be longer than simple CRUD requests | | Secrets scoped server-side | Provider keys must never reach the browser | | Error logs connected | Provider and tool failures need operational visibility | ## Next [#next] Debug common failures in [Troubleshooting](/docs/frameworks/sveltekit/08-troubleshooting). Add telemetry with [Observability](/docs/guides/observability/tracing). # 08 Troubleshooting (/docs/frameworks/sveltekit/08-troubleshooting) Most failures come from importing server code into the browser, missing env vars, or buffered streams. ## `OPENAI_API_KEY is required` [#openai_api_key-is-required] Use `$env/static/private` in server-only modules. Do not read provider keys in `+page.svelte` or `+page.ts`. ```ts import { OPENAI_API_KEY } from "$env/static/private"; ``` ## `Cannot import server-only module into client code` [#cannot-import-server-only-module-into-client-code] Move Anvia imports into `src/lib/server/...` and call them from `+server.ts`. ## Route Body Parsing Fails [#route-body-parsing-fails] Validate unknown JSON before calling the agent: ```ts const parsed = SupportRequest.safeParse(await request.json()); if (!parsed.success) { return Response.json({ error: { code: "bad_request" } }, { status: 400 }); } ``` ## Stream Does Not Flush [#stream-does-not-flush] Check the adapter and hosting platform. Return a `Response` with `application/x-ndjson` and avoid wrapping the stream in `json(...)`. ```ts return new Response(agent.prompt(message).readableStream(), { headers: { "Content-Type": "application/x-ndjson" }, }); ``` ## Provider Failures [#provider-failures] Catch provider errors at the route boundary, log the internal detail, and return a stable application error. ## Next [#next] Add reviewer workflows in [Human in the Loop](/docs/frameworks/sveltekit/09-human-in-the-loop). Related guides: [Tool Errors](/docs/guides/tools/tool-errors), [Readable Streams](/docs/guides/streaming/readable-streams), and [Tracing](/docs/guides/observability/tracing). # 09 Human in the Loop (/docs/frameworks/sveltekit/09-human-in-the-loop) SvelteKit can expose both the agent endpoint and reviewer decision endpoints. Anvia provides hooks; your app provides the approval runtime. ## 1. Use Studio During Development [#1-use-studio-during-development] ```ts import { Studio } from "@anvia/studio"; import { supportAgent } from "$lib/server/ai/support-agent"; new Studio([supportAgent]).start({ port: 4021 }); ``` Studio is useful locally. Production approval storage, reviewer permissions, and notifications belong to your app. ## 2. Create A Hook [#2-create-a-hook] ```ts import { createHook } from "@anvia/core"; import { approvalRuntime } from "$lib/server/approvals/runtime"; export function createApprovalHook(input: { userId: string; approvalRunId: string }) { return createHook({ async onToolCall({ toolName, args, tool }) { if (toolName !== "refund_order") { return tool.run(); } const approved = await approvalRuntime.waitForDecision({ userId: input.userId, approvalRunId: input.approvalRunId, toolName, args, }); return approved ? tool.run() : tool.skip("Refund was not approved."); }, }); } ``` `approvalRuntime` is not an Anvia API. Create it next to your database, queue, notification, and reviewer UI code. ## 3. Create The Approval Runtime [#3-create-the-approval-runtime] ```ts type ApprovalRequest = { userId: string; approvalRunId: string; toolName: string; args: string; }; type ApprovalDecision = { approved: boolean; reason?: string; }; export function createApprovalRuntime() { const waiters = new Map void>(); return { async waitForDecision(request: ApprovalRequest): Promise { const approval = await db.approval.create({ data: { ...request, status: "pending" }, }); await notifyReviewers({ approvalId: approval.id }); const decision = await new Promise((resolve) => { waiters.set(approval.id, resolve); }); waiters.delete(approval.id); return decision.approved; }, async decide(input: { approvalId: string; approved: boolean; reason?: string }) { await db.approval.update({ where: { id: input.approvalId }, data: { status: input.approved ? "approved" : "rejected", decisionReason: input.reason, resolvedAt: new Date(), }, }); waiters.get(input.approvalId)?.({ approved: input.approved, reason: input.reason, }); }, }; } export const approvalRuntime = createApprovalRuntime(); ``` The `Map` is only a local waiter. Use durable storage plus queue, pub/sub, websocket, or polling workers for production. ## 4. Add Reviewer Routes [#4-add-reviewer-routes] ```ts import { json, type RequestHandler } from "@sveltejs/kit"; import { z } from "zod"; import { approvalRuntime } from "$lib/server/approvals/runtime"; const DecisionRequest = z.object({ approved: z.boolean(), reason: z.string().optional(), }); export const POST: RequestHandler = async ({ params, request, locals }) => { if (!locals.userId) { return json({ error: { code: "unauthorized" } }, { status: 401 }); } const decision = DecisionRequest.parse(await request.json()); await approvalRuntime.decide({ approvalId: params.id, ...decision, }); return json({ ok: true }); }; ``` ## Next [#next] Add SvelteKit tests in [Setup Tests](/docs/frameworks/sveltekit/10-setup-tests). Core concepts: [Human in the Loop](/docs/guides/human-in-the-loop), [Approval by Hooks](/docs/guides/human-in-the-loop/tool-approval), and [Approval Runtimes](/docs/guides/human-in-the-loop/approval-handlers). # 10 Setup Tests (/docs/frameworks/sveltekit/10-setup-tests) Test endpoint behavior with mocked agents. Keep provider integration tests separate and opt-in. ## 1. Install Test Tools [#1-install-test-tools] ```sh pnpm add -D vitest ``` ## 2. Test The JSON Endpoint [#2-test-the-json-endpoint] ```ts import { describe, expect, it, vi } from "vitest"; import { POST } from "./+server"; vi.mock("$lib/server/ai/support-agent", () => ({ supportAgent: { prompt: () => ({ send: async () => ({ output: "Reset links expire after 30 minutes.", usage: { totalTokens: 12 }, messages: [], }), }), }, })); describe("POST /api/support", () => { it("returns the agent output", async () => { const response = await POST({ request: new Request("http://test.local/api/support", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ message: "How long does a reset link last?" }), }), locals: {}, params: {}, url: new URL("http://test.local/api/support"), } as never); expect(response.status).toBe(200); await expect(response.json()).resolves.toMatchObject({ output: "Reset links expire after 30 minutes.", }); }); }); ``` ## 3. Test The Stream Endpoint [#3-test-the-stream-endpoint] ```ts const stream = new ReadableStream({ start(controller) { controller.enqueue(new TextEncoder().encode('{"type":"final"}\n')); controller.close(); }, }); vi.mock("$lib/server/ai/support-agent", () => ({ supportAgent: { prompt: () => ({ readableStream: () => stream, }), }, })); ``` Assert the endpoint returns `application/x-ndjson` and a readable body. ## 4. Test Studio Without A Port [#4-test-studio-without-a-port] ```ts import { Studio } from "@anvia/studio"; import { supportAgent } from "$lib/server/ai/support-agent"; const studio = new Studio([supportAgent]); const response = await studio.fetch( new Request("http://studio.test/agents/support/runs", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ message: "Hello" }), }), ); expect(response.status).toBe(200); ``` ## Next [#next] Related guides: [Testing](/docs/guides/testing), [Tools and Pipelines](/docs/guides/testing/tools-and-pipelines), and [Studio and Providers](/docs/guides/testing/studio-and-providers). # 01 Prep (/docs/frameworks/tanstack-start/01-prep) Use this path when Anvia will run inside a TanStack Start application. ## 1. Create or Open a Start Project [#1-create-or-open-a-start-project] Create a TanStack Start React project from the current TanStack starter, or use an existing app with `@tanstack/react-start` installed. ```sh pnpm create @tanstack/start@latest anvia-start cd anvia-start ``` ## 2. Install Anvia [#2-install-anvia] ```sh pnpm add @anvia/core @anvia/openai zod ``` Install other provider packages only when you need them: ```sh pnpm add @anvia/anthropic @anvia/gemini @anvia/mistral ``` ## 3. Add Environment Variables [#3-add-environment-variables] Anvia clients require explicit configuration. ```txt OPENAI_API_KEY=sk_... ``` Read this value in server-side modules or server route handlers: ```ts const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) { throw new Error("OPENAI_API_KEY is required"); } ``` ## 4. Choose File Boundaries [#4-choose-file-boundaries] TanStack Start gives you two useful server shapes: | File | Purpose | | ----------------------------- | ------------------------------------------------------------------------- | | `src/ai/support-agent.ts` | Provider client, model, tools, and reusable agent | | `src/routes/api/support.ts` | Raw HTTP server route for JSON and streaming | | `src/ai/support.functions.ts` | Optional `createServerFn(...)` wrapper callable from routes or components | Use server routes when you need raw `Request` and `Response` control. Use server functions when you want typed RPC-style calls inside the app. ## Next [#next] Build the reusable agent in [Setup Anvia](/docs/frameworks/tanstack-start/02-setup-anvia). Read [How Anvia Works](/docs/guides/sdk-fundamentals/runtime-boundaries) for the SDK boundaries. # 02 Setup Anvia (/docs/frameworks/tanstack-start/02-setup-anvia) Create clients, models, and shared tools in a server-side module. Import this module only from server routes, server functions, loaders, or tests. ## 1. Create `src/ai/support-agent.ts` [#1-create-srcaisupport-agentts] ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) { throw new Error("OPENAI_API_KEY is required"); } const client = new OpenAIClient({ apiKey }); export const model = client.completionModel("gpt-5.5"); const lookupPolicy = createTool({ name: "lookup_policy", description: "Look up a short support policy by key.", input: z.object({ key: z.enum(["password_reset", "priority_support"]), }), output: z.object({ text: z.string(), }), async execute({ key }) { const policies = { password_reset: "Password reset links expire after 30 minutes.", priority_support: "Enterprise customers receive priority support.", }; return { text: policies[key] }; }, }); export const supportAgent = new AgentBuilder("support", model) .instructions("Answer support questions clearly. Use tools for policy facts.") .tool(lookupPolicy) .defaultMaxTurns(3) .build(); ``` ## 2. Optional Server Function Wrapper [#2-optional-server-function-wrapper] Use `createServerFn(...)` when app code wants a typed server call instead of calling a raw HTTP route. ```ts import { createServerFn } from "@tanstack/react-start"; import { z } from "zod"; import { supportAgent } from "./support-agent"; const SupportInput = z.object({ message: z.string().min(1), }); export const askSupport = createServerFn({ method: "POST" }) .inputValidator(SupportInput) .handler(async ({ data }) => { const response = await supportAgent.prompt(data.message).send(); return { output: response.output, usage: response.usage, }; }); ``` ## Next [#next] Expose the agent through a server route in [Route Handler](/docs/frameworks/tanstack-start/03-route-handler). Related guides: [Creating Agents](/docs/guides/agents/creating-agents), [Tools](/docs/guides/tools/creating-tools), and [Provider Clients](/docs/guides/sdk-fundamentals/clients-and-models). # 03 Route Handler (/docs/frameworks/tanstack-start/03-route-handler) TanStack Start server routes live in `src/routes` and can return Web `Response` objects. ## 1. Create `src/routes/api/support.ts` [#1-create-srcroutesapisupportts] ```ts import { createFileRoute } from "@tanstack/react-router"; import { supportAgent } from "~/ai/support-agent"; type SupportRequest = { message?: string; }; export const Route = createFileRoute("/api/support")({ server: { handlers: { POST: async ({ request }) => { const body = (await request.json()) as SupportRequest; const message = body.message?.trim(); if (!message) { return Response.json( { error: { code: "bad_request", message: "message is required" } }, { status: 400 }, ); } const response = await supportAgent.prompt(message).send(); return Response.json({ output: response.output, usage: response.usage, messages: response.messages, }); }, }, }, }); ``` ## 2. Call The Route [#2-call-the-route] ```ts const response = await fetch("/api/support", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ message: "Can enterprise customers use priority support?", }), }); const data = await response.json(); console.log(data.output); ``` ## 3. Use Server Functions For App-Internal Calls [#3-use-server-functions-for-app-internal-calls] If you created `askSupport`, app code can call the server function instead: ```ts const result = await askSupport({ data: { message: "How long does a reset link last?" }, }); ``` Use the raw server route for external clients, webhooks, and streaming. ## Next [#next] Return live Anvia events in [Streaming](/docs/frameworks/tanstack-start/04-streaming). For prompt response fields, read [Prompt Responses](/docs/guides/sdk-fundamentals/prompt-responses). # 04 Streaming (/docs/frameworks/tanstack-start/04-streaming) Use a server route for streaming because it gives direct access to `Response`. ## 1. Add `POST` Streaming Handler [#1-add-post-streaming-handler] ```ts import { createFileRoute } from "@tanstack/react-router"; import { supportAgent } from "~/ai/support-agent"; type SupportStreamRequest = { message?: string; }; export const Route = createFileRoute("/api/support/stream")({ server: { handlers: { POST: async ({ request }) => { const body = (await request.json()) as SupportStreamRequest; const message = body.message?.trim(); if (!message) { return Response.json( { error: { code: "bad_request", message: "message is required" } }, { status: 400 }, ); } return new Response(supportAgent.prompt(message).readableStream(), { headers: { "Content-Type": "application/x-ndjson", "Cache-Control": "no-cache", }, }); }, }, }, }); ``` ## 2. Consume Stream Lines [#2-consume-stream-lines] ```ts const response = await fetch("/api/support/stream", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ message: "Draft a support reply." }), }); const reader = response.body?.getReader(); const decoder = new TextDecoder(); while (reader) { const next = await reader.read(); if (next.done) break; for (const line of decoder.decode(next.value).split("\n")) { if (line.trim()) console.log(JSON.parse(line)); } } ``` ## 3. Persist After `final` [#3-persist-after-final] Do not save partial text deltas as conversation history. Save the final output or use Anvia memory/event storage. ## Next [#next] Add authorization and retrieval in [Tools and Context](/docs/frameworks/tanstack-start/05-tools-and-context). Related guides: [Readable Streams](/docs/guides/streaming/readable-streams) and [Streaming Events](/docs/guides/streaming/streaming-events). # 05 Tools and Context (/docs/frameworks/tanstack-start/05-tools-and-context) Resolve auth and request data in the server route or server function. If a tool needs that data, create a scoped tool or agent for the request. ## 1. Create a Scoped Agent Factory [#1-create-a-scoped-agent-factory] ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { z } from "zod"; import { model } from "~/ai/support-agent"; import { orders } from "~/db/orders"; type SupportScope = { userId: string; }; export function createSupportAgent(scope: SupportScope) { const lookupOrder = createTool({ name: "lookup_order", description: "Look up one order owned by the current user.", input: z.object({ orderId: z.string(), }), output: z.object({ status: z.string(), }), async execute({ orderId }) { return orders.findForUser(scope.userId, orderId); }, }); return new AgentBuilder("support", model) .instructions("Use tools for account-specific data.") .tool(lookupOrder) .defaultMaxTurns(3) .build(); } ``` ## 2. Use It From a Server Route [#2-use-it-from-a-server-route] ```ts import { createFileRoute } from "@tanstack/react-router"; import { requireUser } from "~/auth/server"; import { createSupportAgent } from "~/ai/create-support-agent"; export const Route = createFileRoute("/api/support")({ server: { handlers: { POST: async ({ request }) => { const user = await requireUser(request); const { message } = (await request.json()) as { message?: string }; if (!message?.trim()) { return Response.json({ error: "message is required" }, { status: 400 }); } const agent = createSupportAgent({ userId: user.id }); const response = await agent.prompt(message).send(); return Response.json({ output: response.output }); }, }, }, }); ``` ## 3. Add Retrieval Context [#3-add-retrieval-context] ```ts const agent = new AgentBuilder("support", model) .instructions("Use retrieved support docs when relevant.") .dynamicContext(supportDocsIndex, { topK: 3, threshold: 0.7, }) .tool(lookupOrder) .build(); ``` Build retrieval indexes during ingestion, startup, or background work, not inside every route call. ## Next [#next] Persist history in [Persistence](/docs/frameworks/tanstack-start/06-persistence). Related guides: [Runtime Context](/docs/guides/agents/runtime-context), [RAG Context](/docs/guides/retrieval/rag-context), and [Tool Handlers](/docs/guides/tools/tool-handlers). # 06 Persistence (/docs/frameworks/tanstack-start/06-persistence) Persist conversation state in server code. Client components should call server routes or server functions. ## 1. Explicit Transcript Storage [#1-explicit-transcript-storage] ```ts import { Message } from "@anvia/core"; import { supportAgent } from "~/ai/support-agent"; import { conversations } from "~/db/conversations"; export async function runSupportTurn(input: { conversationId: string; message: string; }) { const history = await conversations.loadMessages(input.conversationId); const response = await supportAgent .prompt([...history, Message.user(input.message)]) .send(); await conversations.saveMessages(input.conversationId, [ ...history, ...response.messages, ]); return response; } ``` Call this helper from a server route or `createServerFn(...)`. ## 2. Agent Memory [#2-agent-memory] ```ts const agent = new AgentBuilder("support", model) .memory(memoryStore, { savePolicy: "message" }) .build(); const response = await agent .session(conversationId, { userId }) .prompt(message) .send(); ``` Use memory when the route should not load and append messages manually. ## 3. Streaming Persistence [#3-streaming-persistence] For streaming routes, wait for the terminal event in the client or use memory on the agent. Use an event store when you need replayable runtime events. ## Next [#next] Review deployment checks in [Deploy](/docs/frameworks/tanstack-start/07-deploy). Related guides: [Memory](/docs/guides/memory), [Raw SQL](/docs/guides/memory/raw-sql), and [Event Store](/docs/guides/agents/event-store). # 07 Deploy (/docs/frameworks/tanstack-start/07-deploy) ## Runtime Checklist [#runtime-checklist] | Area | Check | | ---------------- | ----------------------------------------------------------------------- | | Server placement | Provider clients, agents, and tools stay in server modules | | Secrets | Provider keys are available only to server code | | Streaming | Host and proxy support long-lived response bodies | | Timeouts | Route timeouts exceed expected model and tool duration | | Storage | Conversations, memory, retrieval indexes, and traces use durable stores | ## Prefer Server Routes For HTTP Surfaces [#prefer-server-routes-for-http-surfaces] Server functions are useful inside the app. Server routes are the clearest contract for external clients and streaming endpoints because they return `Response` directly. ## Add Trace Metadata [#add-trace-metadata] ```ts const response = await supportAgent .prompt(message) .withTrace({ name: "support-route", userId, sessionId: conversationId, tags: ["tanstack-start"], }) .send(); ``` Attach observers for logs, metrics, Langfuse, or OpenTelemetry. ## Deployment Smoke Test [#deployment-smoke-test] ```sh curl -X POST "$APP_URL/api/support" \ -H "Content-Type: application/json" \ -d '{"message":"Say hello"}' ``` ## Next [#next] Use [Troubleshooting](/docs/frameworks/tanstack-start/08-troubleshooting) for common failures. Related guides: [Observers](/docs/guides/observability/observers), [Langfuse](/docs/guides/observability/langfuse), and [OpenTelemetry](/docs/guides/observability/otel). # 08 Troubleshooting (/docs/frameworks/tanstack-start/08-troubleshooting) ## Server Function Works But HTTP Client Cannot Call It [#server-function-works-but-http-client-cannot-call-it] Server functions are app-internal RPC-style calls. Use a server route under `src/routes` when external clients need a normal HTTP endpoint. ## Route Returns `message is required` [#route-returns-message-is-required] Send JSON with the expected field: ```sh curl -X POST http://localhost:3000/api/support \ -H "Content-Type: application/json" \ -d '{"message":"Hello"}' ``` ## Provider Key Is Missing [#provider-key-is-missing] Create the provider client only where server environment variables are available: ```ts const apiKey = process.env.OPENAI_API_KEY; if (!apiKey) throw new Error("OPENAI_API_KEY is required"); ``` ## Server Code Reaches The Client Bundle [#server-code-reaches-the-client-bundle] Keep provider clients, agents, database clients, and scoped tool factories out of client components. Put client-safe schemas and types in separate files. ## Streaming Does Not Flush [#streaming-does-not-flush] Use a server route, return a `Response`, and set NDJSON headers: ```ts return new Response(agent.prompt(message).readableStream(), { headers: { "Content-Type": "application/x-ndjson" }, }); ``` Also check host buffering and route timeouts. ## Tool Authorization Is Wrong [#tool-authorization-is-wrong] Resolve the current user in the route or server function. Close over that user in request-scoped tools instead of trusting model-provided identifiers. ## Next [#next] Revisit [Route Handler](/docs/frameworks/tanstack-start/03-route-handler), [Tools and Context](/docs/frameworks/tanstack-start/05-tools-and-context), and [Tool Errors](/docs/guides/tools/tool-errors). # 09 Human in the Loop (/docs/frameworks/tanstack-start/09-human-in-the-loop) Human-in-the-loop work belongs in server routes or server functions. The agent can wait on your approval service before a protected tool runs. ## 1. Use Studio During Development [#1-use-studio-during-development] Add approval metadata to a tool and register the same built agent in Studio: ```ts import { Studio } from "@anvia/studio"; import { supportAgent } from "~/ai/support-agent"; new Studio([supportAgent]).start({ port: 4021 }); ``` Studio handles the approval UI for tools with `approval` metadata. Your TanStack Start app can still expose its own routes for production users. ## 2. Use A Request Hook In Server Code [#2-use-a-request-hook-in-server-code] ```ts import { createHook } from "@anvia/core"; import { approvalRuntime } from "~/approvals/runtime"; function createApprovalHook(input: { userId: string; runId: string }) { return createHook({ async onToolCall({ toolName, args, tool }) { if (toolName !== "refund_order") { return tool.run(); } const approved = await approvalRuntime.waitForDecision({ userId: input.userId, runId: input.runId, toolName, args, }); return approved ? tool.run() : tool.skip("Refund was not approved."); }, }); } ``` `approvalRuntime` is your own module. It is not imported from `@anvia/core` or any Anvia package. Attach the hook from a server route: ```ts const response = await supportAgent .prompt(message) .requestHook(createApprovalHook({ userId, runId })) .send(); ``` ## 3. Create The Approval Runtime [#3-create-the-approval-runtime] `approvalRuntime` is your application runtime for reviewer state. Anvia only waits for the promise returned by `waitForDecision(...)`; your app creates the pending record, shows it to reviewers, accepts the decision, and resolves the waiting promise. ```ts type ApprovalRequest = { userId: string; runId: string; toolName: string; args: string; }; type ApprovalDecision = { approved: boolean; reason?: string; }; export function createApprovalRuntime() { const waiters = new Map void>(); return { async waitForDecision(request: ApprovalRequest): Promise { const approval = await db.approval.create({ data: { userId: request.userId, runId: request.runId, toolName: request.toolName, args: request.args, status: "pending", }, }); await notifyReviewers({ approvalId: approval.id }); const decision = await new Promise((resolve) => { waiters.set(approval.id, resolve); }); waiters.delete(approval.id); return decision.approved; }, async decide(input: { approvalId: string; reviewerId: string; approved: boolean; reason?: string; }): Promise { await db.approval.update({ where: { id: input.approvalId }, data: { status: input.approved ? "approved" : "rejected", reviewerId: input.reviewerId, decisionReason: input.reason, resolvedAt: new Date(), }, }); waiters.get(input.approvalId)?.({ approved: input.approved, reason: input.reason, }); }, }; } export const approvalRuntime = createApprovalRuntime(); ``` This in-memory waiter works for a single running process. For production, store approvals durably and resolve waiters through your queue, pub/sub, websocket, or polling worker. ## 4. Return Pending State From App Routes [#4-return-pending-state-from-app-routes] If a run can wait for approval longer than your HTTP timeout, split the workflow: | Route | Purpose | | ---------------------------------- | ---------------------------------------- | | `POST /api/support/runs` | Create a run and pending approval record | | `GET /api/approvals` | List pending approvals for the reviewer | | `POST /api/approvals/:id/decision` | Resolve the waiting approval promise | For short internal workflows, the route can wait directly. For user-facing flows, store state and notify the client. ## Next [#next] Add route tests in [Setup Tests](/docs/frameworks/tanstack-start/10-setup-tests). Core concepts: [Human in the Loop](/docs/guides/human-in-the-loop), [Approval by Hooks](/docs/guides/human-in-the-loop/tool-approval), [Approval Runtimes](/docs/guides/human-in-the-loop/approval-handlers), and [Studio Tool Approvals](/docs/studio/human-in-the-loop/tool-approvals). # 10 Setup Tests (/docs/frameworks/tanstack-start/10-setup-tests) Keep tests close to the boundary you own: server functions, route helpers, and the agent wrapper. ## 1. Install Test Tools [#1-install-test-tools] ```sh pnpm add -D vitest ``` ## 2. Extract Route Logic [#2-extract-route-logic] Put route behavior in a testable helper: ```ts import { supportAgent } from "~/ai/support-agent"; export async function handleSupportPost(request: Request): Promise { const { message } = (await request.json()) as { message?: string }; if (!message?.trim()) { return Response.json({ error: "message is required" }, { status: 400 }); } const response = await supportAgent.prompt(message).send(); return Response.json({ output: response.output }); } ``` Use the helper from the route: ```ts export const Route = createFileRoute("/api/support")({ server: { handlers: { POST: ({ request }) => handleSupportPost(request), }, }, }); ``` ## 3. Test The Helper [#3-test-the-helper] ```ts import { describe, expect, it } from "vitest"; import { handleSupportPost } from "~/routes/api/support"; describe("POST /api/support", () => { it("rejects empty messages", async () => { const response = await handleSupportPost( new Request("http://test.local/api/support", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ message: "" }), }), ); expect(response.status).toBe(400); }); }); ``` ## 4. Test Server Functions [#4-test-server-functions] ```ts import { askSupport } from "~/ai/support.functions"; const result = await askSupport({ data: { message: "How long does a reset link last?" }, }); expect(result.output).toContain("30 minutes"); ``` Use a mocked agent for route/unit tests. Reserve real provider calls for narrow integration tests. ## 5. Test Studio Without a Port [#5-test-studio-without-a-port] ```ts import { Studio } from "@anvia/studio"; import { supportAgent } from "~/ai/support-agent"; const studio = new Studio([supportAgent]); const response = await studio.fetch( new Request("http://studio.test/agents/support/runs", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ message: "Hello" }), }), ); expect(response.status).toBe(200); ``` ## Next [#next] Related guides: [Testing](/docs/guides/testing), [Tools and Pipelines](/docs/guides/testing/tools-and-pipelines), and [Studio and Providers](/docs/guides/testing/studio-and-providers). # Configure Defaults (/docs/guides/agents/agent-configuration) Agent configuration is split between build-time defaults and request-time overrides. Put stable behavior on the agent. Put user-specific history, trace metadata, request hooks, and one-off limits on the prompt request. ## Identity [#identity] ```ts const agent = new AgentBuilder("support", model) .name("Support Agent") .description("Answers customer support questions.") .build(); ``` Use the id for logs, traces, metrics, and references from other workflows. Use `name` and `description` for human-facing surfaces. ## Model Defaults [#model-defaults] ```ts const agent = new AgentBuilder("support", model) .temperature(0.2) .maxTokens(800) .additionalParams({ reasoning: { effort: "low" }, }) .build(); ``` `temperature` and `maxTokens` are normalized request options. Use `additionalParams(...)` for provider-specific fields that Anvia should pass through. For provider setup, model creation, embedding models, and provider-specific capabilities, see [Models](/docs/models). ## Tool Behavior [#tool-behavior] ```ts const agent = new AgentBuilder("support", model) .tool(lookupOrder) .toolChoice("auto") .defaultMaxTurns(3) .build(); ``` `defaultMaxTurns(...)` controls the model-tool loop. Start low: one model call, one tool call, and one final model call is usually enough for a simple tool workflow. `toolChoice(...)` accepts: | Value | Meaning | | ---------------------------- | ------------------------------- | | `"auto"` | The model may call a tool | | `"required"` | The model should call a tool | | `"none"` | The model should not call tools | | `{ type: "function", name }` | Prefer a specific tool | ## Request Overrides [#request-overrides] Use request-level methods when the setting belongs to one prompt. ```ts import { Message } from "@anvia/core"; const response = await agent .prompt([...history, Message.user(userInput)]) .maxTurns(1) .withTrace({ name: "support-chat", userId }) .send(); ``` Request options do not mutate the agent. Reuse the same agent across requests. ## Production Shape [#production-shape] Create agents once near application startup or dependency wiring. ```ts const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); export const supportAgent = new AgentBuilder("support", model) .instructions("Answer support questions using the available tools.") .tool(lookupOrder) .defaultMaxTurns(3) .build(); ``` Avoid creating provider clients, models, and agents inside every request handler unless the configuration truly changes per request. # Agent History (/docs/guides/agents/agent-history) Anvia history is a plain `Message[]`. For explicit stateless history, pass the whole transcript to `agent.prompt([...])`; the last message is the active prompt and earlier messages are history. ## Basic Shape [#basic-shape] ```ts import type { Message } from "@anvia/core"; const history: Message[] = [ { role: "user", content: [{ type: "text", text: "My checkout failed." }], }, { role: "assistant", content: [{ type: "text", text: "What error did you see?" }], }, ]; ``` The helper API creates the same shape: ```ts import { Message } from "@anvia/core"; const history = [ Message.user("My checkout failed."), Message.assistant("What error did you see?"), ]; ``` ## Use History In a Prompt [#use-history-in-a-prompt] ```ts const history = await conversations.loadMessages(conversationId); const currentPrompt = Message.user(userInput); const response = await agent.prompt([...history, currentPrompt]).send(); await conversations.saveMessages(conversationId, [ ...history, ...response.messages, ]); ``` `response.messages` is the new part of the conversation created by the run. Append it to your stored history if you want the next request to remember it. ## History With Tool Calls [#history-with-tool-calls] Tool calls are assistant content. Tool results are tool content. ```ts import type { Message } from "@anvia/core"; const history: Message[] = [ { role: "user", content: [{ type: "text", text: "Where is order A-100?" }], }, { role: "assistant", content: [ { type: "tool_call", id: "call_1", function: { name: "lookup_order", arguments: { orderId: "A-100" }, }, }, ], }, { role: "tool", content: [ { type: "tool_result", id: "call_1", content: [{ type: "text", text: "{\"status\":\"shipped\"}" }], }, ], }, { role: "assistant", content: [{ type: "text", text: "Order A-100 has shipped." }], }, ]; ``` Persist the plain message objects returned by Anvia. Only create tool-call history yourself if you are importing an existing transcript or replaying a known run. # Agent Instructions (/docs/guides/agents/agent-instructions) Instructions describe how the agent should behave. ## Instructions [#instructions] Use `.instructions(...)` for stable behavior that should apply to every prompt. ```ts const agent = new AgentBuilder("support", model) .instructions("Answer in a concise, friendly tone.") .instructions("Ask for missing details before guessing.") .instructions("Do not invent order status. Use tools when order data is needed.") .build(); ``` Multiple instruction blocks are joined together. Keep each block focused so it is easy to move, remove, or test. For facts the agent should use while answering, read [Runtime Context](/docs/guides/agents/runtime-context). # Agent Tools (/docs/guides/agents/agent-tools) Tools let an agent call application-owned behavior. The model chooses the tool and arguments; your code owns validation, permissions, side effects, and returned data. For tool definitions, schemas, handlers, results, and reusable tool sets, read [Tools](/docs/guides/tools/creating-tools). ## Add One Tool [#add-one-tool] ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { z } from "zod"; const lookupOrder = createTool({ name: "lookup_order", description: "Look up an order by order id.", input: z.object({ orderId: z.string(), }), output: z.object({ status: z.string(), }), async execute({ orderId }) { return orders.findById(orderId); }, }); const agent = new AgentBuilder("support", model) .instructions("Use tools when order status is needed.") .tool(lookupOrder) .defaultMaxTurns(3) .build(); ``` Use clear tool names and descriptions. The model relies on them when deciding whether to call a tool. ## Add Multiple Tools [#add-multiple-tools] ```ts const agent = new AgentBuilder("support", model) .tools([lookupOrder, createReturnLabel, updateAddress]) .defaultMaxTurns(4) .build(); ``` Keep tools narrow. A focused tool is easier to validate, approve, test, and observe. ## Control Tool Choice [#control-tool-choice] ```ts const agent = new AgentBuilder("support", model) .tool(lookupOrder) .toolChoice("auto") .build(); ``` | Choice | Behavior | | -------------------------------------------- | ---------------------------------------------------------------- | | `"auto"` | The model may call a tool | | `"required"` | The model should call a tool | | `"none"` | Tools are available on the agent but disabled for the model call | | `{ type: "function", name: "lookup_order" }` | Prefer one specific tool | Provider support for strict tool-choice behavior can vary. ## Tool Loop [#tool-loop] When the model calls a tool, Anvia: 1. validates tool arguments 2. calls your tool handler 3. appends a tool-result message 4. calls the model again 5. returns the final assistant answer Set a low turn limit while building the workflow. ```ts const response = await agent.prompt("Where is order A-100?").maxTurns(3).send(); ``` ## Require Approval [#require-approval] Use hook-based tool approval when a tool should not run until an approval-capable runtime approves it. ```ts import { createHook } from "@anvia/core"; const approvalHook = createHook({ onToolCall({ toolName, tool }) { if (toolName !== "refund_order") { return tool.run(); } return tool.requestApproval({ reason: "Refunds require staff approval.", rejectMessage: "Refund was not approved.", }); }, }); const response = await agent .prompt("Refund order A-100.") .requestHook(approvalHook) .send(); ``` Studio handles `tool.requestApproval(...)` automatically. Without an approval handler, core cancels clearly instead of running the guarded tool. For the full approval flow, read [Human in the Loop](/docs/guides/human-in-the-loop). # Creating Agents (/docs/guides/agents/creating-agents) `AgentBuilder` is the main entry point for creating an agent. It binds a completion model to instructions, context, tools, output rules, hooks, observers, and run defaults. ## Minimal Agent [#minimal-agent] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); const agent = new AgentBuilder("support", model) .name("Support Agent") .description("Answers product support questions.") .instructions("Answer clearly. Ask for missing details before guessing.") .build(); const response = await agent.prompt("How do I reset my password?").send(); console.log(response.output); ``` The first constructor argument is the stable agent id. Use an id that is safe to log, trace, and reference from other workflows. ## Builder Shape [#builder-shape] Most agents start with the same sequence: 1. Create a provider client. 2. Create a completion model. 3. Pass the model into `AgentBuilder`. 4. Add instructions, context, tools, and runtime defaults. 5. Call `.build()` once and reuse the agent. ```ts const agent = new AgentBuilder("support", model) .instructions("You are a support assistant for Acme.") .context("Password reset links expire after 30 minutes.", "password-policy") .defaultMaxTurns(3) .build(); ``` ## Common Builder Methods [#common-builder-methods] | Method | Use it for | | ---------------------------- | -------------------------------------------------------------- | | `.name(...)` | Human-readable display name for dashboards and traces | | `.description(...)` | Explain what the agent does, especially when exposed as a tool | | `.instructions(...)` | Persistent behavior rules sent with every prompt | | `.context(...)` | Small static facts included with every prompt | | `.dynamicContext(...)` | Runtime retrieval from a vector index | | `.tool(...)` / `.tools(...)` | Application actions the model may call | | `.mcp(...)` | Tools from MCP servers | | `.skills(...)` | Skill instructions and tools | | `.outputSchema(...)` | Schema-constrained final output | | `.defaultMaxTurns(...)` | Default tool-call loop limit | | `.observe(...)` | Run, generation, tool, usage, and trace observers | ## Prompt the Agent [#prompt-the-agent] `.prompt(...)` creates a request. Chain request-level options before `.send()`. ```ts import { Message } from "@anvia/core"; const history = await conversations.loadMessages(conversationId); const response = await agent .prompt([...history, Message.user("What did we decide earlier?")]) .maxTurns(2) .withTrace({ name: "support-follow-up", userId: "user_123" }) .send(); ``` `response.messages` contains the new messages created by this run. Store those messages when you need the next prompt to continue the conversation. ## Use an Agent as a Tool [#use-an-agent-as-a-tool] Agents can be exposed to other agents as tools. This is useful when one agent owns a narrow workflow. ```ts const refundAgent = new AgentBuilder("refunds", model) .description("Handles refund policy questions.") .instructions("Answer only refund-related questions.") .build(); const triageAgent = new AgentBuilder("triage", model) .instructions("Route refund questions to the refund agent.") .tool( refundAgent.asTool({ name: "ask_refund_agent", maxTurns: 2, }), ) .defaultMaxTurns(3) .build(); ``` Keep nested agent tools narrow. A broad agent calling another broad agent is harder to debug than a focused delegation boundary. # Event Store (/docs/guides/agents/event-store) An event store records runtime events from `PromptRequest.stream()`. Use it when you need to replay or inspect what happened during a run after the live stream has ended. Most applications start by rendering stream events directly to a UI: ```ts for await (const event of agent.prompt(prompt).stream()) { render(event); } ``` That is enough for live output, but the event data is gone once the stream is consumed. An event store gives you a durable runtime log for debugging, local inspection, Studio-like replay, audits, tests, or post-run analytics. ## Why Event Store Exists [#why-event-store-exists] Memory and event storage solve different problems. | API | Stores | Used as future model context | | ----------------- | --------------------------------------------------------------------------- | ---------------------------- | | `memory(...)` | Conversation messages: user prompts, assistant messages, final tool results | Yes | | `eventStore(...)` | Runtime events: text deltas, tool progress, nested child-agent events | No | Memory is deliberately transcript-shaped because it is loaded back into future prompts. If Anvia stored every streamed text delta, tool progress event, nested child-agent turn, or UI-only runtime marker in memory, future model calls would receive noisy partial state instead of a clean conversation. The event store exists so you can keep that runtime detail without polluting future model context. Use memory when the model should remember something. Use event store when your application should remember how a run executed. ## When To Use It [#when-to-use-it] Use an event store when you need any of these: * replay a run after the live stream is finished * inspect nested `asTool({ stream: true })` child-agent progress * debug which tool or child agent produced a bad result * build a run timeline or Studio-style viewer * calculate latency, tool usage, or agent activity after the run * keep an audit log of runtime execution separate from conversation memory Skip it when you only need the final response, or when your UI consumes live stream events and does not need replay later. ## Design Boundary [#design-boundary] Anvia treats model transcript, runtime events, and observability as separate surfaces: | Surface | Main question | Typical consumer | | ----------- | ------------------------------------- | ------------------------------ | | Memory | What should the model know next time? | Future prompt runs | | Event Store | What happened during this run? | Product UI, replay, debugging | | Observers | What telemetry should be exported? | Tracing and monitoring systems | This separation keeps each storage layer small and predictable. Your application can store events in the same database as memory if you want, but the APIs stay separate so you can apply different retention, indexing, privacy, and replay policies. ## Configure an Event Store [#configure-an-event-store] ```ts const agent = new AgentBuilder("support", model) .eventStore(eventStore, { include: "all" }) .build(); ``` Anvia calls your store during streaming runs: ```ts interface AgentEventStore { append(input: AgentEventAppendInput): Promise; load(runId: string): Promise; clear?(runId: string): Promise; } type AgentEventStoreOptions = { include?: "all" | "agent_tool_events"; }; ``` `include: "all"` stores every parent stream event and every nested child-agent event. Choose this when you want a full run replay. `include: "agent_tool_events"` stores only nested child-agent events emitted by `asTool({ stream: true })`. Choose this when the parent stream is already easy to reconstruct from memory, but child-agent progress would otherwise be lost. Event storage is tied to streaming runs. A normal `.send()` call stays opaque and does not emit or persist runtime stream events. ## Store Shape [#store-shape] ```ts type AgentEventAppendInput = { runId: string; agentId: string; agentName?: string; turn?: number; toolName?: string; toolCallId?: string; internalCallId?: string; event: unknown; }; ``` The `event` field is `unknown` because an event store is a durable log boundary. Store it as JSON or another format your application controls. Use `runId` to group one run, and use `toolName`, `internalCallId`, and `agentId` to group nested child-agent progress. The terminal `final` stream event also includes `runId`, so an application can load the saved runtime log after the live stream ends: ```ts let runId: string | undefined; for await (const event of agent.prompt(prompt).stream()) { render(event); if (event.type === "final") { runId = event.runId; } } const savedEvents = runId === undefined ? [] : await eventStore.load(runId); ``` ## Streaming Agent Tools [#streaming-agent-tools] Event stores are especially useful with streaming multi-agent tools: ```ts const coordinator = new AgentBuilder("coordinator", model) .tools([ supportAgent.asTool({ name: "ask_support_agent", stream: true }), engineeringAgent.asTool({ name: "ask_engineering_agent", stream: true }), ]) .eventStore(eventStore, { include: "all" }) .build(); ``` During `.stream()`, callers receive live `agent_tool_event` values and the event store receives the same runtime history for replay. The parent agent and child agent models must both support streaming for nested progress to appear; otherwise the agent-tool still returns its final result normally. ```ts for await (const event of coordinator.prompt(prompt).stream()) { if (event.type === "agent_tool_event") { console.log(event.agentId, event.event.type); } } ``` The parent model still receives only the final child-agent output as the normal `tool_result`. Partial child deltas are for UI, debugging, replay, and inspection. ## Production Notes [#production-notes] `append(...)` runs in the streaming path before each event is yielded. Keep it fast: write to a local database, enqueue work, or batch outside the stream if your storage backend can add noticeable latency. Apply retention and redaction policies to event storage separately from memory. Event logs may contain partial deltas, tool arguments, intermediate tool results, and nested child-agent output that you might not want to keep as long as conversation memory. ## Minimal In-Memory Store [#minimal-in-memory-store] ```ts class InMemoryAgentEventStore implements AgentEventStore { readonly records: AgentEventRecord[] = []; async append(input: AgentEventAppendInput): Promise { this.records.push({ ...input, createdAt: new Date() }); } async load(runId: string): Promise { return this.records.filter((record) => record.runId === runId); } async clear(runId: string): Promise { const remaining = this.records.filter((record) => record.runId !== runId); this.records.length = 0; this.records.push(...remaining); } } ``` For a runnable example, see `examples/cookbook/07_multi_agent/04-agent-event-store.ts`. # Multi-Agent Workflows (/docs/guides/agents/multi-agent-workflows) Use multiple agents when one coordinator should break a task into specialist work. In Anvia, the simplest pattern is to turn specialist agents into tools with `agent.asTool(...)`, then attach those tools to a coordinator agent. ## Build Specialist Agents [#build-specialist-agents] ```ts import { AgentBuilder } from "@anvia/core"; const supportAgent = new AgentBuilder("support", model) .name("Support Specialist") .description("Analyze customer impact and support next steps.") .instructions("Return concise support triage notes using only the provided facts.") .build(); const engineeringAgent = new AgentBuilder("engineering", model) .name("Engineering Specialist") .description("Analyze technical causes, diagnostics, and engineering next steps.") .instructions("Return concrete diagnostics and avoid unverified root-cause claims.") .build(); ``` Specialist agents should have narrow instructions. Give each one a clear role and output shape. ## Expose Agents as Tools [#expose-agents-as-tools] ```ts const coordinator = new AgentBuilder("coordinator", model) .name("Incident Coordinator") .instructions( [ "Coordinate specialist agents through tools.", "Give each specialist a short task based only on the user request.", "Combine their findings into one concise incident brief.", ].join("\n"), ) .tools([ supportAgent.asTool({ name: "ask_support_agent" }), engineeringAgent.asTool({ name: "ask_engineering_agent" }), ]) .defaultMaxTurns(4) .build(); ``` `asTool(...)` creates a tool with one input field: ```ts { prompt: string } ``` When the coordinator calls the tool, Anvia prompts the specialist agent and returns the specialist's final text as the tool result. By default, a specialist exposed with `asTool(...)` is opaque while it runs. The parent stream shows the parent `tool_call` and final `tool_result`, but not the child agent's intermediate turns. Enable nested streaming when your UI should show specialist progress: ```ts const coordinator = new AgentBuilder("coordinator", model) .tools([ supportAgent.asTool({ name: "ask_support_agent", stream: true }), engineeringAgent.asTool({ name: "ask_engineering_agent", stream: true }), ]) .build(); ``` When the parent runs with `.stream()`, child agent events arrive as `agent_tool_event` values. The parent model still receives only the final specialist output as the normal tool result. Nested progress is best-effort: the parent agent and child agent models must both support streaming. If nested streaming is unavailable, the agent-tool still behaves like a normal opaque `asTool(...)` call and returns the specialist's final output. Add an [event store](/docs/guides/agents/event-store) when you need to replay or inspect nested child-agent progress after the run. For memory boundaries in coordinator/specialist systems, see [Multi-Agent Memory](/docs/guides/memory/multi-agent). ## Run With Tool Concurrency [#run-with-tool-concurrency] ```ts const response = await coordinator .prompt("Prepare an incident brief for a webhook retry failure.") .withToolConcurrency(2) .send(); console.log(response.output); ``` Use `.withToolConcurrency(...)` when independent specialist tools can run at the same time. ## When to Use This Pattern [#when-to-use-this-pattern] | Pattern | Use it when | | -------------------------------- | --------------------------------------------------------------------------------------------------------------- | | `agent.asTool(...)` | A coordinator should decide which specialists to call during a prompt run and child progress can remain hidden | | `agent.asTool({ stream: true })` | A coordinator should decide which specialists to call and the caller should see child progress during streaming | | Parallel pipelines | You already know every branch should run | | Studio multiple agents | You want several agents available in one local UI | For runnable examples, see `examples/cookbook/07_multi_agent/01-agent-as-tool.ts`, `examples/cookbook/07_multi_agent/03-streaming-agent-tools.ts`, and `examples/cookbook/07_multi_agent/04-agent-event-store.ts`. # Run Lifecycle (/docs/guides/agents/run-lifecycle) A prompt run starts from `agent.prompt(...)` and ends with either a final assistant response or an error. Anvia owns the model-tool loop during that run. ## Basic Flow [#basic-flow] 1. Convert the prompt into a user message. 2. Combine stored history with messages created during the run. 3. Add instructions, static context, dynamic context, tools, and output schema. 4. Call the completion model. 5. Run requested tools. 6. Repeat until the model returns final text or the turn limit is reached. ```ts import { Message } from "@anvia/core"; const response = await agent .prompt([...history, Message.user("Where is order A-100?")]) .maxTurns(3) .send(); console.log(response.output); ``` ## Request-Level Options [#request-level-options] Use request-level methods for values that change per prompt. ```ts const response = await agent .prompt([...history, Message.user(userInput)]) .withToolConcurrency(2) .withTrace({ name: "support-message", userId, metadata: { channel: "chat" }, }) .send(); ``` These options do not mutate the agent. The next prompt starts from the agent defaults again. ## Messages Created During a Run [#messages-created-during-a-run] `response.messages` contains only the new messages produced by this prompt run. ```ts await conversations.saveMessages(conversationId, [ ...history, ...response.messages, ]); ``` When tools are used, `response.messages` can include: | Message | Role | Why it appears | | ------------ | ----------- | ---------------------------------------- | | Prompt | `user` | The prompt passed to `agent.prompt(...)` | | Tool call | `assistant` | The model requested a tool | | Tool result | `user` | Anvia returned tool output to the model | | Final answer | `assistant` | The model returned final text | ## Dynamic Context Timing [#dynamic-context-timing] Dynamic context is searched at model-call time, using the latest prompt text for that turn. ```ts const agent = new AgentBuilder("support", model) .dynamicContext(supportDocsIndex, { topK: 3 }) .build(); ``` Static context and dynamic context are both sent as documents to the completion model. Static context is always included; dynamic context is searched per prompt. ## Send vs Stream [#send-vs-stream] Use `.send()` when you only need the final response. ```ts const response = await agent.prompt("Summarize this ticket.").send(); ``` Use `.stream()` when your UI should receive incremental text, tool calls, tool results, and the final response. ```ts for await (const event of agent.prompt("Where is order A-100?").stream()) { if (event.type === "text_delta") { process.stdout.write(event.delta); } } ``` For the complete event model, read [Streaming Events](/docs/guides/streaming/streaming-events). # Run Limits (/docs/guides/agents/run-limits) Turn limits keep model-tool loops bounded. Cancellation lets hooks stop a run before unsafe or unwanted work continues. ## Set a Default Turn Limit [#set-a-default-turn-limit] ```ts const agent = new AgentBuilder("support", model) .tool(lookupOrder) .defaultMaxTurns(3) .build(); ``` For a simple tool flow, `3` is usually enough: 1. model decides to call a tool 2. Anvia returns the tool result 3. model writes the final answer ## Override Per Prompt [#override-per-prompt] ```ts const response = await agent .prompt("Where is order A-100?") .maxTurns(2) .send(); ``` Use request-level limits when a specific workflow should be stricter than the agent default. ## Handle Max Turns [#handle-max-turns] If the model keeps calling tools past the limit, Anvia throws `MaxTurnsError`. ```ts import { MaxTurnsError } from "@anvia/core"; try { await agent.prompt(userInput).send(); } catch (error) { if (error instanceof MaxTurnsError) { logger.warn({ messages: error.messages }, "agent exceeded max turns"); } throw error; } ``` Keep the stored messages from the error only if your product wants to preserve incomplete runs. ## Cancel With Hooks [#cancel-with-hooks] ```ts import { PromptCancelledError, cancelPrompt, createHook } from "@anvia/core"; const hook = createHook({ onCompletionCall({ prompt }) { if (containsSensitiveData(prompt)) { return cancelPrompt("Sensitive data is not allowed in this workflow."); } }, }); try { await agent.prompt(userInput).requestHook(hook).send(); } catch (error) { if (error instanceof PromptCancelledError) { return { message: "This request cannot be processed.", }; } throw error; } ``` Cancellation stops the run and prevents the next model or tool action. ## Skip Instead of Cancel [#skip-instead-of-cancel] Use `skipTool(...)` when only one tool call should be blocked and the model can still continue. ```ts const hook = createHook({ onToolCall({ toolName }) { if (toolName === "refund_order" && !currentUser.canRefund) { return skipTool("Current user cannot create refunds."); } }, }); ``` Use cancellation when the whole prompt should stop. Use skip when the run can continue with a safe tool result. # Runtime Context (/docs/guides/agents/runtime-context) Context provides facts the agent should use while answering. ## Static Context [#static-context] Use `.context(...)` for short facts that should be included with every request. ```ts const agent = new AgentBuilder("support", model) .instructions("Use the support policy when answering.") .context("Password reset links expire after 30 minutes.", "password-policy") .context("Priority support is available for enterprise plans.", "priority-support") .build(); ``` Static context is a poor fit for large or frequently changing knowledge bases. Use retrieval for that. ## Dynamic Context [#dynamic-context] Use `.dynamicContext(...)` when the relevant facts depend on the prompt. ```ts const agent = new AgentBuilder("support", model) .instructions("Use retrieved context when it is relevant.") .dynamicContext(supportDocsIndex, { topK: 3, format: (result) => ({ id: result.id, text: `${result.document.title}\n${result.document.body}`, }), }) .build(); ``` Anvia searches dynamic context at prompt time and adds matching documents to the completion request. ## What Goes Where [#what-goes-where] | Put it in | When | | --------------- | ----------------------------------------------- | | Instructions | It is a behavioral rule | | Static context | It is short and applies to every prompt | | Dynamic context | It is searched or changes often | | Tools | The agent must read or change application state | | History | It comes from the conversation | ## Keep Context Small [#keep-context-small] Prefer specific context over broad dumps. ```ts const agent = new AgentBuilder("billing", model) .instructions("Answer only billing questions.") .context("Invoices are generated on the first day of each month.", "invoice-cycle") .build(); ``` Large context makes answers slower, more expensive, and harder to debug. If a document can be searched, put it behind retrieval instead. # Runtime Hooks (/docs/guides/agents/runtime-hooks) Hooks run inside a prompt request and can inspect or alter runtime behavior. Use observers for telemetry. Use hooks for guardrails, approval checks, skipped tools, and cancellation. Use [tool middleware](/docs/guides/tools/tool-middleware) when you need to transform tool result text before the model receives it. ## Create a Hook [#create-a-hook] ```ts import { createHook } from "@anvia/core"; const hook = createHook({ onCompletionCall({ prompt, history }) { console.log("model_call", { role: prompt.role, history: history.length }); }, onCompletionResponse({ response }) { console.log("model_response", response.usage.totalTokens); }, onToolCall({ toolName, args }) { console.log("tool_call", { toolName, args }); }, onToolResult({ toolName, result }) { console.log("tool_result", { toolName, result }); }, }); const agent = new AgentBuilder("support", model).hook(hook).build(); ``` Register an agent hook with `.hook(...)`, or set a hook for one prompt request with `.requestHook(...)`. A request hook replaces the agent hook for that request. ## Control a Tool Call [#control-a-tool-call] `onToolCall(...)` receives a `tool` control helper. ```ts const hook = createHook({ onToolCall({ toolName, tool }) { if (toolName === "lookup_order" && maintenanceMode) { return tool.skip("Order lookup is temporarily unavailable."); } return tool.run(); }, }); ``` `tool.run()` executes the tool. `tool.skip(...)` does not execute the tool; the message becomes the tool result sent back to the model. `tool.requestApproval(...)` asks an approval-capable runtime such as Studio to pause the tool call for a human decision. Tool result middleware runs after this decision produces a result string and before `onToolResult(...)` observes it. ## Await Human Approval [#await-human-approval] Approval can be requested from inside the hook. Studio handles this action automatically when the agent is run through Studio. ```ts const hook = createHook({ onToolCall({ toolName, tool }) { if (toolName !== "refund_order") { return tool.run(); } return tool.requestApproval({ reason: "Refunds require staff approval.", rejectMessage: "Refund was not approved.", }); }, }); ``` Without Studio or another approval handler, `tool.requestApproval(...)` cancels clearly instead of running the tool. If your application owns a custom approval system, you can still await it in the hook and return `tool.run()` or `tool.skip(...)` yourself. Timeouts are optional. If the awaited approval never resolves, the agent run keeps waiting; add a timeout in your app code when the caller needs bounded latency. ## Cancel a Prompt [#cancel-a-prompt] Completion and result hooks receive a `run` helper. ```ts const hook = createHook({ onCompletionCall({ prompt, run }) { if (containsBlockedContent(prompt)) { return run.cancel("Prompt blocked by policy."); } return run.continue(); }, }); ``` Tool-call hooks can also cancel the whole run. ```ts const hook = createHook({ onToolCall({ toolName, tool }) { if (toolName === "delete_account") { return tool.cancel("Account deletion is blocked."); } return tool.run(); }, }); ``` Cancellation throws `PromptCancelledError`. Catch it at the application boundary when you want to return a user-facing message. # Comparison (/docs/guides/comparison) Anvia is an application-owned AI runtime for TypeScript teams. There is no single best TypeScript agent SDK. The right choice depends on who should own the runtime, how much framework surface you want, and whether you want provider-agnostic or provider-specific agent primitives. ## The Position [#the-position] Anvia is for teams building AI features inside an existing TypeScript product. It gives you provider-agnostic runtime primitives: agents, tools, extractors, pipelines, retrieval, streaming, hooks, and observability. Your application keeps ownership of auth, data, permissions, persistence, queues, deployment, and side effects. If you want a framework or platform to own more of the agent stack, choose one built around that surface. If you want one provider's agent runtime, tool model, or realtime stack to define the architecture, choose that provider's SDK. If you want graph orchestration, context-augmented data workflows, or UI-first streaming primitives, choose a framework centered on those jobs. ## Comparison [#comparison] | Tool | Center of gravity | Best fit | | --------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Anvia | Application-owned AI runtime | TypeScript apps that need provider-agnostic agents, tools, extraction, pipelines, RAG, and Studio while keeping infrastructure in app code | | [Mastra](https://mastra.ai/ai-agent-framework?ref=https://anvia.dev) | All-in-one TypeScript agent framework | Teams that want agents, workflows, memory, tool approval, evals, tracing, and production tooling in one framework | | [OpenAI Agents SDK](https://openai.github.io/openai-agents-js/?ref=https://anvia.dev) | OpenAI-maintained agent loop and realtime agent SDK | Products that want a small set of primitives for agent loops, handoffs, guardrails, sessions, tracing, MCP tools, and voice/realtime agents | | [Vercel AI SDK](https://ai-sdk.dev/docs/introduction?ref=https://anvia.dev) | TypeScript toolkit for AI applications and agents | React, Next.js, Vue, Svelte, Node.js, or similar apps that need provider integration, text/object generation, tool calls, streaming, and chat or generative UI hooks | | [LangChain.js / LangGraph.js](https://docs.langchain.com/oss/javascript/langchain/overview?ref=https://anvia.dev) | Agent and workflow orchestration ecosystem | Teams that want a larger ecosystem for agents, integrations, graph workflows, durable execution, memory, debugging, deployment, and observability through the LangChain/LangGraph stack | | [LlamaIndex.TS](https://ts.llamaindex.ai/?ref=https://anvia.dev) | Context-augmented data and agent workflows | Document-heavy or data-heavy apps that need ingestion, indexing, retrieval, query/chat engines, agents over data, and workflows | | [Google ADK](https://adk.dev/?ref=https://anvia.dev) / [Genkit](https://genkit.dev/docs/get-started/?ref=https://anvia.dev) | Agent development, flows, evaluation, and deployment tooling | Teams that want Google's agent and app-development tooling, including multi-agent orchestration, graph workflows, evaluation, flow-based APIs, Developer UI, plugins, and flexible deployment paths | | [VoltAgent](https://voltagent.dev/docs/?ref=https://anvia.dev) | Open-source TypeScript agent framework plus VoltOps platform | Teams that want a TypeScript agent framework with memory, RAG, guardrails, tools, MCP, voice, workflows, and optional observability/eval/deployment platform surface | | [Claude Agent SDK](https://code.claude.com/docs/en/agent-sdk/overview?ref=https://anvia.dev) | Claude Code as a library for production agents | Products that want to build agents on the Claude Code harness, including built-in file, command, and code-editing tools, with Claude API or supported cloud-provider authentication | ## The Boundary [#the-boundary] Anvia owns the AI runtime boundary: * model adapters * agent execution * tool calling * schema validation * extraction * streaming events * pipeline composition * retrieval primitives * observer hooks * local Studio iteration Your application owns the product boundary: * users and auth * databases and memory * permissions and approvals * queues and retries * audit logs * deployment * billing, CRM, internal systems, and side effects This boundary is deliberate. Agents, tools, extractors, and pipelines are ordinary TypeScript objects that can move through your routes, jobs, tests, queues, or local Studio runtime. The AI behavior is portable; the product infrastructure stays in your application. ## Decision Rule [#decision-rule] Choose Anvia if you want AI behavior to be explicit TypeScript objects inside your app. Choose something else if you want the framework, provider, or platform to own more of the runtime. ## Next [#next] Read [Design Philosophy](/docs/guides/design-philosophy) to see why Anvia is shaped around application-owned objects instead of a framework-owned runtime. # Cookbook (/docs/guides/cookbook) The cookbook is the runnable companion to the guides. Use it when you want to verify a concept from the command line before adapting it into product code. Each level introduces one layer at a time: | Level | Focus | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------- | | Basics | Text calls, chat history, static context, streaming, and `ReadableStream` output | | Tools | Tool calls, streamed tool events, hooks, concurrency, conditional tools, application state, guarded tools, and dynamic tool selection | | Structured output | Extraction, output schemas, context, retries, and extraction with history | | Providers and multimodal | Provider adapters, model capabilities, model listing, reasoning streams, attachments, image generation, audio generation, and transcription | | Pipelines | Step transforms, composition, named parallel branches, batching, agents, extraction, and richer workflows | | Retrieval | Embeddings, vector search, metadata filters, RAG context, document loaders, vector stores, and embedding provider variants | | Multi-agent | Basic agent-tools, pipeline-backed parallel specialists, streaming agent-tools, and event stores | | Evals | Deterministic metrics, semantic similarity, custom metrics, agent eval targets, and LLM judge/score | | Studio | Single-agent, multi-agent, and subagent runners, tool approvals, questions, and Knowledge inspection | | Integrations | MCP tools, local skills, Langfuse tracing, and Langfuse eval reporting | ## 1. Install Dependencies [#1-install-dependencies] From the repository root: ```sh pnpm install ``` Create a local `.env` file for examples that call provider APIs: ```sh OPENAI_API_KEY=... OPENAI_BASEURL=... ANTHROPIC_API_KEY=... GEMINI_API_KEY=... MISTRAL_API_KEY=... ``` Anvia clients still receive credentials explicitly in code. The cookbook uses `dotenv` only as a local configuration source. ## 2. Run the First Example [#2-run-the-first-example] ```sh pnpm cookbook:basics:01 ``` That runs `examples/cookbook/01_basics/01-text-call.ts`. ## 3. Follow the Path [#3-follow-the-path] Run the default example for each level: ```sh pnpm cookbook:basics pnpm cookbook:tools pnpm cookbook:structured-output pnpm cookbook:providers pnpm cookbook:pipelines pnpm cookbook:retrieval pnpm cookbook:multi-agent pnpm cookbook:evals pnpm cookbook:studio pnpm cookbook:integrations ``` Numbered scripts are available when you want to step through a level in order: ```sh pnpm cookbook:tools:01 pnpm cookbook:pipelines:04 pnpm cookbook:retrieval:05 pnpm cookbook:studio:06 pnpm cookbook:integrations:04 ``` Legacy names such as `cookbook:basic:05`, `cookbook:intermediate:14`, `cookbook:pipeline:04`, `cookbook:rag:05`, and `cookbook:multimodal:03` remain available as aliases. ## 4. Use Chroma Examples [#4-use-chroma-examples] Start ChromaDB before running the Chroma-backed RAG examples: ```sh docker compose -f examples/cookbook/compose.cookbook.yml up -d pnpm cookbook:retrieval:05 pnpm cookbook:retrieval:06 pnpm cookbook:retrieval:07 pnpm cookbook:retrieval:08 ``` Use the in-memory and Transformers examples when you do not need a separate vector database. ## 5. Map Examples to Guides [#5-map-examples-to-guides] | Goal | Cookbook | Guide | | ---------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------- | | Add a tool | `tools:01` | [Add Tools](/docs/guides/learning-paths/add-tools) | | Return structured data | `structured-output:01`, `structured-output:02` | [Structured Output](/docs/guides/structured-output/schemas) | | Inspect model capabilities | `providers:03` | [Provider Clients and Models](/docs/guides/sdk-fundamentals/clients-and-models) | | List provider models | `providers:10` | [Model Listing](/docs/reference/core/model-listing) | | Stream agent events | `tools:02` | [Streaming Events](/docs/guides/streaming/streaming-events) | | Render reasoning summaries | `providers:04` | [Streaming Events](/docs/guides/streaming/streaming-events) | | Select dynamic tools | `tools:09` | [Tool Sets](/docs/guides/tools/tool-sets) | | Add approval behavior | `tools:08`, `studio:03` | [Human in the Loop](/docs/guides/human-in-the-loop) | | Add retrieval | `retrieval:01` through `retrieval:06` | [Add Retrieval](/docs/guides/learning-paths/add-retrieval) | | Run evals | `evals:01` through `evals:05`, `integrations:04` | [Evals](/docs/guides/testing/evals) | | Generate or transcribe media | `providers:07` through `providers:09` | [Image Generation](/docs/reference/core/image-generation) | | Inspect locally in Studio | `studio:01`, `studio:05`, `studio:06` | [Run Studio](/docs/studio/run-studio) | Before changing public APIs, add or update a cookbook example so behavior is easy to verify from the command line. # Design Philosophy (/docs/guides/design-philosophy) Anvia is designed as an application-owned AI runtime for TypeScript teams, not a full application platform. The core idea is simple: AI features should be ordinary TypeScript objects that your application creates, configures, tests, and passes around. Anvia owns the AI runtime boundary. Your application owns product behavior. That boundary is deliberate: * Anvia owns model adapters, agent execution, tool calling, schemas, streaming, pipelines, retrieval primitives, observers, and local Studio iteration. * Your application owns users, auth, databases, memory, permissions, approvals, queues, audit logs, deployment, and side effects. ## Stable Runtime Objects [#stable-runtime-objects] Builders configure behavior. Built agents, extractors, tools, models, and pipelines are stable values that can be passed into routes, jobs, tests, Studio, or other workflows. This makes behavior easier to reason about. An agent does not depend on a hidden global registry or framework runtime to discover its model, tools, context, or observers. ```ts // app/ai.ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ apiKey: process.env.OPENAI_API_KEY }); const model = client.completionModel("gpt-5.5"); export const supportAgent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .defaultMaxTurns(3) .build(); ``` ```ts // app/routes/support.ts import { supportAgent } from "../ai"; export async function POST(request: Request) { const { message } = await request.json(); const response = await supportAgent.prompt(message).send(); return Response.json({ output: response.output }); } ``` Advantage: the route receives an already-built object. Tests can import the same shape or replace it with a fake agent. There is no hidden app container that must be booted before the route can run. ## Application-Owned Infrastructure [#application-owned-infrastructure] Anvia does not try to be the source of truth for your database, auth system, permission policy, queue, deployment topology, memory layer, or approval operations. Those decisions usually affect product correctness and security, so they should stay in application code. Anvia gives you hooks, tool approval metadata, Studio approval UI for local iteration, observers, and storage interfaces where integration is useful, but the ownership boundary remains explicit. ```ts import { createTool } from "@anvia/core"; import { z } from "zod"; import { requirePermission } from "./auth"; import { billing } from "./billing"; export function createRefundTool(userId: string) { return createTool({ name: "issue_refund", description: "Issue a refund for a paid order.", input: z.object({ orderId: z.string(), amount: z.number().positive(), }), async execute({ orderId, amount }) { await requirePermission(userId, "refund:create"); return billing.refund({ orderId, amount }); }, }); } ``` Advantage: the side effect remains ordinary product code. Your auth, audit logs, database transactions, and retries stay where your team already manages them. Approval follows the same boundary. Core hooks can run, skip, or cancel a tool call, and Studio can interpret tool approval metadata while you are testing the agent. Your product still decides who is allowed to approve, where decisions are stored, how notifications work, and what audit trail is required. ```ts import { createHook } from "@anvia/core"; const permissionHook = createHook({ async onToolCall({ toolName, tool }) { if (toolName === "read_payroll") { return tool.skip("Payroll data is restricted."); } if (toolName === "delete_account") { const approved = await approvals.waitForDecision({ toolName, reason: "Account deletion requires human approval.", }); return approved ? tool.run() : tool.cancel("Account deletion was not approved."); } return tool.run(); }, }); ``` Advantage: Anvia owns the execution control point, but your application owns the permission decision and approval runtime. ## Provider-Agnostic Model Boundary [#provider-agnostic-model-boundary] Provider clients create reusable completion and embedding models. Agents and extractors depend on normalized model capabilities, not on one provider's runtime model. This lets you use OpenAI, Anthropic, Gemini, Mistral, OpenAI-compatible APIs, or local embeddings without changing the way your application composes agents and workflows. ```ts import { AgentBuilder } from "@anvia/core"; import { AnthropicClient } from "@anvia/anthropic"; import { OpenAIClient } from "@anvia/openai"; const openai = new OpenAIClient({ apiKey: process.env.OPENAI_API_KEY }); const anthropic = new AnthropicClient({ apiKey: process.env.ANTHROPIC_API_KEY }); const model = process.env.PROVIDER === "anthropic" ? anthropic.completionModel("claude-opus-4-6") : openai.completionModel("gpt-5.5"); export const agent = new AgentBuilder("triage", model) .instructions("Classify the customer request.") .build(); ``` Advantage: provider choice is a model construction concern. The agent, tools, pipeline, and calling code can stay the same when you change providers. ## Typed Boundaries Over Hidden Magic [#typed-boundaries-over-hidden-magic] Tools use Zod schemas. Extractors use schemas. Pipelines preserve TypeScript input and output types across steps. Prompt responses, stream events, and observer events use explicit contracts. The goal is not to remove all runtime errors. The goal is to make boundaries visible, testable, and hard to misuse casually. ```ts import { ExtractorBuilder, PipelineBuilder, createTool } from "@anvia/core"; import { z } from "zod"; const ticketSchema = z.object({ customer: z.string(), priority: z.enum(["low", "medium", "high"]), summary: z.string(), }); const lookupCustomer = createTool({ name: "lookup_customer", input: z.object({ email: z.string().email() }), execute: async ({ email }) => crm.lookup(email), }); const extractor = new ExtractorBuilder(model, ticketSchema).build(); const pipeline = new PipelineBuilder() .step((input) => input.trim()) .extract(extractor) .step((ticket) => ({ ...ticket, needsEscalation: ticket.priority === "high", })) .build(); ``` Advantage: the boundaries are visible in code. Tool input is validated before execution, extraction has a schema, and pipeline output types evolve step by step. ## Composition Before Orchestration [#composition-before-orchestration] Anvia favors small composable objects over a large orchestration layer. Agents can become tools. Pipelines can call agents and extractors. Tools can wrap application services. Studio can run already-built agents while you are still shaping the workflow. This keeps simple workflows small while still allowing larger systems to be built from the same primitives. ```ts const refundAgent = new AgentBuilder("refunds", model) .description("Answers refund-policy questions.") .instructions("Only answer refund-related questions.") .build(); const supportAgent = new AgentBuilder("support", model) .instructions("Route refund questions to the refund specialist.") .tool( refundAgent.asTool({ name: "ask_refund_agent", maxTurns: 2, }), ) .build(); const workflow = new PipelineBuilder() .prompt(supportAgent) .extract(extractor) .build(); ``` Advantage: larger workflows are assembled from the same objects used in small workflows. You can test the refund agent, support agent, extractor, and pipeline separately. ## Studio as an Iteration Runtime [#studio-as-an-iteration-runtime] Studio exists so you can test and inspect built agents before investing in product frontend, backend routes, internal dashboards, or approval consoles. The design goal is not for Studio to replace your production application. The goal is to shorten the loop while the agent is still unstable: prompt it, inspect messages and tool calls, review traces, try sessions, and exercise approval metadata from a local UI. ```ts import { Studio } from "@anvia/studio"; // Build the agent first. Use Studio to test it before building product UI. new Studio([supportAgent]).start({ port: 3000 }); ``` Advantage: teams can focus on agent behavior first. Once the agent contract is stable, the same built agent can move into your route, job, queue worker, or product UI. ## Optional Integrations [#optional-integrations] MCP, skills, retrieval, Chroma, Langfuse, and transformers are optional layers. They are useful when the application needs them, but they are not required to understand or run the core agent model. The advantage is that teams can start with a single model call or agent and add retrieval, tools, tracing, pipelines, or local Studio testing incrementally. ```ts import { langfuse } from "@anvia/langfuse"; const tracedAgent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .observe(langfuse.create({ publicKey, secretKey })) .build(); ``` Advantage: integrations add capabilities around the same runtime objects. You do not need to adopt Langfuse, retrieval, MCP, or a production UI before the basic agent model is useful. ## Tradeoffs [#tradeoffs] | Design choice | Advantage | Tradeoff | | -------------------------------------- | -------------------------------------------------------- | ---------------------------------------------------- | | Stable built objects | Easier testing, sharing, and reasoning | You wire objects explicitly | | Application-owned infrastructure | Better fit for existing products and security boundaries | Less automatic platform behavior | | Provider-agnostic models | Easier provider switching and mixed-provider systems | Some provider-specific features stay behind adapters | | Typed tools, extractors, and pipelines | Clear contracts and safer composition | More upfront schema definition | | Studio as an iteration runtime | Faster agent testing before product UI/API work | Studio is not your production product surface | | Optional integrations | Smaller core and incremental adoption | You choose and configure the integrations you need | ## Practical Rule [#practical-rule] If a decision affects product correctness, security, or data ownership, keep it in application code or a tool. If a decision affects how a model is prompted, constrained, streamed, or observed, configure it on the agent or prompt request. # Getting Started (/docs/guides/getting-started) This guide creates a small TypeScript project, runs one Anvia agent, then adds a tool and conversation history. ## 1. Create a Project [#1-create-a-project] ```sh mkdir anvia-quickstart cd anvia-quickstart pnpm init ``` Install Anvia and the TypeScript runner used in this guide: ```sh pnpm add @anvia/core @anvia/openai @anvia/anthropic @anvia/gemini @anvia/mistral pnpm add -D tsx typescript @types/node ``` Install `zod` if you want to add tools or structured output: ```sh pnpm add zod ``` ## 2. Set a Provider Key [#2-set-a-provider-key] Load provider credentials through your app's normal configuration system. Anvia clients use explicit constructor values and do not read environment variables themselves. ## 3. Create `main.ts` [#3-create-maints] Create a `main.ts` file: ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); const agent = new AgentBuilder("support", model) .name("Support Agent") .description("Answers product support questions.") .instructions("Answer clearly. Ask for missing details before guessing.") .build(); const response = await agent .prompt("A user cannot reset their password. What should I check first?") .send(); console.log(response.output); console.log(`Tokens: ${response.usage.totalTokens}`); ``` Run it: ```sh pnpm exec tsx main.ts ``` You now have the basic Anvia loop: 1. `new OpenAIClient({ apiKey })` configures provider access. 2. `client.completionModel("gpt-5.5")` creates a reusable model. 3. `AgentBuilder` configures the agent. 4. `agent.prompt(...).send()` runs one prompt and returns a final response. ## 4. Use a Different Provider [#4-use-a-different-provider] Provider clients have the same basic shape. Swap only the client and model name. OpenAI-compatible endpoint: ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl, apiKey, }); const model = client.completionModel("openai/gpt-5.5"); const agent = new AgentBuilder("support", model) .instructions("Answer clearly.") .build(); ``` Anthropic: ```ts import { AgentBuilder } from "@anvia/core"; import { AnthropicClient } from "@anvia/anthropic"; const client = new AnthropicClient({ apiKey }); const model = client.completionModel("claude-opus-4-6"); const agent = new AgentBuilder("support", model) .instructions("Answer clearly.") .build(); ``` Gemini: ```ts import { AgentBuilder } from "@anvia/core"; import { GeminiClient } from "@anvia/gemini"; const client = new GeminiClient({ apiKey }); const model = client.completionModel("gemini-3.1-flash-lite-preview"); const agent = new AgentBuilder("support", model) .instructions("Answer clearly.") .build(); ``` Mistral: ```ts import { AgentBuilder } from "@anvia/core"; import { MistralClient } from "@anvia/mistral"; const client = new MistralClient({ apiKey }); const model = client.completionModel("mistral-large-latest"); const agent = new AgentBuilder("support", model) .instructions("Answer clearly.") .build(); ``` Provider support can vary for attachments, streaming details, tool behavior, and model-specific parameters. Anvia normalizes the SDK surface, but it does not make every provider model identical. ## 5. Add Static Context [#5-add-static-context] Use `.context(...)` for small facts that should be included with every prompt. ```ts const agent = new AgentBuilder("support", model) .instructions("Use the support policy when answering.") .context("Password reset links expire after 30 minutes.", "password-policy") .build(); const response = await agent.prompt("How long does a reset link last?").send(); ``` Use retrieval and dynamic context for larger knowledge bases. ## 6. Add a Tool [#6-add-a-tool] Tools expose application-owned behavior to the agent. Anvia validates tool input with Zod before calling your function. ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; const lookupOrder = createTool({ name: "lookup_order", description: "Look up an order by order id.", input: z.object({ orderId: z.string().describe("The order id to inspect."), }), output: z.object({ status: z.string(), carrier: z.string().optional(), }), async execute({ orderId }) { return { status: orderId === "A-100" ? "shipped" : "unknown", carrier: "DHL", }; }, }); const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); const agent = new AgentBuilder("support", model) .instructions("Use tools when order status is needed.") .tool(lookupOrder) .defaultMaxTurns(3) .build(); const response = await agent.prompt("Where is order A-100?").send(); console.log(response.output); ``` `defaultMaxTurns(3)` gives the agent room to ask the model, run the tool, and ask the model for the final answer. Keep turn limits low until the workflow is well understood. ## 7. Persist History [#7-persist-history] Anvia history is a plain `Message[]`. Store it wherever your application stores conversations. ```ts import { Message } from "@anvia/core"; const history = await conversations.loadMessages(conversationId); const response = await agent .prompt([...history, Message.user(userInput)]) .send(); await conversations.saveMessages(conversationId, [ ...history, ...response.messages, ]); ``` `response.messages` contains only the new prompt, assistant messages, and tool result messages created during the run. Append it to your stored history when you want the next turn to remember it. ## 8. Common Mistakes [#8-common-mistakes] | Mistake | Better approach | | ------------------------------------------------ | ----------------------------------------------------------- | | Creating a provider client for every prompt | Create clients and models once, then reuse them | | Saving only `response.output` | Save `response.messages` when you need conversation history | | Putting large knowledge bases in `.context(...)` | Use retrieval and dynamic context | | Letting tools perform unchecked side effects | Enforce product permissions inside tool code | | Setting high turn limits immediately | Start low and increase only when the workflow needs it | ## 9. Next Steps [#9-next-steps] Read these next: 1. [Build an Agent](/docs/guides/learning-paths/build-an-agent) 2. [Add Tools](/docs/guides/learning-paths/add-tools) 3. [Persist Conversations](/docs/guides/learning-paths/persist-conversations) 4. [Return Structured Output](/docs/guides/learning-paths/return-structured-output) 5. [Cookbook](/docs/guides/cookbook) 6. [Prepare for Production](/docs/guides/learning-paths/prepare-for-production) # Approval Runtimes (/docs/guides/human-in-the-loop/approval-handlers) Approval runtimes are userland code. A hook can call a database, wait on a queue, show a UI, notify reviewers, apply permission checks, or enforce timeout policy before it returns a tool action. Studio gives you a local approval UI. Build your own runtime when approval needs to integrate with product permissions, Slack, ticketing systems, audit logs, or production queues. ## Runtime Shape [#runtime-shape] `approvalRuntime` is a name for your own application object. Do not import it from Anvia; create it next to your database, queue, notification, or admin UI code. ```ts const approvalRuntime = { async waitForDecision(request: { toolName: string; args: string; reason?: string; }): Promise { const approval = await db.approval.create({ data: { toolName: request.toolName, args: request.args, reason: request.reason, status: "pending", }, }); await notifyReviewers({ approvalId: approval.id }); const decision = await waitUntilResolved(approval.id); return decision.status === "approved"; }, }; ``` Anvia does not define this object. It is your app boundary for approval state, notifications, optional timeouts, and audit records. ## Use It from a Hook [#use-it-from-a-hook] ```ts const approvalHook = createHook({ async onToolCall({ toolName, args, tool }) { if (toolName !== "refund_order") { return tool.run(); } const approved = await approvalRuntime.waitForDecision({ toolName, args, reason: "Refunds require staff approval.", }); return approved ? tool.run() : tool.skip("Refund was not approved."); }, }); ``` ## Store Review State [#store-review-state] Persist enough data to explain what happened later. ```ts type ApprovalRecord = { id: string; runId: string; agentId: string; toolName: string; args: string; reason?: string; status: "pending" | "approved" | "rejected" | "timed_out"; requestedAt: string; resolvedAt?: string; reviewerId?: string; decisionReason?: string; }; ``` The model prompt is not an audit log. Store tool arguments, reviewer identity, decision reason, and timestamps in your application database. ## Notify Reviewers [#notify-reviewers] Your runtime can notify one or more review surfaces. ```ts async function waitForDecision(request: ApprovalRequest): Promise { const approval = await approvalStore.create(request); await Promise.all([ slack.sendApprovalMessage(approval), adminEvents.publish("approval.created", approval), ]); const decision = await approvalStore.waitUntilResolved(approval.id); return decision.status === "approved"; } ``` `approvalStore` is also your code. It can be a Prisma model wrapper, SQL repository, queue-backed service, or any persistence layer your application already uses. The hook does not care whether the decision came from Studio, your app, Slack, email, or a queue worker. It only awaits a boolean or a richer decision object. ## Optional Timeouts [#optional-timeouts] Timeouts are not required by Anvia. If the approval promise never resolves, the agent run keeps waiting. Use a timeout when the caller needs bounded latency, especially in production HTTP requests, queues, or UIs. ```ts const approved = await Promise.race([ approvalRuntime.waitForDecision({ toolName, args, reason: "Refunds require staff approval.", }), sleep(60_000).then(() => false), ]); return approved ? tool.run() : tool.skip("No reviewer approved this action in time."); ``` Use clear messages because the model sees them as the skipped tool result. ## Rich Decisions [#rich-decisions] Return more than a boolean when the final tool result should include reviewer context. ```ts const decision = await approvalRuntime.waitForDecision({ toolName, args, reason: "Refunds require staff approval.", }); if (decision.status === "approved") { return tool.run(); } return tool.skip( decision.reason ?? "The reviewer rejected this action.", ); ``` Keep the skipped message concise. It becomes the tool result the model uses to produce the final response. # Approval Settings (/docs/guides/human-in-the-loop/approval-settings) Use tool approval settings when a side-effect tool should usually require review in Studio. The policy stays next to the tool definition, so users can move between Studio and their own frontend/backend without changing the tool contract. Core only stores this metadata. Studio interprets it by installing a per-run request hook. ## Protect a Tool [#protect-a-tool] ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { Studio } from "@anvia/studio"; import { z } from "zod"; const issueRefund = createTool({ name: "issue_refund", description: "Issue a customer refund.", input: z.object({ orderId: z.string(), amount: z.number().positive(), reason: z.string(), }), output: z.object({ refundId: z.string(), status: z.enum(["issued"]), }), approval: { when: ({ args }) => args.amount > 100, reason: ({ args }) => `Review refund of $${args.amount} for ${args.orderId}.`, rejectMessage: "Refund was not approved.", }, async execute({ orderId }) { return { refundId: `rf_${orderId.toLowerCase()}`, status: "issued" as const, }; }, }); const agent = new AgentBuilder("support", model) .tool(issueRefund) .defaultMaxTurns(3) .build(); new Studio([agent]).start(); ``` When the model calls `issue_refund`, Studio evaluates `approval.when(...)`. If it returns `true`, Studio creates a pending approval. If the reviewer approves, Studio runs the tool. If the reviewer rejects, Studio skips the tool and returns the rejection message to the model. ## Approval Context [#approval-context] ```ts type ToolApprovalContext = { toolName: string; args: TArgs; rawArgs: string; toolCallId?: string; internalCallId: string; run: { agentId: string; runId: string; sessionId?: string; metadata?: JsonObject; }; }; ``` `args` is parsed with the tool input schema. Use it for conditional policy. ```ts approval: { when: ({ args, run }) => args.amount > 100 || run.metadata?.environment === "production", } ``` ## Dynamic Reasons [#dynamic-reasons] Use `reason` to show reviewers what they are deciding. ```ts approval: { when: ({ args }) => args.amount > 100, reason: ({ args }) => `Approve ${args.reason} refund of $${args.amount} for ${args.orderId}.`, rejectMessage: ({ args }) => `Refund ${args.orderId} was not approved.`, } ``` `reason` and `rejectMessage` may be strings or async functions. ## When to Use Hooks Instead [#when-to-use-hooks-instead] Use [approval by hooks](/docs/guides/human-in-the-loop/tool-approval) when approval depends on request-local state, app permissions, external policy services, or policy that is not tied to one tool definition. Agent hooks run before Studio approval metadata. If an agent hook returns `tool.skip(...)` or `tool.cancel(...)`, Studio does not ask for approval. If it returns `tool.requestApproval(...)`, Studio handles that approval request directly. # Ask Question (/docs/guides/human-in-the-loop/ask-question) Use an `ask_question` tool when the model needs information from a person before it can continue. The tool is still a normal tool. The schema is portable: Studio provides a built-in UI for it, and your own app can render the same contract in a custom frontend. ## Question Tool Contract [#question-tool-contract] ```ts const questionChoiceSchema = z.object({ label: z.string(), value: z.string(), }); const askQuestion = createTool({ name: "ask_question", description: "Ask the human operator one or more follow-up questions.", input: z.object({ questions: z.array( z.object({ id: z.string(), question: z.string(), choices: z.array(questionChoiceSchema).min(1), }), ), }), output: z.object({ answers: z.array( z.object({ questionId: z.string(), answer: z.string(), choice: z.string().optional(), custom: z.boolean().optional(), }), ), }), async execute({ questions }) { return humanInput.ask({ questions }); }, }); ``` Studio intercepts `ask_question` before `execute(...)` runs, renders the questions, waits for answers, and sends the answers back to the model as the tool result. Outside Studio, the same `execute(...)` can call your own UI, queue, websocket, or backend workflow. ## Multiple Questions [#multiple-questions] Ask related questions in one tool call instead of forcing several back-and-forth calls. ```ts { "questions": [ { "id": "priority", "question": "What is the priority level?", "choices": [ { "label": "Low", "value": "low" }, { "label": "High", "value": "high" }, { "label": "Critical", "value": "critical" } ] }, { "id": "channel", "question": "Which support channel should we use?", "choices": [ { "label": "Email", "value": "email" }, { "label": "Phone", "value": "phone" }, { "label": "Other", "value": "other" } ] } ] } ``` Studio shows each question as a compact control. Choices become buttons, and Studio always shows a custom input after the choices. Custom input is UI behavior, not a separate tool schema setting. ## Agent Instructions [#agent-instructions] Tell the model when to ask and how to batch questions. ```ts const agent = new AgentBuilder("support", model) .instructions( [ "Use ask_question when priority, channel, or operator context is missing.", "Ask multiple questions in one ask_question call when you need multiple answers.", "Use choices for bounded decisions.", "Studio always shows a custom input after the choices.", "After the human answers, continue the workflow with the confirmed values.", ].join("\n"), ) .tool(askQuestion) .build(); ``` ## Outside Studio [#outside-studio] If you are not using Studio, keep the same schema and implement the waiting boundary yourself. ```ts const humanInput = { async ask(request: { questions: Array<{ id: string; question: string; choices: Array<{ label: string; value: string }>; }>; }): Promise<{ answers: Array<{ questionId: string; answer: string; choice?: string; custom?: boolean; }>; }> { const pending = await db.humanQuestion.create({ data: { questions: request.questions, status: "pending", }, }); await ui.notifyUser({ questionId: pending.id }); return waitForUserAnswers(pending.id); }, }; ``` The run waits while `humanInput.ask(...)` awaits. Your frontend can render the same `questions` array as buttons, selects, radios, or custom text inputs. Add an app-owned timeout if the request must finish within a deadline. ## Cookbook [#cookbook] Run the Studio question example: ```bash pnpm cookbook:studio:05 ``` # Human in the Loop (/docs/guides/human-in-the-loop) Human-in-the-loop means the agent run waits for a person before continuing. Use it for approvals, operator feedback, missing context, escalation decisions, outbound messages, refunds, account changes, deletes, and other workflows where the model should not decide alone. Anvia keeps the core execution model small: tools run, hooks can run/skip/cancel/request approval, and awaited promises pause the run. Studio adds a zero-config UI layer for common approval and question flows. ## Choose a Pattern [#choose-a-pattern] | Pattern | Best for | Where it lives | | ------------------------------------------------------------------------------ | --------------------------------------------------------------- | ----------------- | | [Approval settings in tools](/docs/guides/human-in-the-loop/approval-settings) | Studio approval UI with no custom hook | Tool metadata | | [Approval by hooks](/docs/guides/human-in-the-loop/tool-approval) | Dynamic policy or request-specific rules | `onToolCall(...)` | | [Ask question tools](/docs/guides/human-in-the-loop/ask-question) | Missing user input while a run is active | Normal tools | | [Approval runtimes](/docs/guides/human-in-the-loop/approval-handlers) | Databases, queues, notifications, audit logs, optional timeouts | Your app | ## Studio Approval Metadata [#studio-approval-metadata] Put approval policy next to the side-effect tool when you want Studio to handle the approval UI. ```ts const refundOrder = createTool({ name: "refund_order", description: "Issue a customer refund.", input: z.object({ orderId: z.string(), amount: z.number().positive(), }), approval: { when: ({ args }) => args.amount > 100, reason: ({ args }) => `Review refund of $${args.amount} for ${args.orderId}.`, rejectMessage: "Refund was not approved.", }, async execute({ orderId, amount }) { return refunds.create(orderId, amount); }, }); new Studio([agent]).start(); ``` Core stores this metadata but does not enforce it. Studio reads it and installs a per-run request hook that waits for a decision. ## Hook Approval [#hook-approval] Use a hook when approval is not a property of the tool itself. Studio handles `tool.requestApproval(...)` with the same approval UI and API used for tool metadata. ```ts const approvalHook = createHook({ onToolCall({ toolName, tool }) { if (toolName !== "refund_order") { return tool.run(); } return tool.requestApproval({ reason: "Refunds require staff approval.", rejectMessage: "Refund was not approved.", }); }, }); await agent.prompt("Refund order A-100.").requestHook(approvalHook).send(); ``` The model sees skipped tools as tool results, so write rejection messages as text the model can use in its final answer. ## Ask for Feedback [#ask-for-feedback] Use a normal tool when the model needs information from a person. Studio recognizes `ask_question` and renders a compact question UI. ```ts const askQuestion = createTool({ name: "ask_question", description: "Ask the human operator one or more follow-up questions.", input: z.object({ questions: z.array( z.object({ id: z.string(), question: z.string(), choices: z.array(z.object({ label: z.string(), value: z.string(), })).min(1), }), ), }), async execute() { return { answers: [] }; }, }); ``` Outside Studio, implement `ask_question` by awaiting your app UI, queue, or websocket and returning the answer as the tool result. ## What Anvia Owns [#what-anvia-owns] | Anvia owns | Your app owns | | -------------------------------------------- | -------------------------------------------------------- | | detecting the tool call | choosing which tools need approval | | awaiting hook and tool promises | showing UI, sending notifications, or waiting on a queue | | running the tool after `tool.run()` | deciding who can approve | | returning `tool.skip(...)` text to the model | storing audit records | ## Timeouts [#timeouts] Timeouts are optional. If a hook, tool, or Studio question waits forever, the run waits forever. Add a timeout in your runtime when the caller needs bounded latency. # Approval by Hooks (/docs/guides/human-in-the-loop/tool-approval) Use `onToolCall(...)` when approval is runtime behavior rather than tool metadata. A hook can request approval before it returns `tool.run()`, `tool.skip(...)`, or `tool.cancel(...)`. Studio handles `tool.requestApproval(...)` automatically. Without Studio or another approval handler, the run cancels clearly instead of executing the guarded tool. ## Require Approval for One Tool [#require-approval-for-one-tool] ```ts import { createHook } from "@anvia/core"; const approvalHook = createHook({ onToolCall({ toolName, tool }) { if (toolName !== "refund_order") { return tool.run(); } return tool.requestApproval({ reason: "Review this refund request.", rejectMessage: "Refund was not approved.", }); }, }); ``` Use the `args` value when the reviewer needs to see exactly what the model is trying to do. ## Attach the Hook [#attach-the-hook] Attach the hook to one request when approval rules are request-specific. ```ts const response = await agent .prompt("Refund order A-100.") .requestHook(approvalHook) .send(); ``` Attach the hook to the agent when the rule should apply to every request. ```ts const agent = new AgentBuilder("support", model) .tool(refundOrder) .hook(approvalHook) .defaultMaxTurns(3) .build(); ``` ## Require Approval by Tool Name [#require-approval-by-tool-name] Keep the approval rule explicit. ```ts const sensitiveTools = new Set(["refund_order", "cancel_subscription"]); const approvalHook = createHook({ onToolCall({ toolName, tool }) { if (!sensitiveTools.has(toolName)) { return tool.run(); } return tool.requestApproval({ reason: `${toolName} requires human review.`, rejectMessage: `${toolName} was not approved.`, }); }, }); ``` Use normal tool execution for safe read-only tools such as lookup or search. ## Require Approval by Argument [#require-approval-by-argument] Parse the pending tool arguments when only some calls need review. ```ts import { createHook, parseToolArgs } from "@anvia/core"; const approvalHook = createHook({ onToolCall({ toolName, args, tool }) { if (toolName !== "issue_refund") { return tool.run(); } const parsed = parseToolArgs(args); if ( typeof parsed !== "object" || parsed === null || !("amount" in parsed) || typeof parsed.amount !== "number" ) { return tool.cancel("Invalid refund request."); } if (parsed.amount <= 100) { return tool.run(); } return tool.requestApproval({ reason: `Refund amount is $${parsed.amount}.`, rejectMessage: "Refund was not approved.", }); }, }); ``` Use `tool.cancel(...)` for malformed or unsafe runs that should stop immediately. Use `tool.skip(...)` when the model can continue with a tool result message. ## Request Hook vs Agent Hook [#request-hook-vs-agent-hook] Use `.requestHook(...)` when approval rules are specific to one prompt request. ```ts await agent .prompt("Refund order A-100.") .requestHook(approvalHook) .send(); ``` Use `.hook(...)` when the rule is a stable part of the agent. ```ts const agent = new AgentBuilder("support", model) .tool(refundOrder) .hook(approvalHook) .build(); ``` ## Rejected Approval [#rejected-approval] When approval is rejected, Studio sends the configured rejection message back to the model as the tool result. ```ts return tool.requestApproval({ reason: "Review this action.", rejectMessage: "The reviewer rejected this action.", }); ``` The model receives this text as the tool result, then it can produce a final answer. ## Timeout Policy [#timeout-policy] Anvia does not require a timeout. Studio approval waits until a reviewer approves or rejects. If your application owns a custom approval system, add a timeout in that app code when the caller needs bounded latency. ```ts const approved = await Promise.race([ approvalRuntime.waitForDecision({ toolName, args }), sleep(60_000).then(() => false), ]); return approved ? tool.run() : tool.skip("No reviewer approved this action in time."); ``` # Introduction (/docs/guides) Anvia is a TypeScript SDK for building agents, tools, structured extraction, retrieval, pipelines, and observability inside your application code. It is designed for teams that want more structure than raw model calls without giving up ownership of data, permissions, persistence, deployment, and side effects. ## Core Primitives [#core-primitives] | Primitive | What it does | Start here | | ------------ | -------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | | Client | Configures provider access, credentials, base URLs, and provider SDK wiring | [Provider Clients and Models](/docs/guides/sdk-fundamentals/clients-and-models) | | Model | Provides a reusable completion or embedding capability | [Provider Clients and Models](/docs/guides/sdk-fundamentals/clients-and-models) | | Agent | Runs prompts with instructions, context, tools, hooks, turn limits, and output schemas | [Creating Agents](/docs/guides/agents/creating-agents) | | Tool | Exposes typed application-owned behavior to agents | [Creating Tools](/docs/guides/tools/creating-tools) | | Extractor | Converts unstructured text into schema-shaped data | [Extractors](/docs/guides/structured-output/extractors) | | Pipeline | Composes functions, agents, extractors, and parallel branches | [Pipeline Builder](/docs/guides/pipelines/pipeline-builder) | | Vector Store | Stores and searches embedded documents for retrieval workflows | [Vector Stores](/docs/guides/retrieval/vector-stores) | | Observer | Records runs, generations, tool calls, usage, and traces | [Observers](/docs/guides/observability/observers) | ## The Basic Shape [#the-basic-shape] Most Anvia workflows follow the same shape: 1. Create a provider client. 2. Create a reusable model. 3. Build an agent around that model. 4. Prompt the agent from application code. 5. Add tools, history, context, structured output, retrieval, or tracing as the workflow grows. ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly and concisely.") .build(); const response = await agent.prompt("How do I reset my password?").send(); console.log(response.output); ``` ## Why Anvia [#why-anvia] Anvia keeps the runtime small and explicit: * TypeScript-first APIs and Zod-backed validation * provider-agnostic model, message, tool, and streaming shapes * application-owned side effects through tools * clear boundaries between clients, models, agents, and requests * testable workflows that can use fake completion models * optional layers for MCP, skills, retrieval, tracing, and Studio The SDK does not need to own your database, auth system, queue, retries, or deployment. Keep those in your product code and pass only the model capabilities, context, and tools a workflow needs. ## Positioning [#positioning] Read [Comparison](/docs/guides/comparison) for a neutral comparison between Anvia, Mastra, OpenAI Agents SDK, Vercel AI SDK, LangChain.js, LlamaIndex.TS, Google ADK, Genkit, VoltAgent, and Claude Agent SDK. Read [Design Philosophy](/docs/guides/design-philosophy) to understand why Anvia favors stable TypeScript objects, application-owned infrastructure, provider-neutral model boundaries, typed contracts, composition, and optional integrations. ## Capabilities [#capabilities] | Capability | Use it when | Docs | | ----------------- | ---------------------------------------------------------------------------- | -------------------------------------------------------------------------- | | Agents | You need a promptable runtime with instructions, tools, context, and history | [Agents](/docs/guides/agents/creating-agents) | | Tools | The model needs to call application-owned behavior | [Tools](/docs/guides/tools/creating-tools) | | History | You need multi-turn conversation state | [Messages and History](/docs/guides/sdk-fundamentals/messages-and-history) | | Structured Output | You need schema-shaped data instead of free-form text | [Structured Output](/docs/guides/structured-output/schemas) | | Pipelines | You need explicit workflow composition | [Pipelines](/docs/guides/pipelines/pipeline-builder) | | Retrieval | You need embeddings, vector search, or dynamic context | [Retrieval](/docs/guides/retrieval/embeddings) | | MCP | You need to connect Model Context Protocol servers | [MCP](/docs/guides/mcp/connections) | | Streaming | You need incremental text, tool, and final events | [Streaming Events](/docs/guides/streaming/streaming-events) | | Observability | You need run, generation, tool, usage, and trace events | [Observers](/docs/guides/observability/observers) | ## Where To Start [#where-to-start] Choose the path that matches what you are building: | Goal | First page | Then read | | ---------------------------- | -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | | Run your first agent | [Getting Started](/docs/guides/getting-started) | [Build an Agent](/docs/guides/learning-paths/build-an-agent) | | Run examples locally | [Cookbook](/docs/guides/cookbook) | [Testing](/docs/guides/testing) | | Understand the SDK shape | [How Anvia Works](/docs/guides/sdk-fundamentals/runtime-boundaries) | [Provider Clients and Models](/docs/guides/sdk-fundamentals/clients-and-models) | | Persist conversations | [Persist Conversations](/docs/guides/learning-paths/persist-conversations) | [Agent History](/docs/guides/agents/agent-history) | | Send images or documents | [Attachments](/docs/guides/sdk-fundamentals/attachments) | [Messages and History](/docs/guides/sdk-fundamentals/messages-and-history) | | Add application actions | [Add Tools](/docs/guides/learning-paths/add-tools) | [Agent Tools](/docs/guides/agents/agent-tools) | | Return typed data | [Return Structured Output](/docs/guides/learning-paths/return-structured-output) | [Agent Output](/docs/guides/structured-output/agent-output) | | Compose multi-step workflows | [Build a Pipeline](/docs/guides/learning-paths/build-a-pipeline) | [Parallel Branches](/docs/guides/pipelines/parallel-branches) | | Add retrieval | [Add Retrieval](/docs/guides/learning-paths/add-retrieval) | [RAG Context](/docs/guides/retrieval/rag-context) | | Add traces | [Add Observability](/docs/guides/learning-paths/add-observability) | [Tracing](/docs/guides/observability/tracing) | | Inspect agents locally | [Studio](/docs/studio/overview) | [Run Studio](/docs/studio/run-studio) | | Prepare to ship | [Prepare for Production](/docs/guides/learning-paths/prepare-for-production) | [Errors and Cancellation](/docs/guides/sdk-fundamentals/errors) | Continue with [Getting Started](/docs/guides/getting-started) for a runnable first agent. # Add Observability (/docs/guides/learning-paths/add-observability) Use this path when you need to inspect what happened during an agent run. ## Goal [#goal] By the end, you should know how to observe: * prompt run start and end * model generation requests and responses * tool calls and tool results * usage data * trace metadata ## Path [#path] 1. Read [Observers](/docs/guides/observability/observers) to attach runtime observers. 2. Read [Trace Groups](/docs/guides/observability/trace-groups) to group related work. 3. Read [Tracing](/docs/guides/observability/tracing) for trace metadata and integrations. 4. Read [Langfuse](/docs/guides/observability/langfuse) to send Anvia traces to Langfuse. 5. Read [Streaming Events](/docs/guides/streaming/streaming-events) if your UI needs live events. 6. Read [Prompt Responses](/docs/guides/sdk-fundamentals/prompt-responses) to understand response usage and trace fields. ## What To Log First [#what-to-log-first] Start with: * agent id * prompt run id or conversation id * model name * total tokens * tool names called * error type and message * trace id when available Do not log sensitive prompt, document, or tool data unless your product policy allows it. ## Minimal Observer Shape [#minimal-observer-shape] Observers are plain TypeScript interfaces, so object literals work. For app code, prefer a class when the observer will keep state, share configuration, or be reused across agents. ```ts import { AgentBuilder, type AgentObserver, type AgentRunObserver, type AgentRunStartArgs } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; class ConsoleObserver implements AgentObserver { startRun(args: AgentRunStartArgs): AgentRunObserver { console.log("run_start", { prompt: args.prompt.role, maxTurns: args.maxTurns, trace: args.trace, }); return { startGeneration({ turn, request }) { console.log("generation_start", { turn, tools: request.tools.map((tool) => tool.name), }); return { end({ response }) { console.log("generation_end", response.usage.totalTokens); }, }; }, startTool({ toolName }) { console.log("tool_start", toolName); return { end({ result, skipped }) { console.log("tool_end", { skipped, result }); }, }; }, end({ usage }) { console.log("run_end", usage.totalTokens); }, error({ error }) { console.error("run_error", error); }, }; } } const model = new OpenAIClient({ apiKey }).completionModel("gpt-5.5"); const observer = new ConsoleObserver(); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .observe(observer) .build(); const response = await agent .prompt("How do I reset my password?") .withTrace({ name: "support-question", userId: "user_123", metadata: { surface: "docs-example" }, }) .send(); console.log(response.trace); ``` ## Add Next [#add-next] | Need | Read | | ----------------------- | ---------------------------------------------------------------------------- | | Send traces to Langfuse | [Langfuse](/docs/guides/observability/langfuse) | | Stream to a UI | [Readable Streams](/docs/guides/streaming/readable-streams) | | Debug tool loops | [Run Limits](/docs/guides/agents/run-limits) | | Production checklist | [Prepare for Production](/docs/guides/learning-paths/prepare-for-production) | # Add Retrieval (/docs/guides/learning-paths/add-retrieval) Use this path when an agent needs searchable knowledge that should not be placed directly in static context. ## Goal [#goal] By the end, you should know how to: * choose an embedding model * embed documents * store vectors * filter results * attach retrieved context to an agent run ## Path [#path] 1. Read [Embeddings](/docs/guides/retrieval/embeddings) to choose an embedding model. 2. Read [Embed Documents](/docs/guides/retrieval/embed-documents) to convert text into vectors. 3. Read [Vector Stores](/docs/guides/retrieval/vector-stores) to store and search embeddings. 4. Read [RAG Context](/docs/guides/retrieval/rag-context) to attach retrieved documents to an agent. 5. Read [Metadata Filters](/docs/guides/retrieval/metadata-filters) when retrieval should respect tenant, user, or document filters. 6. Read [LSH](/docs/guides/retrieval/lsh) when local search needs candidate narrowing. ## Static Context vs Retrieval [#static-context-vs-retrieval] | Use static context when | Use retrieval when | | ------------------------------------- | ----------------------------------------------- | | The text is short | The knowledge base is large | | The same fact applies to every prompt | The relevant facts depend on the prompt | | The content changes rarely | The content is searched, filtered, or refreshed | ## Retrieval Flow [#retrieval-flow] Think about retrieval in two phases: | Phase | What happens | | ---------- | -------------------------------------------------------------------------------- | | Preprocess | Load documents, normalize or chunk text, create embeddings, and build an index. | | Retrieve | Search the index for the current prompt and pass the matches into the agent run. | ## 1. Preprocess Documents [#1-preprocess-documents] Run preprocessing before the user prompt. In a real app, this usually belongs in a build step, startup task, admin action, or background ingestion job. ```ts import { InMemoryVectorStore, embedDocuments } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ apiKey }); const embeddings = client.embeddingModel("text-embedding-3-small"); const documents = [ { id: "password-reset", title: "Password reset policy", body: "Password reset links expire after 30 minutes.", }, { id: "priority-support", title: "Priority support", body: "Enterprise customers receive priority support.", }, ]; const preprocessed = documents.map((doc) => ({ ...doc, title: doc.title.trim(), body: doc.body.replace(/\s+/g, " ").trim(), })); const embedded = await embedDocuments(embeddings, preprocessed, { id: (doc) => doc.id, content: (doc) => `${doc.title}\n${doc.body}`, metadata: (doc) => ({ title: doc.title }), }); export const supportDocs = InMemoryVectorStore.fromDocuments(embedded); export const supportDocsIndex = supportDocs.index(embeddings); ``` For production, store the embedded documents in a durable vector store instead of rebuilding the index on every request. ## 2. Retrieve During a Prompt [#2-retrieve-during-a-prompt] Use the preprocessed index at runtime. The agent receives only the matches for the current prompt. ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { supportDocsIndex } from "./support-docs"; const model = new OpenAIClient({ apiKey }).completionModel("gpt-5.5"); const agent = new AgentBuilder("support", model) .instructions("Use retrieved context when answering.") .dynamicContext(supportDocsIndex, { topK: 2, format: (result) => ({ id: result.id, text: `${result.document.title}\n${result.document.body}`, }), }) .build(); const response = await agent.prompt("How long does a reset link last?").send(); console.log(response.output); ``` ## Search Tool Shape [#search-tool-shape] Use the same preprocessed index as a tool when the model should decide when to search. ```ts const searchDocs = supportDocsIndex.asTool({ name: "search_docs", description: "Search support documentation.", topK: 3, }); const agent = new AgentBuilder("support", model) .instructions("Search docs before answering policy questions.") .tool(searchDocs) .defaultMaxTurns(3) .build(); ``` ## Add Next [#add-next] | Need | Read | | --------------------------- | ------------------------------------------------------------------------------- | | Local vector search | [LSH](/docs/guides/retrieval/lsh) | | Provider embeddings | [Provider Clients and Models](/docs/guides/sdk-fundamentals/clients-and-models) | | Tracing retrieval workflows | [Add Observability](/docs/guides/learning-paths/add-observability) | # Add Tools (/docs/guides/learning-paths/add-tools) Use this path when the model needs to inspect data, call services, or perform actions owned by your application. ## Goal [#goal] By the end, you should have: * one Zod-backed tool * an agent with that tool registered * a low turn limit * a clear permission boundary around side effects ## Path [#path] 1. Read [Creating Tools](/docs/guides/tools/creating-tools) to define a tool with input validation. 2. Read [Tool Schemas](/docs/guides/tools/tool-schemas) to understand how Zod schemas become provider tool definitions. 3. Read [Agent Tools](/docs/guides/agents/agent-tools) to register tools on an agent. 4. Read [Tool Results](/docs/guides/tools/tool-results) to understand what gets sent back to the model. 5. Read [Tool Errors](/docs/guides/tools/tool-errors) to decide when to return an expected result and when to throw. ## Minimal Shape [#minimal-shape] ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; const lookupOrder = createTool({ name: "lookup_order", description: "Look up an order by order id.", input: z.object({ orderId: z.string(), }), async execute({ orderId }) { return { orderId, status: "shipped" }; }, }); const model = new OpenAIClient({ apiKey }).completionModel("gpt-5.5"); const agent = new AgentBuilder("support", model) .instructions("Use tools when order status is needed.") .tool(lookupOrder) .defaultMaxTurns(3) .build(); ``` ## Production Notes [#production-notes] * Keep permission checks inside your tool code. * Return explicit values for expected product states such as `not_found`. * Throw only when the workflow should fail or be retried. * Start with low turn limits and increase only when the workflow needs it. ## Add Next [#add-next] | Need | Read | | -------------- | ----------------------------------------------------------- | | Multiple tools | [Tool Sets](/docs/guides/tools/tool-sets) | | Human approval | [Human in the Loop](/docs/guides/human-in-the-loop) | | MCP tools | [Server Tools](/docs/guides/mcp/server-tools) | | Tool events | [Streaming Events](/docs/guides/streaming/streaming-events) | # Build a Pipeline (/docs/guides/learning-paths/build-a-pipeline) Use this path when one prompt is not enough and the workflow needs explicit steps. ## Goal [#goal] By the end, you should know how to: * create a pipeline * add transform steps * call agents from a pipeline * run extractors * add parallel branches when work can happen independently ## Path [#path] 1. Read [Pipeline Builder](/docs/guides/pipelines/pipeline-builder) for the core API. 2. Read [Steps](/docs/guides/pipelines/steps) for ordinary transform steps. 3. Read [Prompt Steps](/docs/guides/pipelines/prompt-steps) to call agents. 4. Read [Extractor Steps](/docs/guides/pipelines/extractor-steps) to return typed data. 5. Read [Parallel Branches](/docs/guides/pipelines/parallel-branches) when independent work can run side by side. 6. Read [Composition Patterns](/docs/guides/pipelines/composition-patterns) for larger workflows. ## When To Use a Pipeline [#when-to-use-a-pipeline] Use a pipeline when the workflow has named stages that should be easy to test: * normalize input * ask an agent * extract fields * enrich with a tool or database call * run parallel checks * return a final object Do not use a pipeline just to send one prompt. Start with an agent and add a pipeline when the workflow needs shape. ## Minimal Shape [#minimal-shape] ```ts import { PipelineBuilder } from "@anvia/core"; type TicketInput = { customer: string; subject: string; body: string; }; const pipeline = new PipelineBuilder() .step((ticket) => ({ customer: ticket.customer.trim(), subject: ticket.subject.trim(), body: ticket.body.trim(), })) .step((ticket) => ({ title: ticket.subject.toLowerCase(), customer: ticket.customer, words: ticket.body.split(/\s+/).length, })) .build(); const result = await pipeline.run({ customer: " Acme Co. ", subject: " Checkout is failing ", body: "Enterprise checkout fails after payment retries.", }); console.log(result.title); ``` ## Agent and Extractor Shape [#agent-and-extractor-shape] ```ts import { AgentBuilder, ExtractorBuilder, PipelineBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; const model = new OpenAIClient({ apiKey }).completionModel("gpt-5.5"); const summarizer = new AgentBuilder("summarizer", model) .instructions("Summarize support tickets in one paragraph.") .build(); const extractor = new ExtractorBuilder( model, z.object({ priority: z.enum(["low", "medium", "high"]), summary: z.string(), }), ).build(); const pipeline = new PipelineBuilder() .step( (ticket) => [ `Customer: ${ticket.customer}`, `Subject: ${ticket.subject}`, `Body: ${ticket.body}`, ].join("\n"), ) .prompt(summarizer) .extract(extractor) .build(); const result = await pipeline.run({ customer: "Acme Co.", subject: "Checkout is failing", body: "Checkout is failing for all enterprise users.", }); console.log(result.priority); ``` For object inputs, format the prompt in a step before `.prompt(...)` or `.extract(...)`. ## Parallel Branch Shape [#parallel-branch-shape] ```ts const checks = new PipelineBuilder() .step((ticket) => ticket.body) .parallel({ uppercase: new PipelineBuilder() .step((value) => value.toUpperCase()) .build(), length: new PipelineBuilder() .step((value) => value.length) .build(), urgent: new PipelineBuilder() .step((value) => value.toLowerCase().includes("failing")) .build(), }) .build(); const result = await checks.run({ customer: "Acme Co.", subject: "Checkout is failing", body: "Checkout is failing for all enterprise users.", }); console.log(result.uppercase, result.length, result.urgent); ``` ## Add Next [#add-next] | Need | Read | | --------------- | -------------------------------------------------------------------------------- | | Many inputs | [Batch Runs](/docs/guides/pipelines/batch-runs) | | Agents as steps | [Agents in Pipelines](/docs/guides/pipelines/agents-in-pipelines) | | Typed outputs | [Return Structured Output](/docs/guides/learning-paths/return-structured-output) | # Build an Agent (/docs/guides/learning-paths/build-an-agent) Use this path when you want to run a model through an Anvia agent with clear instructions and a stable runtime id. ## Goal [#goal] By the end, you should have: * a provider client * a reusable completion model * an agent with instructions * one prompt request that returns a final response ## Path [#path] 1. Install Anvia and run the first agent in [Getting Started](/docs/guides/getting-started). 2. Read [How Anvia Works](/docs/guides/sdk-fundamentals/runtime-boundaries) to understand which object owns which responsibility. 3. Read [Provider Clients and Models](/docs/guides/sdk-fundamentals/clients-and-models) to choose the provider client and model. 4. Read [Creating Agents](/docs/guides/agents/creating-agents) to configure the agent. 5. Read [Prompt Requests](/docs/guides/sdk-fundamentals/prompt-requests) to understand what happens when you call `agent.prompt(...).send()`. ## Minimal Shape [#minimal-shape] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("How do I reset my password?").send(); console.log(response.output); ``` ## Add Next [#add-next] | Need | Read | | ------------------------ | -------------------------------------------------------------------------------- | | Multi-turn conversations | [Persist Conversations](/docs/guides/learning-paths/persist-conversations) | | Application actions | [Add Tools](/docs/guides/learning-paths/add-tools) | | Typed responses | [Return Structured Output](/docs/guides/learning-paths/return-structured-output) | | Streaming output | [Streaming Events](/docs/guides/streaming/streaming-events) | # Persist Conversations (/docs/guides/learning-paths/persist-conversations) Use this path when an agent needs to remember previous turns. ## Goal [#goal] By the end, you should know: * the `Message[]` history shape * how to pass an explicit transcript into a prompt * how to use durable session memory * how to append `response.messages` * where tool calls and tool results appear in history ## Path [#path] 1. Read [Messages and History](/docs/guides/sdk-fundamentals/messages-and-history) to understand the raw `Message[]` shape. 2. Read [Prompt Responses](/docs/guides/sdk-fundamentals/prompt-responses) to understand `response.messages`. 3. Read [Memory and Sessions](/docs/guides/sdk-fundamentals/memory-and-sessions) for the core-managed durable conversation model. 4. Read [Memory](/docs/guides/memory) for raw SQL, Prisma, and Drizzle storage adapters. 5. Read [Agent History](/docs/guides/agents/agent-history) for agent-specific history examples. 6. Read [Messages and History](/docs/guides/sdk-fundamentals/messages-and-history) if your history includes attachments or rich content. ## Minimal Shape [#minimal-shape] ```ts import { Message } from "@anvia/core"; const history = await conversations.loadMessages(conversationId); const currentPrompt = Message.user(userInput); const response = await agent.prompt([...history, currentPrompt]).send(); await conversations.saveMessages(conversationId, [ ...history, ...response.messages, ]); ``` ## Key Rule [#key-rule] `response.messages` is only the new part of the run. Append it to the history you loaded if you want a full transcript. For core-managed durable conversations, configure memory and prompt through a session: ```ts const response = await agent.session(conversationId).prompt(userInput).send(); ``` ## Add Next [#add-next] | Need | Read | | ------------------------- | -------------------------------------------------------------------------- | | Tool-call history | [Messages and History](/docs/guides/sdk-fundamentals/messages-and-history) | | Streaming conversation UI | [Readable Streams](/docs/guides/streaming/readable-streams) | | Long-term knowledge | [Add Retrieval](/docs/guides/learning-paths/add-retrieval) | # Prepare for Production (/docs/guides/learning-paths/prepare-for-production) Use this path when an Anvia workflow is moving from a prototype into product code. ## Goal [#goal] By the end, you should have reviewed: * provider configuration * tool permissions * history persistence * turn limits * structured output validation * retries and error handling * logging and traces ## Checklist [#checklist] | Area | Check | | ----------------- | ----------------------------------------------------------------------------------------------------------- | | Providers | Create clients and models once, configure keys through your normal secret system | | Tools | Enforce auth and permissions inside tool code | | History | Persist `Message[]` and append `response.messages` | | Turn limits | Keep limits low enough to prevent unbounded tool loops | | Structured output | Validate output before using it in product workflows | | Retrieval | Filter by tenant, user, or document ownership when needed | | Observability | Log run metadata, usage, tool calls, errors, and trace ids | | Errors | Classify setup, provider, validation, tool, and runtime-limit failures | | Testing | Cover tools, deterministic pipeline steps, retrieval filters, and Studio routes before broad provider tests | ## Path [#path] 1. Read [How Anvia Works](/docs/guides/sdk-fundamentals/runtime-boundaries) to confirm ownership is clear. 2. Read [Errors and Cancellation](/docs/guides/sdk-fundamentals/errors) to plan failure handling. 3. Read [Human in the Loop](/docs/guides/human-in-the-loop) for guarded actions. 4. Read [Messages and History](/docs/guides/sdk-fundamentals/messages-and-history) for persistence shape. 5. Read [Output Validation](/docs/guides/structured-output/output-validation) for typed workflows. 6. Read [Observers](/docs/guides/observability/observers) for runtime visibility. 7. Read [Testing](/docs/guides/testing) for verification boundaries. ## Practical Rule [#practical-rule] Keep product-critical decisions in your application code. Use Anvia to coordinate prompts, tools, context, retrieval, output shape, and observability around those decisions. ## Deployment Shape [#deployment-shape] Create provider clients, models, and long-lived agents at application startup when possible. Keep per-request data in the prompt request: | Startup-owned | Request-owned | | ---------------------------------------- | -------------------------------------- | | provider clients and model instances | user input and message history | | reusable agents, tools, and pipelines | trace metadata and session ids | | durable stores and connection registries | request-specific turn limits and hooks | If a workflow runs in a queue or background worker, pass the same explicit inputs you would pass in an HTTP route: current message, stored history, product metadata, and any request-local authorization context. ## Usage and Rate Limits [#usage-and-rate-limits] Use `response.usage` and trace metadata to build product-level controls: ```ts await usageEvents.record({ userId: options.userId, inputTokens: response.usage.inputTokens, outputTokens: response.usage.outputTokens, totalTokens: response.usage.totalTokens, traceId: response.trace?.traceId, }); ``` Keep retry policy in application code. Provider failures, rate limits, and tool failures need different product decisions, so do not hide them behind a generic retry loop. ## Runtime Wrapper Shape [#runtime-wrapper-shape] Put agent calls behind a small application-owned wrapper so logging, history, trace metadata, and error handling are consistent. ```ts import { MaxTurnsError, Message, PromptCancelledError, type Agent, type Message as MessageType, } from "@anvia/core"; type RunSupportAgentOptions = { userId: string; conversationId: string; input: string; history: MessageType[]; }; export async function runSupportAgent( agent: Agent, options: RunSupportAgentOptions, ) { try { const response = await agent .prompt([...options.history, Message.user(options.input)]) .withTrace({ name: "support-agent", userId: options.userId, sessionId: options.conversationId, }) .maxTurns(3) .send(); await conversations.saveMessages(options.conversationId, [ ...options.history, ...response.messages, ]); await usageEvents.record({ userId: options.userId, totalTokens: response.usage.totalTokens, traceId: response.trace?.traceId, }); return response.output; } catch (error) { if (error instanceof MaxTurnsError) { await incidents.record("support-agent-max-turns", error); return "The support workflow could not finish safely."; } if (error instanceof PromptCancelledError) { return "The support request was cancelled."; } await incidents.record("support-agent-failed", error); throw error; } } ``` Keep the wrapper small. It should coordinate application policy around the agent, not hide all SDK concepts from the rest of the codebase. # Return Structured Output (/docs/guides/learning-paths/return-structured-output) Use this path when a workflow needs JSON-shaped data your application can trust. ## Goal [#goal] By the end, you should know when to use: * output schemas on agents * extractors for schema-first extraction * validation and failure handling ## Path [#path] 1. Read [Schemas](/docs/guides/structured-output/schemas) to define the target shape. 2. Read [Zod Schema](/docs/guides/structured-output/zod-schema) to understand schema conversion. 3. Read [Agent Output](/docs/guides/structured-output/agent-output) when the agent should respond in a structured shape. 4. Read [Extractors](/docs/guides/structured-output/extractors) when you need to extract data from existing text. 5. Read [Output Validation](/docs/guides/structured-output/output-validation) and [Failure Handling](/docs/guides/structured-output/failure-handling). ## Choosing the Primitive [#choosing-the-primitive] | Need | Use | | ------------------------------------------- | ------------------- | | The agent should produce typed final output | Agent output schema | | Existing text should be converted into data | Extractor | | A pipeline step should normalize data | Extractor step | | Tool output should be validated | Tool output schema | ## Minimal Extractor Shape [#minimal-extractor-shape] Use an extractor when you have text and want validated data back. ```ts import { ExtractorBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; const ticketSchema = z.object({ customer: z.string(), priority: z.enum(["low", "medium", "high"]), summary: z.string(), }); const model = new OpenAIClient({ apiKey }).completionModel("gpt-5.5"); const extractor = new ExtractorBuilder(model, ticketSchema) .instructions("Extract support ticket fields.") .retries(1) .build(); const ticket = await extractor.extract(` Acme Co. reports checkout failures for all users. The issue is urgent and blocking revenue. `); console.log(ticket.priority); ``` ## Agent Output Shape [#agent-output-shape] Use an agent output schema when the agent itself should produce structured final output. ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; const model = new OpenAIClient({ apiKey }).completionModel("gpt-5.5"); const agent = new AgentBuilder("classifier", model) .instructions("Classify support messages.") .outputSchema( z.object({ category: z.enum(["billing", "technical", "account"]), confidence: z.number(), }), ) .build(); const response = await agent.prompt("I cannot update my payment method.").send(); ``` ## Add Next [#add-next] | Need | Read | | --------------------------- | --------------------------------------------------------------- | | Extraction inside workflows | [Extractor Steps](/docs/guides/pipelines/extractor-steps) | | Structured tool results | [Tool Results](/docs/guides/tools/tool-results) | | Schema errors | [Errors and Cancellation](/docs/guides/sdk-fundamentals/errors) | # Connection Registry (/docs/guides/mcp/connection-registry) Anvia does not keep a global MCP registry. Your app should own connection storage and lifecycle. ## App-Owned Registry [#app-owned-registry] ```ts import type { McpServer } from "@anvia/core"; class McpRegistry { private readonly servers = new Map(); add(server: McpServer) { this.servers.set(server.name, server); } get(name: string): McpServer | undefined { return this.servers.get(name); } values(): McpServer[] { return [...this.servers.values()]; } async closeAll() { await Promise.all([...this.servers.values()].map((server) => server.close())); this.servers.clear(); } } ``` Use a registry when multiple agents need the same MCP connections. ## Startup Shape [#startup-shape] ```ts const registry = new McpRegistry(); registry.add( await connectMcp( mcp.stdio({ name: "filesystem", command: "npx", args: ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"], }), ), ); registry.add( await connectMcp( mcp.http({ name: "docs", url: "https://mcp.example.com/mcp", }), ), ); ``` ## Use Registered Servers [#use-registered-servers] ```ts const agent = new AgentBuilder("support", model) .mcp(registry.values()) .defaultMaxTurns(4) .build(); ``` If different agents need different MCP tools, filter the registry values before registering them. ```ts const agent = new AgentBuilder("docs-agent", model) .mcp([registry.get("docs")].filter((server): server is McpServer => server !== undefined)) .build(); ``` ## Shutdown [#shutdown] ```ts process.on("SIGTERM", async () => { await registry.closeAll(); }); ``` Keep connection lifecycle explicit so reconnects, shutdown, and resource ownership are easy to reason about. # Connections (/docs/guides/mcp/connections) Anvia can connect to Model Context Protocol servers and expose their tools to agents. ## Connect to a Stdio Server [#connect-to-a-stdio-server] ```ts import { connectMcp, mcp } from "@anvia/core"; const filesystem = await connectMcp( mcp.stdio({ name: "filesystem", command: "npx", args: ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"], }), ); ``` `connectMcp(...)` connects once, lists the server tools, and returns an `McpServer`. ## Connect to an HTTP Server [#connect-to-an-http-server] ```ts const docsServer = await connectMcp( mcp.http({ name: "docs", url: "https://mcp.example.com/mcp", }), ); ``` Use HTTP when the MCP server is remote or managed outside the current process. ## Use the Server on an Agent [#use-the-server-on-an-agent] ```ts const agent = new AgentBuilder("support", model) .instructions("Use MCP tools when external context is needed.") .mcp([filesystem, docsServer]) .defaultMaxTurns(4) .build(); ``` MCP tools behave like normal Anvia tools once registered on the agent. ## Close Connections [#close-connections] Close MCP servers when the process or request scope ends. ```ts await filesystem.close(); await docsServer.close(); ``` For long-lived application processes, connect during startup and close during shutdown. # Errors and Reconnects (/docs/guides/mcp/errors-and-reconnects) MCP failures can happen while connecting, listing tools, calling tools, or closing the transport. ## Connect Errors [#connect-errors] `connectMcp(...)` connects and lists tools immediately. ```ts try { const docsServer = await connectMcp( mcp.http({ name: "docs", url: "https://mcp.example.com/mcp", }), ); } catch (error) { logger.error(error, "failed to connect MCP server"); } ``` If startup requires that server, fail fast. If the server is optional, build the agent without it and retry in the background. ## Tool Call Errors [#tool-call-errors] MCP tool errors are treated like tool errors. The model receives the failure text and can continue the run. ```ts const agent = new AgentBuilder("support", model) .mcp([docsServer]) .defaultMaxTurns(3) .build(); ``` Keep turn limits low so a failing MCP tool cannot create an unbounded retry loop. ## Reconnect Shape [#reconnect-shape] Anvia does not manage reconnects for you. Own that in your application registry. ```ts import type { McpConnection, McpServer } from "@anvia/core"; type McpRegistry = { get(name: string): McpServer | undefined; add(server: McpServer): void; }; async function reconnect(connection: McpConnection, registry: McpRegistry) { const previous = registry.get(connection.name); await previous?.close(); const next = await connectMcp(connection); registry.add(next); } ``` After reconnecting, build new agents or update the `ToolSet` used by your agents. ## Cleanup [#cleanup] Always close long-lived MCP servers during shutdown. ```ts await registry.closeAll(); ``` For request-scoped connections, use `try/finally`. ```ts const server = await connectMcp(connection); try { return await runAgent(server); } finally { await server.close(); } ``` # Result Handling (/docs/guides/mcp/result-handling) Anvia normalizes MCP tool results into strings before sending them back to the model. ## Text Results [#text-results] MCP text content is joined together. ```ts { content: [ { type: "text", text: "First line." }, { type: "text", text: "Second line." } ] } ``` The model receives: ```txt First line.Second line. ``` If you need separators, return them from the MCP server as part of the text. ## Structured Results [#structured-results] Some MCP servers return `toolResult`. Anvia serializes it like normal tool output. ```ts { toolResult: { status: "ok", count: 3 } } ``` The model receives: ```json {"status":"ok","count":3} ``` ## Images and Resources [#images-and-resources] Image content becomes a data URL. ```txt data:image/png;base64,... ``` Resource content becomes: ```txt data:text/plain;file:///tmp/report.txt:contents ``` Use these result types only when the downstream model and provider can handle the content meaningfully. ## MCP Error Results [#mcp-error-results] If the MCP server returns `isError: true`, Anvia throws using the text content as the error message. ```ts { isError: true, content: [{ type: "text", text: "Access denied." }] } ``` During an agent run, tool errors are returned to the model as tool result text, the same as local tool errors. # Server Tools (/docs/guides/mcp/server-tools) `connectMcp(...)` lists tools from the MCP server and converts them into Anvia tools. ## Register Stdio MCP Tools [#register-stdio-mcp-tools] Use stdio when your app starts and owns the MCP server process. ```ts const mathServer = await connectMcp( mcp.stdio({ name: "math", command: "node", args: ["./mcp-math-server.js"], }), ); const agent = new AgentBuilder("calculator", model) .instructions("Use math tools for arithmetic.") .mcp([mathServer]) .defaultMaxTurns(3) .build(); ``` The model sees the MCP tools in the same tool list as local Anvia tools. ## Register HTTP MCP Tools [#register-http-mcp-tools] Use HTTP when the MCP server is remote or managed outside your process. ```ts const docsServer = await connectMcp( mcp.http({ name: "docs", url: "https://mcp.example.com/mcp", }), ); const agent = new AgentBuilder("support", model) .instructions("Use docs tools when product documentation is needed.") .mcp([docsServer]) .defaultMaxTurns(3) .build(); ``` The agent registration is the same for stdio and HTTP. The connection shape is the only difference. ## Mix Local and MCP Tools [#mix-local-and-mcp-tools] ```ts const agent = new AgentBuilder("support", model) .tool(lookupOrder) .mcp([docsServer]) .defaultMaxTurns(4) .build(); ``` Use local tools for application-owned behavior. Use MCP tools for capabilities owned by external servers. ## Inspect Tools [#inspect-tools] ```ts for (const tool of docsServer.tools) { console.log(await tool.definition("")); } ``` Each MCP tool has a name, description, and JSON Schema input definition from the MCP server. ## Tool Calls [#tool-calls] When the model calls an MCP tool, Anvia forwards JSON object arguments to the MCP client. ```ts await docsServer.tools[0]?.call({ query: "agent history", }); ``` During agent runs, Anvia handles this call automatically and sends the normalized result back to the model. # Tool Adapters (/docs/guides/mcp/tool-adapters) MCP tools are adapted automatically by `connectMcp(...)`. ## What Gets Adapted [#what-gets-adapted] An MCP tool definition: ```ts { name: "search_docs", description: "Search documentation.", inputSchema: { type: "object", properties: { query: { type: "string" } }, required: ["query"] } } ``` becomes an Anvia tool definition: ```ts { name: "search_docs", description: "Search documentation.", parameters: { type: "object", properties: { query: { type: "string" } }, required: ["query"] } } ``` The MCP `inputSchema` is passed through as the provider-facing `parameters`. ## Calls [#calls] When the adapted tool is called, Anvia sends the MCP server: ```ts { name: "search_docs", arguments: { query: "agent history" } } ``` Arguments must be a JSON object. Non-object arguments are rejected before the MCP call. ## When You Need a Local Adapter [#when-you-need-a-local-adapter] If you want to rename, filter, or wrap MCP behavior, create a normal Anvia tool that calls your MCP server or service. ```ts const searchInternalDocs = createTool({ name: "search_internal_docs", description: "Search internal support documentation.", input: z.object({ query: z.string() }), async execute({ query }) { return docsServer.tools .find((tool) => tool.name === "search_docs") ?.call({ query }); }, }); ``` Prefer the automatic adapter unless you need product-specific names, filtering, auditing, or permissions. # Drizzle (/docs/guides/memory/drizzle) This example uses Drizzle with PostgreSQL. Store messages as `jsonb`, then load them by `sessionId` in insertion order. ## Tables [#tables] ```ts import { index, integer, jsonb, pgTable, text, timestamp, bigserial } from "drizzle-orm/pg-core"; import type { Message } from "@anvia/core"; export const agentMemoryMessages = pgTable( "agent_memory_messages", { id: bigserial("id", { mode: "number" }).primaryKey(), sessionId: text("session_id").notNull(), userId: text("user_id"), runId: text("run_id").notNull(), turn: integer("turn").notNull(), message: jsonb("message").$type().notNull(), metadata: jsonb("metadata"), createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(), }, (table) => ({ sessionOrderIdx: index("agent_memory_messages_session_id_id_idx").on( table.sessionId, table.id, ), }), ); export const agentMemoryErrors = pgTable( "agent_memory_errors", { id: bigserial("id", { mode: "number" }).primaryKey(), sessionId: text("session_id").notNull(), userId: text("user_id"), runId: text("run_id").notNull(), error: jsonb("error").notNull(), messages: jsonb("messages").$type().notNull(), createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(), }, (table) => ({ sessionCreatedIdx: index("agent_memory_errors_session_id_created_at_idx").on( table.sessionId, table.createdAt, ), }), ); ``` ## Store [#store] ```ts import type { MemoryAppendInput, MemoryContext, MemoryErrorInput, MemoryStore, Message, } from "@anvia/core"; import { asc, eq } from "drizzle-orm"; import type { NodePgDatabase } from "drizzle-orm/node-postgres"; import { agentMemoryErrors, agentMemoryMessages } from "./schema"; export class DrizzleMemoryStore implements MemoryStore { constructor(private readonly db: NodePgDatabase) {} async load(context: MemoryContext): Promise { const rows = await this.db .select({ message: agentMemoryMessages.message }) .from(agentMemoryMessages) .where(eq(agentMemoryMessages.sessionId, context.sessionId)) .orderBy(asc(agentMemoryMessages.id)); return rows.map((row) => row.message); } async append(input: MemoryAppendInput): Promise { if (input.messages.length === 0) { return; } await this.db.insert(agentMemoryMessages).values( input.messages.map((message) => ({ sessionId: input.context.sessionId, userId: input.context.userId, runId: input.runId, turn: input.turn, message, metadata: input.context.metadata, })), ); } async clear(context: MemoryContext): Promise { await this.db .delete(agentMemoryMessages) .where(eq(agentMemoryMessages.sessionId, context.sessionId)); } async recordError(input: MemoryErrorInput): Promise { await this.db.insert(agentMemoryErrors).values({ sessionId: input.context.sessionId, userId: input.context.userId, runId: input.runId, error: serializeError(input.error), messages: input.messages, }); } } function serializeError(error: unknown): Record { if (error instanceof Error) { return { name: error.name, message: error.message, stack: error.stack, }; } return { message: String(error) }; } ``` ## Use It [#use-it] ```ts const memory = new DrizzleMemoryStore(db); const agent = new AgentBuilder("support", model) .memory(memory) .build(); await agent.session("thread_123", { userId: "user_456" }).prompt("Hello").send(); ``` # Memory (/docs/guides/memory) Memory is Anvia's durable conversation API. Core owns when to load and append messages; your application owns where those messages are stored. ```ts const agent = new AgentBuilder("support", model) .memory(memoryStore) .build(); const response = await agent .session("thread_123", { userId: "user_456" }) .prompt("Continue from earlier.") .send(); ``` ## Mental Model [#mental-model] Use `agent.prompt("...")` for stateless one-off requests. Use `agent.prompt([...messages])` when your application already owns an explicit transcript. The last message is the active prompt and earlier messages are temporary request history. Use `agent.session(id).prompt("...")` when Anvia should load and save durable conversation messages through the configured memory store. Memory stores model transcript messages for future context. It does not store runtime stream events such as text deltas, tool progress, or child-agent events from `asTool({ stream: true })`. Use [Event Store](/docs/guides/agents/event-store) when you need replay/debug history for runtime events. ## Public API [#public-api] ```ts type MemorySavePolicy = "message" | "turn" | "run"; type MemoryContext = { sessionId: string; userId?: string; metadata?: JsonObject; }; interface MemoryStore { load(context: MemoryContext): Promise; append(input: { context: MemoryContext; runId: string; turn: number; messages: Message[]; }): Promise; clear(context: MemoryContext): Promise; recordError?(input: { context: MemoryContext; runId: string; error: unknown; messages: Message[]; }): Promise; } type MemoryOptions = { savePolicy?: MemorySavePolicy; }; ``` Configure the store and optional save policy on the agent: ```ts const agent = new AgentBuilder("support", model) .memory(memoryStore, { savePolicy: "message" }) .build(); ``` ## Save Policy [#save-policy] Memory defaults to `savePolicy: "message"`. | Policy | Behavior | | ----------- | --------------------------------------------------------------------------------------------------- | | `"message"` | Save the user prompt, completed assistant messages, and completed tool result messages immediately. | | `"turn"` | Save completed messages after each model/tool turn. | | `"run"` | Save only after a successful final response. | On failure, stores that implement `recordError(...)` receive the error and partial run messages. For delegation-specific behavior, read [Multi-Agent Memory](/docs/guides/memory/multi-agent). ## Adapter Examples [#adapter-examples] Choose the adapter style that matches your application: | Storage style | Guide | | ----------------------------------- | -------------------------------------- | | SQL client and hand-written queries | [Raw SQL](/docs/guides/memory/raw-sql) | | Prisma ORM | [Prisma](/docs/guides/memory/prisma) | | Drizzle ORM | [Drizzle](/docs/guides/memory/drizzle) | # Multi-Agent Memory (/docs/guides/memory/multi-agent) When an agent uses another agent through `asTool(...)`, memory still follows the active prompt request. ```ts const supportAgent = new AgentBuilder("support", model) .instructions("Return support triage notes.") .build(); const coordinator = new AgentBuilder("coordinator", model) .memory(memoryStore) .tool(supportAgent.asTool({ name: "ask_support_agent" })) .build(); await coordinator.session("thread_123").prompt("Triage this incident.").send(); ``` In this setup, `memoryStore` saves the coordinator session transcript: | Message | Saved in coordinator memory | | ------------------------------------------- | --------------------------- | | User prompt | Yes | | Coordinator assistant tool call | Yes | | Final `ask_support_agent` tool result | Yes | | Coordinator final answer | Yes | | Support agent internal text deltas or turns | No | The specialist result is saved as the parent tool result because that is the content the coordinator model receives on the next turn. The specialist's internal run is not appended to the coordinator session as separate user/assistant messages. ## Streaming Agent Tools [#streaming-agent-tools] Streaming does not change memory behavior: ```ts const coordinator = new AgentBuilder("coordinator", model) .memory(memoryStore) .tool(supportAgent.asTool({ name: "ask_support_agent", stream: true })) .build(); ``` With `stream: true`, the caller can see child-agent progress as `agent_tool_event` stream events, but memory still stores only transcript messages and final tool results. Use [Event Store](/docs/guides/agents/event-store) if you need to persist those nested runtime events. ## Specialist Memory [#specialist-memory] If a specialist needs its own durable history, configure memory on that specialist and run it through a session from application code: ```ts const supportAgent = new AgentBuilder("support", model) .memory(memoryStore) .build(); const response = await supportAgent .session("support_thread_123") .prompt("Continue support investigation.") .send(); ``` `agent.asTool(...)` prompts the child agent directly, so it does not automatically create a child session. For manager/specialist workflows, keep the coordinator session as the durable user-facing conversation, and use explicit specialist sessions only when the specialist has its own long-lived thread. # Prisma (/docs/guides/memory/prisma) Use Prisma when your application already stores conversation data through a Prisma client. The key is to store each Anvia `Message` as JSON and load messages in insertion order. ## Prisma Schema [#prisma-schema] ```prisma model AgentMemoryMessage { id BigInt @id @default(autoincrement()) sessionId String userId String? runId String turn Int message Json metadata Json? createdAt DateTime @default(now()) @@index([sessionId, id]) } model AgentMemoryError { id BigInt @id @default(autoincrement()) sessionId String userId String? runId String error Json messages Json createdAt DateTime @default(now()) @@index([sessionId, createdAt]) } ``` ## Store [#store] ```ts import type { MemoryAppendInput, MemoryContext, MemoryErrorInput, MemoryStore, Message, } from "@anvia/core"; import { Prisma, PrismaClient } from "@prisma/client"; export class PrismaMemoryStore implements MemoryStore { constructor(private readonly prisma: PrismaClient) {} async load(context: MemoryContext): Promise { const rows = await this.prisma.agentMemoryMessage.findMany({ where: { sessionId: context.sessionId }, orderBy: { id: "asc" }, select: { message: true }, }); return rows.map((row) => row.message as unknown as Message); } async append(input: MemoryAppendInput): Promise { await this.prisma.agentMemoryMessage.createMany({ data: input.messages.map((message) => ({ sessionId: input.context.sessionId, userId: input.context.userId, runId: input.runId, turn: input.turn, message: toPrismaJson(message), metadata: input.context.metadata === undefined ? undefined : toPrismaJson(input.context.metadata), })), }); } async clear(context: MemoryContext): Promise { await this.prisma.agentMemoryMessage.deleteMany({ where: { sessionId: context.sessionId }, }); } async recordError(input: MemoryErrorInput): Promise { await this.prisma.agentMemoryError.create({ data: { sessionId: input.context.sessionId, userId: input.context.userId, runId: input.runId, error: toPrismaJson(serializeError(input.error)), messages: toPrismaJson(input.messages), }, }); } } function toPrismaJson(value: unknown): Prisma.InputJsonValue { return JSON.parse(JSON.stringify(value)) as Prisma.InputJsonValue; } function serializeError(error: unknown): Record { if (error instanceof Error) { return { name: error.name, message: error.message, stack: error.stack, }; } return { message: String(error) }; } ``` ## Use It [#use-it] ```ts const prisma = new PrismaClient(); const memory = new PrismaMemoryStore(prisma); const agent = new AgentBuilder("support", model) .memory(memory, { savePolicy: "message" }) .build(); await agent.session("thread_123", { userId: "user_456" }).prompt("Hello").send(); ``` # Raw SQL (/docs/guides/memory/raw-sql) This example uses PostgreSQL and the `pg` client. Store Anvia messages as JSONB and order them by an auto-incrementing id. ## Schema [#schema] ```sql CREATE TABLE agent_memory_messages ( id BIGSERIAL PRIMARY KEY, session_id TEXT NOT NULL, user_id TEXT, run_id TEXT NOT NULL, turn INTEGER NOT NULL, message JSONB NOT NULL, metadata JSONB, created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); CREATE INDEX agent_memory_messages_session_id_id_idx ON agent_memory_messages (session_id, id); CREATE TABLE agent_memory_errors ( id BIGSERIAL PRIMARY KEY, session_id TEXT NOT NULL, user_id TEXT, run_id TEXT NOT NULL, error JSONB NOT NULL, messages JSONB NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT now() ); ``` ## Store [#store] ```ts import type { MemoryAppendInput, MemoryContext, MemoryErrorInput, MemoryStore, Message, } from "@anvia/core"; import type { Pool } from "pg"; export class SqlMemoryStore implements MemoryStore { constructor(private readonly pool: Pool) {} async load(context: MemoryContext): Promise { const result = await this.pool.query<{ message: Message }>( `SELECT message FROM agent_memory_messages WHERE session_id = $1 ORDER BY id ASC`, [context.sessionId], ); return result.rows.map((row) => row.message); } async append(input: MemoryAppendInput): Promise { const client = await this.pool.connect(); try { await client.query("BEGIN"); for (const message of input.messages) { await client.query( `INSERT INTO agent_memory_messages ( session_id, user_id, run_id, turn, message, metadata ) VALUES ($1, $2, $3, $4, $5::jsonb, $6::jsonb)`, [ input.context.sessionId, input.context.userId ?? null, input.runId, input.turn, JSON.stringify(message), JSON.stringify(input.context.metadata ?? null), ], ); } await client.query("COMMIT"); } catch (error) { await client.query("ROLLBACK"); throw error; } finally { client.release(); } } async clear(context: MemoryContext): Promise { await this.pool.query( "DELETE FROM agent_memory_messages WHERE session_id = $1", [context.sessionId], ); } async recordError(input: MemoryErrorInput): Promise { await this.pool.query( `INSERT INTO agent_memory_errors ( session_id, user_id, run_id, error, messages ) VALUES ($1, $2, $3, $4::jsonb, $5::jsonb)`, [ input.context.sessionId, input.context.userId ?? null, input.runId, JSON.stringify(serializeError(input.error)), JSON.stringify(input.messages), ], ); } } function serializeError(error: unknown): Record { if (error instanceof Error) { return { name: error.name, message: error.message, stack: error.stack, }; } return { message: String(error) }; } ``` ## Use It [#use-it] ```ts const memory = new SqlMemoryStore(pool); const agent = new AgentBuilder("support", model) .memory(memory) .build(); await agent.session("thread_123", { userId: "user_456" }).prompt("Hello").send(); ``` # Langfuse (/docs/guides/observability/langfuse) Use `@anvia/langfuse` when you want Anvia observer events in Langfuse. ## 1. Install the Adapter [#1-install-the-adapter] ```sh pnpm add @anvia/langfuse ``` The adapter is separate from `@anvia/core`. Core Anvia does not export `langfuse`. ## 2. Create the Tracing Observer [#2-create-the-tracing-observer] Pass Langfuse credentials from your application's configuration. ```ts import { langfuse } from "@anvia/langfuse"; const tracing = langfuse.create({ publicKey, secretKey, baseUrl, environment, release, }); ``` `baseUrl`, `environment`, and `release` are optional. ## 3. Attach It to an Agent [#3-attach-it-to-an-agent] ```ts const agent = new AgentBuilder("support", model) .instructions("Answer with a short engineering-focused summary.") .observe(tracing) .defaultMaxTurns(2) .build(); ``` ## 4. Send Trace Metadata [#4-send-trace-metadata] ```ts const response = await agent .prompt("Summarize ticket TICKET-1001 for engineering.") .withTrace({ name: "support-ticket-summary", userId: "user_123", sessionId: "session_456", metadata: { ticketId: "TICKET-1001", surface: "support-console", }, tags: ["support", "anvia"], }) .send(); console.log(response.trace?.traceId); ``` Langfuse receives the run, model generations, tool observations, usage, errors, and trace attributes. ## 5. Score a Trace [#5-score-a-trace] ```ts await tracing.score({ traceId: response.trace?.traceId, observationId: response.trace?.observationId, name: "quality", value: 1, comment: "Good answer", metadata: { source: "review" }, }); ``` Scoring requires `publicKey`, `secretKey`, and a trace id. ## 6. Report Eval Scores [#6-report-eval-scores] Use `createLangfuseEvalReporter(...)` to publish eval outcomes as Langfuse scores. ```ts import { agentEvalTarget, contains, runEvalSuite } from "@anvia/core/evals"; import { createLangfuseEvalReporter } from "@anvia/langfuse"; await runEvalSuite({ name: "support-agent-regression", cases: [{ id: "refund-window", input: "How long are refunds available?", expected: "30 days" }], target: agentEvalTarget(agent), metrics: [contains()], reporters: [createLangfuseEvalReporter(tracing)], }); ``` The reporter uses the prompt response trace when available. You can also attach `traceId` and `observationId` to eval case metadata. ## 7. Flush or Shutdown [#7-flush-or-shutdown] ```ts await tracing.flush(); await tracing.shutdown(); ``` Use `flush()` after short-lived jobs. Use `shutdown()` when the process is exiting. # Observers (/docs/guides/observability/observers) Observers are plain TypeScript objects or classes that receive runtime events from an agent run. Use them for logs, metrics, traces, and debugging. ## 1. Create an Observer [#1-create-an-observer] ```ts import type { AgentObserver, AgentRunObserver, AgentRunStartArgs, } from "@anvia/core"; class ConsoleObserver implements AgentObserver { startRun(args: AgentRunStartArgs): AgentRunObserver { console.log("run_start", { promptRole: args.prompt.role, maxTurns: args.maxTurns, trace: args.trace, }); return { startGeneration({ turn, request }) { console.log("generation_start", { turn, tools: request.tools.map((tool) => tool.name), }); return { end({ response }) { console.log("generation_end", response.usage.totalTokens); }, }; }, startTool({ toolName }) { console.log("tool_start", toolName); return { end({ result, skipped }) { console.log("tool_end", { skipped, result }); }, }; }, end({ usage }) { console.log("run_end", usage.totalTokens); }, error({ error }) { console.error("run_error", error); }, }; } } ``` ## 2. Attach It to an Agent [#2-attach-it-to-an-agent] ```ts const observer = new ConsoleObserver(); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .observe(observer) .build(); ``` Observers run for `.send()` and `.stream()`. ## 3. Add Trace Metadata Per Request [#3-add-trace-metadata-per-request] ```ts const response = await agent .prompt("How do I reset my password?") .withTrace({ name: "support-question", userId: "user_123", sessionId: "session_456", metadata: { surface: "docs" }, tags: ["support"], }) .send(); console.log(response.trace); ``` Observers receive the trace options in `startRun(...)`. If an observer returns trace ids, Anvia exposes them on the final response. ## 4. Decide Failure Behavior [#4-decide-failure-behavior] Observer failures are swallowed by default so telemetry does not break product behavior. ```ts const agent = new AgentBuilder("support", model) .observe(observer, { failOnObserverError: true }) .build(); ``` Use strict mode in tests or workflows where missing telemetry should fail the run. # OpenTelemetry (/docs/guides/observability/otel) Use `@anvia/otel` when your application already initializes OpenTelemetry and you want Anvia observer events emitted as standard spans. ## 1. Install the Adapter [#1-install-the-adapter] ```sh pnpm add @anvia/otel @opentelemetry/api ``` The adapter is separate from `@anvia/core`. Core Anvia does not export `otel`. For OTLP HTTP export, install the SDK and exporter in your application: ```sh pnpm add @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-http ``` ## 2. Initialize OpenTelemetry [#2-initialize-opentelemetry] Configure the OpenTelemetry SDK before running agents. The endpoint can also come from `OTEL_EXPORTER_OTLP_ENDPOINT` or `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`. ```ts import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http"; import { NodeSDK } from "@opentelemetry/sdk-node"; const sdk = new NodeSDK({ traceExporter: new OTLPTraceExporter(), }); sdk.start(); ``` The adapter does not call `sdk.start()`, `forceFlush()`, or `shutdown()`. ## 3. Create the Tracing Observer [#3-create-the-tracing-observer] ```ts import { otel } from "@anvia/otel"; const tracing = otel.create({ serviceName: "support-agent", }); ``` Pass `tracer` when your application needs a specific tracer instance. Otherwise, the adapter uses `trace.getTracer("@anvia/otel")`. ## 4. Attach It to an Agent [#4-attach-it-to-an-agent] ```ts const agent = new AgentBuilder("support", model) .instructions("Answer with a short engineering-focused summary.") .observe(tracing) .defaultMaxTurns(2) .build(); ``` ## 5. Send Trace Metadata [#5-send-trace-metadata] ```ts const response = await agent .prompt("Summarize ticket TICKET-1001 for engineering.") .withTrace({ name: "support-ticket-summary", userId: "user_123", sessionId: "session_456", metadata: { ticketId: "TICKET-1001", surface: "support-console", }, tags: ["support", "anvia"], }) .send(); console.log(response.trace?.traceId); ``` OpenTelemetry receives spans for the run, model generations, tool calls, token usage, errors, and trace metadata. ## 6. Join an Existing Trace [#6-join-an-existing-trace] Pass a valid 32-character hex trace id to continue an existing distributed trace. ```ts await agent .prompt("Summarize ticket TICKET-1001.") .withTrace({ traceId: incomingTraceId }) .send(); ``` The root Anvia span is parented under a synthetic remote parent so generated spans share that trace id. ## 7. Shutdown Your SDK [#7-shutdown-your-sdk] Your application owns SDK lifecycle. ```ts await sdk.shutdown(); ``` # Trace Groups (/docs/guides/observability/trace-groups) Trace metadata lets observers group related agent runs by user, session, workflow, or product surface. ## 1. Add a Trace Name [#1-add-a-trace-name] ```ts const response = await agent .prompt("Summarize this ticket.") .withTrace({ name: "ticket-summary", }) .send(); ``` Use stable names such as `support-chat`, `ticket-summary`, or `retrieval-answer`. ## 2. Add User and Session IDs [#2-add-user-and-session-ids] ```ts const response = await agent .prompt("Answer the user's support question.") .withTrace({ name: "support-chat", userId: user.id, sessionId: conversation.id, }) .send(); ``` Use the same `sessionId` for related turns in one conversation. ## 3. Add Metadata and Tags [#3-add-metadata-and-tags] ```ts const response = await agent .prompt("Handle this ticket.") .withTrace({ name: "ticket-triage", metadata: { ticketId: ticket.id, plan: customer.plan, }, tags: ["support", "triage"], }) .send(); ``` Keep metadata useful and safe to store. Do not include secrets or sensitive prompt content unless your observability policy allows it. ## 4. Continue an Existing Trace [#4-continue-an-existing-trace] ```ts await agent .prompt("Follow up on this ticket.") .withTrace({ traceId: existingTraceId, name: "ticket-follow-up", }) .send(); ``` Observers can use `traceId` to attach new work to an existing trace. ## 5. Read Trace IDs From the Response [#5-read-trace-ids-from-the-response] ```ts const response = await agent .prompt("Summarize this ticket.") .withTrace({ name: "ticket-summary" }) .send(); console.log(response.trace?.traceId); console.log(response.trace?.observationId); ``` Trace ids are present when an observer returns them. # Tracing (/docs/guides/observability/tracing) Tracing is built on the observer interface. A tracing integration observes runs, model generations, tool calls, errors, usage, and trace metadata. ## 1. Attach a Tracing Observer [#1-attach-a-tracing-observer] ```ts const tracing = createTracingObserver(); const agent = new AgentBuilder("support", model) .observe(tracing) .defaultMaxTurns(3) .build(); ``` Tracing observers use the same `.observe(...)` API as logging observers. ## 2. Pass Trace Metadata [#2-pass-trace-metadata] ```ts const response = await agent .prompt("Summarize ticket TICKET-1001.") .withTrace({ name: "support-ticket-summary", userId: user.id, sessionId: session.id, metadata: { ticketId: "TICKET-1001" }, tags: ["support"], version: "2026-05-01", }) .send(); ``` ## 3. Capture the Returned Trace [#3-capture-the-returned-trace] ```ts console.log(response.trace?.traceId); console.log(response.trace?.observationId); ``` Use these ids to link product logs, support tickets, evaluation scores, or later runs. ## 4. Flush on Shutdown [#4-flush-on-shutdown] Long-lived tracing integrations may buffer telemetry. ```ts await tracing.flush?.(); await tracing.shutdown?.(); ``` Call `flush()` before process exit when you need pending events delivered. Call `shutdown()` during application shutdown. ## 5. Use the Langfuse Adapter [#5-use-the-langfuse-adapter] Anvia ships Langfuse support as a separate package: ```ts import { langfuse } from "@anvia/langfuse"; const tracing = langfuse.create({ publicKey, secretKey, }); ``` Continue with [Langfuse](/docs/guides/observability/langfuse) for the complete setup. # Agents in Pipelines (/docs/guides/pipelines/agents-in-pipelines) Agents fit best in pipelines when a stage needs model reasoning, tools, retrieval, or instructions. Keep the orchestration in the pipeline and the model behavior on the agent. ## 1. Build a Focused Agent [#1-build-a-focused-agent] ```ts const triageAgent = new AgentBuilder("triage", model) .instructions("Classify the urgency of support tickets.") .defaultMaxTurns(2) .build(); ``` ## 2. Format the Prompt in a Step [#2-format-the-prompt-in-a-step] ```ts const pipeline = new PipelineBuilder() .step((ticket) => `Classify this ticket:\n\n${ticket}`) .prompt(triageAgent) .build(); ``` The pipeline owns the changing input. The agent owns the stable instructions. ## 3. Use Agent Tools Normally [#3-use-agent-tools-normally] ```ts const supportAgent = new AgentBuilder("support", model) .instructions("Look up account details before answering account questions.") .tool(lookupAccount) .defaultMaxTurns(3) .build(); const pipeline = new PipelineBuilder() .step((ticket) => `Answer this support ticket:\n\n${ticket}`) .prompt(supportAgent) .build(); ``` The agent can call tools during the prompt step. The pipeline receives the final agent output. ## 4. Use Separate Agents for Separate Jobs [#4-use-separate-agents-for-separate-jobs] ```ts const pipeline = new PipelineBuilder() .parallel({ customerReply: new PipelineBuilder().prompt(replyAgent).build(), internalSummary: new PipelineBuilder().prompt(summaryAgent).build(), }) .build(); ``` Separate agents make instructions, tools, and traces easier to reason about. # Batch Runs (/docs/guides/pipelines/batch-runs) Use `batch(...)` when the same pipeline should process many inputs with a concurrency limit. ## 1. Build a Normal Pipeline [#1-build-a-normal-pipeline] ```ts const normalizeTicket = new PipelineBuilder() .step((ticket) => ticket.trim()) .step((ticket) => ticket.replace(/\s+/g, " ")) .build(); ``` ## 2. Run Many Inputs [#2-run-many-inputs] ```ts const results = await normalizeTicket.batch( [ " checkout failed ", " cannot update card ", " password reset issue ", ], { concurrency: 2 }, ); ``` Results preserve input order. ```ts console.log(results[0]); // "checkout failed" ``` ## 3. Pick a Concurrency Limit [#3-pick-a-concurrency-limit] | Pipeline Work | Starting Concurrency | | ---------------------- | ------------------------- | | CPU-light transforms | `4` to `16` | | Provider calls | `2` to `5` | | Database or API writes | match your service limits | ## 4. Handle Failures at the Batch Boundary [#4-handle-failures-at-the-batch-boundary] If any item throws, the batch rejects. ```ts try { const results = await pipeline.batch(tickets, { concurrency: 3 }); return results; } catch (error) { logger.error(error, "ticket batch failed"); throw error; } ``` For per-item recovery, catch inside a step and return an explicit success or failure object. # Composition Patterns (/docs/guides/pipelines/composition-patterns) Once the basic flow works, split reusable stages into small pipeline operations and compose them with `.use(...)`. ## Reuse a Pipeline Operation [#reuse-a-pipeline-operation] ```ts const normalizeTicket = new PipelineBuilder() .step((ticket) => ticket.trim()) .step((ticket) => ticket.replace(/\s+/g, " ")) .build(); const supportPipeline = new PipelineBuilder() .use(normalizeTicket) .prompt(supportAgent) .build(); ``` Use `.use(...)` when a whole stage is useful in more than one workflow. ## Normalize, Prompt, Extract [#normalize-prompt-extract] ```ts const triagePipeline = new PipelineBuilder() .use(normalizeTicket) .step((ticket) => `Summarize and classify:\n\n${ticket}`) .prompt(triageAgent) .extract(triageExtractor) .build(); ``` This is the default shape for many production workflows: deterministic preprocessing, model reasoning, then schema validation. ## Branch, Then Merge [#branch-then-merge] ```ts const pipeline = new PipelineBuilder() .use(normalizeTicket) .parallel({ reply: new PipelineBuilder().prompt(replyAgent).build(), triage: new PipelineBuilder().prompt(triageAgent).extract(triageExtractor).build(), }) .step(({ reply, triage }) => ({ reply, priority: triage.priority, })) .build(); ``` Use this when the branches do not depend on each other but the final result should combine them. ## Keep the Shape Visible [#keep-the-shape-visible] Prefer a pipeline that reads like the workflow: ```ts const pipeline = new PipelineBuilder() .use(loadTicket) .use(addRetrievalContext) .prompt(agent) .extract(outputExtractor) .use(saveResult) .build(); ``` If a step becomes hard to name, split it into smaller steps. # Extractor Steps (/docs/guides/pipelines/extractor-steps) Use `.extract(extractor)` when a pipeline stage should convert the current text into typed data. ## 1. Build the Extractor [#1-build-the-extractor] ```ts import { ExtractorBuilder, PipelineBuilder } from "@anvia/core"; import { z } from "zod"; const triageSchema = z.object({ priority: z.enum(["low", "medium", "high"]), summary: z.string(), }); const extractor = new ExtractorBuilder(model, triageSchema) .instructions("Extract triage fields from the support summary.") .retries(1) .build(); ``` ## 2. Extract Inside the Pipeline [#2-extract-inside-the-pipeline] ```ts const pipeline = new PipelineBuilder() .step((ticket) => `Summarize this ticket:\n\n${ticket}`) .prompt(summarizer) .extract(extractor) .build(); ``` The output type is now the schema data. ```ts const triage = await pipeline.run("Checkout is down for enterprise users."); console.log(triage.priority); ``` ## 3. Add Application Logic After Extraction [#3-add-application-logic-after-extraction] ```ts const pipeline = new PipelineBuilder() .prompt(summarizer) .extract(extractor) .step((triage) => ({ ...triage, route: triage.priority === "high" ? "incident" : "support", })) .build(); ``` Use extractor steps after text generation and before business logic that needs structured fields. # Parallel Branches (/docs/guides/pipelines/parallel-branches) Use `.parallel(...)` when independent work can run from the same input. Each branch is a pipeline operation, and the output is an object keyed by branch name. ## 1. Create Branches [#1-create-branches] ```ts const summaryBranch = new PipelineBuilder() .step((ticket) => `Summarize:\n\n${ticket}`) .prompt(summarizer) .build(); const priorityBranch = new PipelineBuilder() .step((ticket) => `Classify priority:\n\n${ticket}`) .prompt(priorityAgent) .build(); ``` ## 2. Run Them in Parallel [#2-run-them-in-parallel] ```ts const pipeline = new PipelineBuilder() .parallel({ summary: summaryBranch, priority: priorityBranch, }) .build(); ``` ## 3. Read the Branch Output [#3-read-the-branch-output] ```ts const result = await pipeline.run("Checkout is down for enterprise users."); console.log(result.summary); console.log(result.priority); ``` The branch names become the output keys. ## 4. Continue After Branching [#4-continue-after-branching] ```ts const pipeline = new PipelineBuilder() .parallel({ summary: summaryBranch, priority: priorityBranch, }) .step((result) => ({ title: result.summary.slice(0, 80), priority: result.priority, })) .build(); ``` Use parallel branches for independent checks. If one stage depends on another stage's output, keep it linear. # Pipeline Builder (/docs/guides/pipelines/pipeline-builder) Use `PipelineBuilder` when a workflow should be explicit, testable, and made from named stages instead of one large prompt. ## 1. Start With Input and Output Types [#1-start-with-input-and-output-types] ```ts import { PipelineBuilder } from "@anvia/core"; const pipeline = new PipelineBuilder(); ``` `PipelineBuilder` tracks two types: | Type | Meaning | | -------- | ------------------------------------------------------------------------------------------------------- | | `Input` | The value accepted by `run(...)` after `.build()`. | | `Output` | The current value after the latest stage. It starts as `Input` and changes as stages return new values. | Most pipelines only set the first generic. TypeScript infers the output after every `.step(...)`, `.use(...)`, `.parallel(...)`, `.prompt(...)`, and `.extract(...)`. ## 2. Build, Then Run [#2-build-then-run] ```ts const normalizeTicket = new PipelineBuilder() .step((input) => input.trim()) .step((input) => input.replace(/\s+/g, " ")) .build(); const result = await normalizeTicket.run(" checkout is failing "); console.log(result); ``` The builder describes the pipeline. `.build()` returns the runnable `Pipeline`. `run(...)` and `batch(...)` live on the built pipeline, not on the builder. ## API at a glance [#api-at-a-glance] | API | Input | Output | | --------------------------------- | --------------------------------------------- | ---------------------------------- | | `.step(fn)` | Current output | The value returned by `fn` | | `.use(op)` | Current output | The output of another `PipelineOp` | | `.parallel({ ... })` | Current output, passed to every branch | An object keyed by branch name | | `.prompt(agent)` | Current output converted with `String(input)` | Agent text output | | `.extract(extractor)` | Current output converted with `String(input)` | Extractor schema data | | `.build()` | Builder state | Runnable `Pipeline` | | `pipeline.run(input)` | Original `Input` | Final `Output` | | `pipeline.batch(inputs, options)` | Iterable of original `Input` | Array of final `Output` values | ## 3. Return New Shapes [#3-return-new-shapes] Pipelines are not limited to strings. A step can return an object, array, number, or any other TypeScript value. ```ts type SupportTicketInput = { customer: string; subject: string; body: string; }; type NormalizedTicket = { customer: string; title: string; words: number; }; const normalizeTicket = new PipelineBuilder() .step((ticket): NormalizedTicket => ({ customer: ticket.customer.trim(), title: ticket.subject.trim().toLowerCase(), words: ticket.body.trim().split(/\s+/).length, })) .build(); const ticket = await normalizeTicket.run({ customer: " Acme Co. ", subject: " Checkout is failing ", body: "Enterprise checkout fails after payment retries.", }); console.log(ticket.title); ``` The first step receives `SupportTicketInput`. The built pipeline returns `NormalizedTicket`. ## 4. Use Arrays and Numbers [#4-use-arrays-and-numbers] ```ts const executiveNotes = new PipelineBuilder() .step((notes) => notes.map((note) => `- ${note}`).join("\n")) .build(); const notes = await executiveNotes.run([ "Checkout failures increased after payment retry changes.", "Enterprise customers are affected.", ]); ``` ```ts const formatPrice = new PipelineBuilder() .step((dollars) => Math.round(dollars * 100)) .step((cents) => ({ cents, formatted: `$${(cents / 100).toFixed(2)}`, })) .build(); const price = await formatPrice.run(12.5); console.log(price.formatted); ``` ## 5. Branch Into Different Output Types [#5-branch-into-different-output-types] ```ts const ticketSignals = new PipelineBuilder() .parallel({ title: new PipelineBuilder() .step((ticket) => ticket.split(".")[0] ?? ticket) .build(), wordCount: new PipelineBuilder() .step((ticket) => ticket.trim().split(/\s+/).length) .build(), urgent: new PipelineBuilder() .step((ticket) => ticket.toLowerCase().includes("outage")) .build(), }) .build(); const signals = await ticketSignals.run( "Checkout outage affects enterprise customers. Payment retries fail.", ); console.log(signals.title, signals.wordCount, signals.urgent); ``` The result is typed as: ```ts { title: string; wordCount: number; urgent: boolean; } ``` ## 6. Format Before Prompting or Extracting [#6-format-before-prompting-or-extracting] `.prompt(agent)` and `.extract(extractor)` call `String(input)` internally. For strings, that is usually fine. For objects, add a formatting step first so the model receives intentional text instead of `[object Object]`. ```ts const ticketPipeline = new PipelineBuilder() .step( (ticket) => [ `Customer: ${ticket.customer}`, `Subject: ${ticket.subject}`, `Body: ${ticket.body}`, ].join("\n"), ) .prompt(summarizer) .extract(triageExtractor) .build(); const triage = await ticketPipeline.run({ customer: "Acme Co.", subject: "Checkout is failing", body: "Enterprise checkout fails after payment retries.", }); ``` ## 7. Add Capabilities Gradually [#7-add-capabilities-gradually] Build pipelines in this order: 1. Use [Steps](/docs/guides/pipelines/steps) for ordinary transforms. 2. Use [Prompt Steps](/docs/guides/pipelines/prompt-steps) when a stage needs an agent. 3. Use [Extractor Steps](/docs/guides/pipelines/extractor-steps) when a stage should return typed data. 4. Use [Parallel Branches](/docs/guides/pipelines/parallel-branches) when independent work can run at the same time. 5. Use [Batch Runs](/docs/guides/pipelines/batch-runs) when the same pipeline should process many inputs. 6. Read [Example: Research Pipeline](/docs/guides/pipelines/research-example) for a complete fetch, summarize, and extract workflow. ## Minimal End-to-End Shape [#minimal-end-to-end-shape] ```ts const ticketPipeline = new PipelineBuilder() .step((ticket) => ticket.trim()) .step((ticket) => `Summarize this ticket:\n\n${ticket}`) .prompt(summarizer) .build(); const summary = await ticketPipeline.run("Checkout fails for enterprise users."); ``` Keep the first version linear. Add branching and composition only after the simple flow is clear. # Prompt Steps (/docs/guides/pipelines/prompt-steps) Use `.prompt(agent)` when a pipeline stage should send the current value to an agent and continue with the agent's text output. ## 1. Build the Agent [#1-build-the-agent] ```ts const summarizer = new AgentBuilder("summarizer", model) .instructions("Summarize support tickets in one paragraph.") .build(); ``` ## 2. Prepare the Prompt [#2-prepare-the-prompt] ```ts const pipeline = new PipelineBuilder() .step((ticket) => `Summarize this support ticket:\n\n${ticket}`) .prompt(summarizer) .build(); ``` `.prompt(...)` calls `agent.prompt(String(input)).send()` and passes `response.output` to the next stage. ## 3. Continue After the Prompt [#3-continue-after-the-prompt] ```ts const pipeline = new PipelineBuilder() .step((ticket) => `Summarize this support ticket:\n\n${ticket}`) .prompt(summarizer) .step((summary) => summary.trim()) .build(); ``` Use a normal step before the prompt to format the input. Use a normal step after the prompt to clean or route the output. ## 4. Keep Agent Instructions on the Agent [#4-keep-agent-instructions-on-the-agent] Put stable behavior in the agent instructions, and put per-run data in the pipeline step. ```ts const agent = new AgentBuilder("triage", model) .instructions("Classify support urgency using the company policy.") .build(); const pipeline = new PipelineBuilder() .step((ticket) => `Ticket:\n${ticket}`) .prompt(agent) .build(); ``` # Example: Research Pipeline (/docs/guides/pipelines/research-example) This example shows a common pipeline shape: 1. get data from the internet 2. format the evidence for a model 3. summarize the evidence with an agent 4. extract structured fields from the summary The internet access is ordinary application code. Anvia owns the workflow shape; your app owns how search, scraping, permissions, rate limits, and caching work. ## 1. Define the Research Shape [#1-define-the-research-shape] ```ts import { AgentBuilder, ExtractorBuilder, PipelineBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; type WebResult = { title: string; url: string; snippet: string; }; type ResearchPacket = { query: string; results: WebResult[]; }; const researchReportSchema = z.object({ topic: z.string(), summary: z.string(), keyFindings: z.array( z.object({ finding: z.string(), sourceUrls: z.array(z.string().url()), }), ), risks: z.array(z.string()), followUpQuestions: z.array(z.string()), }); ``` The pipeline starts with a plain `string` query, then turns it into a `ResearchPacket`, then into model text, then into structured schema data. ## 2. Add Internet Access [#2-add-internet-access] ```ts async function searchWeb(query: string): Promise { const response = await fetch( `https://api.example.com/search?q=${encodeURIComponent(query)}`, ); if (!response.ok) { throw new Error(`Search failed: ${response.status}`); } const body = (await response.json()) as { results: WebResult[] }; return body.results.slice(0, 5); } ``` Replace `searchWeb(...)` with your actual web search provider, crawler, internal search service, or retrieval layer. Keep this code outside the agent so it is testable and permissioned by your application. ## 3. Build the Agent and Extractor [#3-build-the-agent-and-extractor] ```ts const model = new OpenAIClient({ apiKey }).completionModel("gpt-5.5"); const summarizer = new AgentBuilder("research-summarizer", model) .instructions( [ "Summarize the research packet using only the provided sources.", "Mention source URLs when making concrete claims.", "Call out uncertainty or missing evidence.", "Return visible final text, not only reasoning.", ].join("\n"), ) .build(); const reportExtractor = new ExtractorBuilder(model, researchReportSchema) .instructions("Extract a concise structured research report from the summary.") .retries(1) .build(); ``` The summarizer creates readable synthesis. The extractor turns that synthesis into application data. ## 4. Compose the Pipeline [#4-compose-the-pipeline] ```ts const researchPipeline = new PipelineBuilder() .step(async (query): Promise => ({ query, results: await searchWeb(query), })) .step(({ query, results }) => [ `Research question: ${query}`, "", "Sources:", ...results.map((result, index) => [ `[${index + 1}] ${result.title}`, `URL: ${result.url}`, `Snippet: ${result.snippet}`, ].join("\n"), ), ].join("\n\n"), ) .prompt(summarizer) .extract(reportExtractor) .build(); ``` The formatting step before `.prompt(...)` is important. `.prompt(...)` sends `String(input)` to the agent, so object values should be converted into intentional prompt text first. ## 5. Run It [#5-run-it] ```ts const report = await researchPipeline.run( "Recent examples of AI agents in browser automation", ); console.log(report.summary); console.log(report.keyFindings); ``` The result is typed from `researchReportSchema`, so application code can store, render, validate, or route the research report without parsing free-form text. ## Production Notes [#production-notes] For production research pipelines: * cache search results when possible * store source URLs with the extracted report * catch search failures inside a step if partial results are acceptable * keep side-effecting fetch, crawl, or scrape logic outside the agent prompt * prefer small result sets first, then add batching or parallel branches when needed # Steps (/docs/guides/pipelines/steps) Steps are the basic unit of a pipeline. Use them for parsing, normalization, enrichment, formatting, and small application-owned decisions. ## 1. Transform the Input [#1-transform-the-input] ```ts type TicketInput = { customer: string; subject: string; body: string; }; const pipeline = new PipelineBuilder() .step((ticket) => ({ ...ticket, customer: ticket.customer.trim(), subject: ticket.subject.trim(), })) .step((ticket) => ({ title: ticket.subject.toLowerCase(), customer: ticket.customer, wordCount: ticket.body.trim().split(/\s+/).length, })) .build(); const result = await pipeline.run({ customer: " Acme Co. ", subject: " Checkout is failing ", body: "Enterprise checkout fails after payment retries.", }); ``` The first step receives the `run(...)` input. ## 2. Return a New Shape [#2-return-a-new-shape] ```ts const pipeline = new PipelineBuilder() .step((dollars) => Math.round(dollars * 100)) .step((cents) => ({ cents, formatted: `$${(cents / 100).toFixed(2)}`, })) .build(); const price = await pipeline.run(12.5); ``` Each next step receives the value returned by the previous step. In this example, the first step returns cents as a number, then the second step returns an object. ## 3. Use Async Steps [#3-use-async-steps] ```ts const pipeline = new PipelineBuilder<{ email: string; subject: string }>() .step(async (input) => ({ user: await users.findByEmail(input.email), subject: input.subject, })) .step(({ user, subject }) => ({ userId: user.id, plan: user.plan, subject, })) .build(); ``` Async steps are awaited before the next stage runs. ## 4. Keep Steps Small [#4-keep-steps-small] Prefer several simple steps over one large function: ```ts const pipeline = new PipelineBuilder() .step(parseInboundEmail) .step(normalizeTicketFields) .step(loadCustomerContext) .build(); ``` Small steps are easier to test and easier to replace with agent, extractor, or retrieval stages later. # Embed Documents (/docs/guides/retrieval/embed-documents) Use `embedDocuments(...)` during preprocessing. This step should usually run before user requests: in a build step, admin action, startup task, or background ingestion job. If your source material starts as local files or PDFs, read [Loaders](/docs/guides/retrieval/loaders) first, then embed the loaded documents. ## 1. Prepare Documents [#1-prepare-documents] ```ts const documents = [ { id: "password-reset", title: "Password reset policy", body: "Password reset links expire after 30 minutes.", product: "support", }, { id: "priority-support", title: "Priority support", body: "Enterprise customers receive priority support.", product: "support", }, ]; ``` Normalize or chunk your source text before embedding it. ## 2. Load Local Files [#2-load-local-files] Use `@anvia/core/loaders` when ingestion starts from local text files or PDFs. Loaders convert files, directories, globs, bytes, and PDFs into the `Document[]` shape that `embedDocuments(...)` expects. ```ts import { FileLoader, fileLoaderToDocuments } from "@anvia/core/loaders"; const documents = await fileLoaderToDocuments( FileLoader.withGlob("content/**/*.md").readWithPath().ignoreErrors(), ); ``` PDF loaders support the same glob, directory, and byte inputs. Call `.byPage()` when each page should become its own retrieval document. ```ts import { PdfFileLoader, pdfPageLoaderToDocuments } from "@anvia/core/loaders"; const pdfPages = await pdfPageLoaderToDocuments( PdfFileLoader.withGlob("manuals/**/*.pdf").readWithPath().byPage().ignoreErrors(), ); ``` Anvia loaders do ingestion only. For text chunking beyond PDF pages, preprocess text in application code before calling `embedDocuments(...)`. For the complete loader workflow, see [Loaders](/docs/guides/retrieval/loaders). ## 3. Embed With Selectors [#3-embed-with-selectors] ```ts import { embedDocuments } from "@anvia/core"; const embedded = await embedDocuments(embeddings, documents, { id: (doc) => doc.id, content: (doc) => `${doc.title}\n${doc.body}`, metadata: (doc) => ({ product: doc.product, title: doc.title, }), }); ``` The `content` selector chooses text to embed. The original document is still kept on the embedded result. ## 4. Embed Multiple Chunks per Document [#4-embed-multiple-chunks-per-document] Return an array from `content(...)` when one document should have multiple embeddings. ```ts const embedded = await embedDocuments(embeddings, articles, { id: (article) => article.slug, content: (article) => article.sections.map((section) => section.text), metadata: (article) => ({ product: article.product }), }); ``` Multiple chunks can match the same document during search. ## 5. Control Ingestion Concurrency [#5-control-ingestion-concurrency] ```ts const embedded = await embedDocuments(embeddings, documents, { id: (doc) => doc.id, content: (doc) => doc.body, concurrency: 2, }); ``` Start low and increase concurrency only when your embedding provider and infrastructure can handle it. # Embeddings (/docs/guides/retrieval/embeddings) Embeddings turn text into vectors so you can search by meaning instead of exact keywords. Retrieval starts by choosing an embedding model. ## 1. Create an Embedding Model [#1-create-an-embedding-model] ```ts import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ apiKey }); const embeddings = client.embeddingModel("text-embedding-3-small"); ``` Use the same embedding model for indexing documents and searching with user queries. ## 2. Embed One Text [#2-embed-one-text] ```ts import { embedText } from "@anvia/core"; const embedding = await embedText(embeddings, "Password reset links expire after 30 minutes."); console.log(embedding.vector.length); ``` ## 3. Embed Many Texts [#3-embed-many-texts] ```ts import { embedTexts } from "@anvia/core"; const results = await embedTexts(embeddings, [ "Password reset links expire after 30 minutes.", "Enterprise customers receive priority support.", ]); ``` Anvia respects the model's batch size when available. ## 4. Use Embeddings Through Higher-Level Helpers [#4-use-embeddings-through-higher-level-helpers] Most retrieval workflows should use [Embed Documents](/docs/guides/retrieval/embed-documents) instead of calling `embedText(...)` directly. Document embedding preserves ids, metadata, and the original document shape. # Loaders (/docs/guides/retrieval/loaders) Loaders are ingestion helpers for retrieval preprocessing. Use them when your source material starts as local files, directories, globs, bytes, or PDFs and you need to turn that material into Anvia `Document[]` values before calling `embedDocuments(...)`. Import loaders from `@anvia/core/loaders`, not from the root `@anvia/core` entry point. The loader subpath is separate because it depends on Node filesystem and PDF extraction packages. ## 1. Load Text Files [#1-load-text-files] Use `FileLoader` for UTF-8 text files such as Markdown, plain text, exported docs, or generated knowledge files. ```ts import { FileLoader, fileLoaderToDocuments } from "@anvia/core/loaders"; const documents = await fileLoaderToDocuments( FileLoader.withGlob("content/**/*.md").readWithPath().ignoreErrors(), ); ``` `readWithPath()` keeps the source path, and `fileLoaderToDocuments(...)` stores that path as the document id plus `source` metadata. ## 2. Load a Directory [#2-load-a-directory] ```ts const documents = await fileLoaderToDocuments( FileLoader.withDir("content/articles").readWithPath().ignoreErrors(), ); ``` `withDir(...)` reads direct files only. Use `withGlob(...)` when you need recursive matching. ## 3. Load Bytes [#3-load-bytes] ```ts const bytes = new TextEncoder().encode("Password reset links expire after 30 minutes."); const documents = await fileLoaderToDocuments( FileLoader.fromBytes(bytes).readWithPath().ignoreErrors(), ); ``` Byte loaders are useful when files come from an upload, object store, or another runtime source instead of a local path. ## 4. Load PDFs [#4-load-pdfs] Use `PdfFileLoader` when source material is a PDF. ```ts import { PdfFileLoader, pdfLoaderToDocuments } from "@anvia/core/loaders"; const documents = await pdfLoaderToDocuments( PdfFileLoader.withGlob("manuals/**/*.pdf").readWithPath().ignoreErrors(), ); ``` This creates one document per PDF with the extracted text and `mediaType: "application/pdf"` metadata. ## 5. Split PDFs by Page [#5-split-pdfs-by-page] ```ts import { PdfFileLoader, pdfPageLoaderToDocuments } from "@anvia/core/loaders"; const pages = await pdfPageLoaderToDocuments( PdfFileLoader.withGlob("manuals/**/*.pdf").readWithPath().byPage().ignoreErrors(), ); ``` Use page splitting when a whole PDF is too broad for retrieval or when page-level source metadata matters. Page documents include `source`, `mediaType`, and `pageNumber` metadata. ## 6. Handle Batch Errors [#6-handle-batch-errors] Loader methods yield `LoaderResult` values by default so one unreadable file does not have to fail the whole batch. ```ts for await (const result of FileLoader.withGlob("content/**/*.md").readWithPath()) { if (result.ok) { console.log(result.value.path); } else { console.error(result.error); } } ``` Call `.ignoreErrors()` when your ingestion job should skip failed files and continue with successful records. ## 7. Embed Loaded Documents [#7-embed-loaded-documents] After loading, pass the documents to `embedDocuments(...)`. ```ts import { embedDocuments } from "@anvia/core"; const embedded = await embedDocuments(embeddings, documents, { id: (document) => document.id, content: (document) => document.text, metadata: (document) => document.additionalProps, }); ``` Loaders do ingestion only. For chunking beyond PDF pages, split text in application code before embedding or return multiple strings from the `content(...)` selector. ## Related Reference [#related-reference] | Topic | Reference | | ------------------ | --------------------------------------------------------- | | Loader API | [Loaders](/docs/reference/core/loaders) | | Document embedding | [Embed Documents](/docs/guides/retrieval/embed-documents) | # LSH (/docs/guides/retrieval/lsh) Locality-sensitive hashing narrows the candidate set before scoring vectors. Use it when an in-memory store has enough documents that brute-force search is too slow. ## 1. Start With the Default [#1-start-with-the-default] ```ts const store = InMemoryVectorStore.fromDocuments(embedded); ``` The default search strategy is brute force. It is simpler and a good starting point for tests, small datasets, and early prototypes. ## 2. Enable LSH [#2-enable-lsh] ```ts const store = InMemoryVectorStore.fromDocuments(embedded, { index: { type: "lsh", numTables: 4, numHyperplanes: 8, seed: 7, }, }); const index = store.index(embeddings); ``` LSH trades exact candidate selection for faster narrowing. Results are still scored after candidates are selected. ## 3. Search Normally [#3-search-normally] ```ts const results = await index.search({ query: "password reset", topK: 3, }); ``` The search API is the same as the default index. ## 4. Tune Carefully [#4-tune-carefully] | Option | Effect | | ---------------- | ----------------------------------------------- | | `numTables` | More tables can improve recall with more memory | | `numHyperplanes` | More hyperplanes make buckets narrower | | `seed` | Makes the index deterministic for tests | For production-scale retrieval, use a durable vector database. Anvia's in-memory LSH is best for local workflows and lightweight deployments. # Metadata Filters (/docs/guides/retrieval/metadata-filters) Metadata filters constrain retrieval before results are returned. Use them for tenant boundaries, product scopes, document status, and other application rules. ## 1. Store Metadata During Embedding [#1-store-metadata-during-embedding] ```ts const embedded = await embedDocuments(embeddings, documents, { id: (doc) => doc.id, content: (doc) => doc.body, metadata: (doc) => ({ tenantId: doc.tenantId, product: doc.product, rank: doc.rank, }), }); ``` Metadata values can be strings, numbers, booleans, or `null`. ## 2. Filter a Search [#2-filter-a-search] ```ts import { vectorFilter } from "@anvia/core"; const results = await index.search({ query: "billing limits", topK: 5, filter: vectorFilter.eq("tenantId", "acme"), }); ``` ## 3. Combine Filters [#3-combine-filters] ```ts const filter = vectorFilter.and( vectorFilter.eq("tenantId", "acme"), vectorFilter.eq("product", "billing"), ); const results = await index.search({ query: "invoice settings", topK: 5, filter, }); ``` Use `and(...)` and `or(...)` to build larger filters. ## 4. Use Range Filters [#4-use-range-filters] ```ts const results = await index.search({ query: "priority support", topK: 5, filter: vectorFilter.gt("rank", 2), }); ``` `gt(...)` and `lt(...)` compare numbers, strings, and booleans. ## 5. Apply Filters to Dynamic Context [#5-apply-filters-to-dynamic-context] ```ts const agent = new AgentBuilder("support", model) .dynamicContext(index, { topK: 3, filter: vectorFilter.eq("tenantId", tenantId), }) .build(); ``` Keep security-sensitive filters in application code. Do not ask the model to enforce tenant or permission boundaries. # RAG Context (/docs/guides/retrieval/rag-context) RAG context searches an index for each prompt and injects the matched documents into the model request. ## 1. Preprocess First [#1-preprocess-first] Build the index before the prompt runs. ```ts const embedded = await embedDocuments(embeddings, documents, { id: (doc) => doc.id, content: (doc) => `${doc.title}\n${doc.body}`, metadata: (doc) => ({ product: doc.product }), }); const index = InMemoryVectorStore.fromDocuments(embedded).index(embeddings); ``` Do not rebuild the index on every user prompt in production. ## 2. Attach Dynamic Context [#2-attach-dynamic-context] ```ts const agent = new AgentBuilder("support", model) .instructions("Use retrieved context when answering support questions.") .dynamicContext(index, { topK: 2, }) .build(); ``` On each prompt, Anvia searches the index using the prompt text and adds matching documents to the completion request. ## 3. Format Retrieved Documents [#3-format-retrieved-documents] ```ts const agent = new AgentBuilder("support", model) .dynamicContext(index, { topK: 2, format: (result) => ({ id: result.id, text: `${result.document.title}\n${result.document.body}`, }), }) .build(); ``` Use `format(...)` when your stored document is an object and the model should receive a specific text shape. ## 4. Add a Threshold [#4-add-a-threshold] ```ts const agent = new AgentBuilder("support", model) .dynamicContext(index, { topK: 3, threshold: 0.75, }) .build(); ``` Thresholds prevent weak matches from entering the prompt. ## 5. Run the Agent [#5-run-the-agent] ```ts const response = await agent .prompt("How long does a password reset link last?") .send(); ``` Use dynamic context for automatic retrieval. Use a search tool when the model should decide whether to search. ## 6. Choose The Retrieval Slot [#6-choose-the-retrieval-slot] Retrieval can feed different parts of a model request. | Pattern | Retrieves | Adds to | Use when | | ---------------------- | ------------------- | ----------- | --------------------------------------------------------------------- | | `.dynamicContext(...)` | documents or facts | `documents` | Every prompt should receive relevant knowledge | | `.dynamicTools(...)` | tool definitions | `tools` | The agent has a large tool catalog and should see only relevant tools | | `.tools([searchDocs])` | a search capability | `tools` | The model should decide whether and when to search | Dynamic tools reuse vector search, but they retrieve capabilities instead of knowledge. ```ts const toolIndex = await createToolIndex(embeddings, supportTools); const agent = new AgentBuilder("support", model) .dynamicContext(policyIndex, { topK: 3 }) .dynamicTools(toolIndex, { topK: 5 }) .build(); ``` # Vector Stores (/docs/guides/retrieval/vector-stores) A vector store keeps embedded documents and exposes a searchable index. Anvia ships an in-memory store for local workflows, tests, and small prototypes. ## 1. Build a Store [#1-build-a-store] ```ts import { InMemoryVectorStore } from "@anvia/core"; const store = InMemoryVectorStore.fromDocuments(embedded); const index = store.index(embeddings); ``` Use the same embedding model that created the document embeddings. ## 2. Search Documents [#2-search-documents] ```ts const results = await index.search({ query: "How long does a password reset link last?", topK: 3, }); console.log(results[0]?.id); console.log(results[0]?.score); console.log(results[0]?.document); ``` `search(...)` returns scored documents sorted by relevance. ## 3. Search Only IDs [#3-search-only-ids] ```ts const matches = await index.searchIds({ query: "priority support", topK: 2, }); ``` Use `searchIds(...)` when you want to load full records from your own database. ## 4. Add or Replace Documents [#4-add-or-replace-documents] ```ts store.addDocuments(await embedDocuments(embeddings, updatedDocs, { id: (doc) => doc.id, content: (doc) => doc.body, })); ``` When a document id already exists, the later document replaces it. ## 5. Expose Search as a Tool [#5-expose-search-as-a-tool] ```ts const searchDocs = index.asTool({ name: "search_docs", description: "Search support documentation.", topK: 3, }); ``` Use a search tool when the model should decide when retrieval is needed. Use [RAG Context](/docs/guides/retrieval/rag-context) when every prompt should receive retrieved context automatically. # Attachments (/docs/guides/sdk-fundamentals/attachments) Attachments are rich content parts inside a user message. Use them when the model should inspect an image, PDF, text document, or provider-supported file input as part of the prompt. Provider support varies. Check the model before building a workflow that depends on images or documents. ## 1. Send an Image [#1-send-an-image] ```ts import { AgentBuilder, Message, UserContent } from "@anvia/core"; const agent = new AgentBuilder("visual-support", model) .instructions("Inspect screenshots and answer with concrete UI observations.") .build(); const response = await agent .prompt( Message.user([ UserContent.text("What error state is visible in this screenshot?"), UserContent.imageUrl("https://example.com/screenshot.png", { detail: "auto", }), ]), ) .send(); ``` Use `imageBase64(...)` when your application already has the image bytes: ```ts Message.user([ UserContent.text("Read the label in this image."), UserContent.imageBase64(base64Image, "image/png"), ]); ``` ## 2. Send a Document [#2-send-a-document] ```ts import { Message, UserContent } from "@anvia/core"; await agent .prompt( Message.user([ UserContent.text("Summarize the attached report."), UserContent.documentUrl("https://example.com/report.pdf", "application/pdf", { filename: "report.pdf", }), ]), ) .send(); ``` Use `documentBase64(...)` for application-owned files and `documentText(...)` for plain text content that should be treated as part of the message. ## 3. Persist Attachment History [#3-persist-attachment-history] Conversation history is still a plain `Message[]`: ```ts const history = await conversations.loadMessages(conversationId); const response = await agent.prompt([...history, currentMessage]).send(); await conversations.saveMessages(conversationId, [ ...history, ...response.messages, ]); ``` Store only attachment data your product is allowed to retain. For large files, consider storing durable file references in your application and reconstructing supported message content at request time. ## 4. Choose Attachments or Retrieval [#4-choose-attachments-or-retrieval] | Use attachments when | Use retrieval when | | ------------------------------------------------ | ---------------------------------------------- | | The file is part of the current user request | The model needs a searchable knowledge base | | The provider model can inspect the file directly | You need tenant filters, freshness, or ranking | | The file is small enough for the prompt budget | The source corpus is large or reused often | For large document collections, preprocess text into embeddings and use [RAG Context](/docs/guides/retrieval/rag-context). ## 5. Provider Notes [#5-provider-notes] | Provider | Current guidance | | -------------------- | --------------------------------------------------------------------------------------- | | OpenAI | Use provider models that support the attachment type you send | | Anthropic | Attachment support depends on the selected model and provider SDK mapping | | Gemini | Supports image and document attachments; selected model limits still apply | | Mistral | Current Anvia Mistral support intentionally rejects image and document file attachments | | Compatible providers | Treat support as endpoint-specific, even when the API shape is compatible | Unsupported attachments usually fail as provider validation errors. Handle them at the same application boundary where you handle model configuration or request failures. # Provider Clients and Models (/docs/guides/sdk-fundamentals/clients-and-models) Anvia separates provider setup from model usage. A client knows how to talk to a provider. A model is the reusable capability you pass into agents, extractors, pipelines, and retrieval code. ## 1. Create a Client [#1-create-a-client] Use the provider client that matches your runtime. ```ts import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ apiKey }); ``` Provider clients only use the values you pass to their constructors. Your app owns whether those values come from environment variables, a secret manager, user input, or test fixtures. ```ts const client = new OpenAIClient({ apiKey, }); ``` ## 2. Create a Completion Model [#2-create-a-completion-model] ```ts const model = client.completionModel("gpt-5.5"); ``` The completion model implements Anvia's normalized `CompletionModel` interface. That means the rest of the SDK can use it without depending on one provider's request format. Every completion model also exposes required metadata: ```ts model.provider; model.defaultModel; model.capabilities; ``` Capabilities describe what the Anvia adapter supports, such as tools, streaming, image input, document file input, and output schemas. Core validates requests against those capabilities before calling the provider SDK, so unsupported requests fail early with an Anvia `CompletionCapabilityError`. For example, OpenAI and Gemini adapters expose image and document input, while Mistral currently rejects those attachment types. ## 3. Pass the Model Into an Agent [#3-pass-the-model-into-an-agent] ```ts import { AgentBuilder } from "@anvia/core"; const agent = new AgentBuilder("triage", model) .instructions("Classify incoming support messages.") .build(); ``` Models are reusable. One model can power several agents: ```ts const supportAgent = new AgentBuilder("support", model).build(); const billingAgent = new AgentBuilder("billing", model).build(); ``` ## 4. Use Provider-Specific Or Compatible Clients When Needed [#4-use-provider-specific-or-compatible-clients-when-needed] OpenAI-compatible APIs: ```ts import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: "https://openrouter.ai/api/v1", apiKey, }); const model = client.completionModel("openai/gpt-5.5"); ``` Anthropic: ```ts import { AnthropicClient } from "@anvia/anthropic"; const client = new AnthropicClient({ apiKey }); const model = client.completionModel("claude-opus-4-6"); ``` Gemini: ```ts import { GeminiClient } from "@anvia/gemini"; const client = new GeminiClient({ apiKey }); const model = client.completionModel("gemini-3.1-flash-lite-preview"); ``` Mistral: ```ts import { MistralClient } from "@anvia/mistral"; const client = new MistralClient({ apiKey }); const model = client.completionModel("mistral-large-latest"); ``` Vertex AI: ```ts const client = new GeminiClient({ vertexai: true, project, location, }); ``` ## 5. Use Embedding Models For Retrieval [#5-use-embedding-models-for-retrieval] Some provider clients can also create embedding models: ```ts const embeddings = client.embeddingModel("text-embedding-3-small"); ``` Embedding models are used by retrieval and vector store workflows. Completion models answer prompts. Keep those capabilities separate so each workflow can swap providers independently. ## Capability Validation [#capability-validation] Swapping providers works best when your request only uses features supported by both adapters. For example, a plain text prompt can move between OpenAI, Anthropic, Gemini, and Mistral. A prompt with image input can use adapters that expose image input, such as OpenAI and Gemini, but cannot use the current Mistral adapter. When you build agents, extractors, or direct completion requests, Anvia checks the selected model's capabilities before transport. Provider-specific options passed through `additionalParams` are still your responsibility; they may make a request non-portable even when the core capability check passes. ## Choosing Where To Store Models [#choosing-where-to-store-models] Create clients and models once per process, request handler module, or dependency container. Then pass the model into builders where needed. Avoid creating a new provider client for every prompt unless your application specifically needs per-request credentials or base URLs. ## Next [#next] Read [Prompt Requests](/docs/guides/sdk-fundamentals/prompt-requests) to see how agents turn prompts, history, context, tools, and runtime options into normalized model requests. # Errors and Cancellation (/docs/guides/sdk-fundamentals/errors) Anvia errors usually fall into one of five groups: 1. invalid setup 2. schema validation 3. provider failures 4. tool failures 5. agent runtime limits Handle them at the boundary where your application can make the right product decision. ## Invalid Setup [#invalid-setup] Provider clients validate required configuration. Pass credentials explicitly when constructing clients. ```ts import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: "https://openrouter.ai/api/v1", apiKey, }); ``` If a required credential is missing, fail early during startup or dependency creation instead of waiting for the first user request. ## Schema Validation [#schema-validation] Tools validate input with Zod before running `execute`. ```ts import { createTool } from "@anvia/core"; import { z } from "zod"; const lookupOrder = createTool({ name: "lookup_order", description: "Look up an order.", input: z.object({ orderId: z.string(), }), async execute({ orderId }) { return { orderId, status: "shipped" }; }, }); ``` If the model calls the tool with invalid arguments, Zod throws. Treat this as a model/tool boundary failure. Log the tool name, validated schema, and raw arguments when possible. ## Provider Failures [#provider-failures] Provider SDKs can throw for network errors, authentication errors, rate limits, unsupported attachments, or provider-specific validation. Keep provider retries and fallback policy in application code: ```ts try { const response = await agent.prompt("Summarize this ticket.").send(); return response.output; } catch (error) { reportAgentError(error); throw error; } ``` ## Tool Failures [#tool-failures] Tool `execute` functions are your application code. They can fail because of permissions, missing records, unavailable services, or unexpected input. Prefer explicit tool results for expected product states: ```ts async execute({ orderId }) { const order = await findOrder(orderId); if (order === undefined) { return { status: "not_found" }; } return { status: order.status }; } ``` Throw only when the workflow should fail or be retried. ## Agent Runtime Limits [#agent-runtime-limits] Anvia agents use turn limits to prevent unbounded tool loops. If the model keeps requesting tools past the configured limit, the prompt fails with `MaxTurnsError`. ```ts import { AgentBuilder, MaxTurnsError } from "@anvia/core"; const agent = new AgentBuilder("support", model) .tool(lookupOrder) .defaultMaxTurns(3) .build(); try { await agent.prompt("Check this order.").send(); } catch (error) { if (error instanceof MaxTurnsError) { console.error("Agent exceeded the configured turn limit."); } } ``` Start with low turn limits. Increase them only when the workflow requires multiple tool calls. ## Prompt Cancellation [#prompt-cancellation] Hooks and approval runtimes can cancel a prompt. Catch cancellation at the same boundary where you would handle a user-denied action or interrupted workflow. ```ts import { PromptCancelledError } from "@anvia/core"; try { await agent.prompt("Run the guarded workflow.").send(); } catch (error) { if (error instanceof PromptCancelledError) { return "The request was cancelled."; } throw error; } ``` ## Practical Handling Pattern [#practical-handling-pattern] Use a small wrapper around agent calls: ```ts export async function runSupportAgent(prompt: string) { try { return await supportAgent.prompt(prompt).send(); } catch (error) { logAgentFailure(error); throw error; } } ``` Inside the wrapper, classify known SDK errors, attach request metadata, and preserve the original error for debugging. # Memory and Sessions (/docs/guides/sdk-fundamentals/memory-and-sessions) Memory is durable conversation state owned by an agent. Configure a memory store once, then choose a session id for each conversation. ```ts import { AgentBuilder, type MemoryAppendInput, type MemoryContext, type MemoryStore, type Message, } from "@anvia/core"; class AppMemoryStore implements MemoryStore { private readonly sessions = new Map(); async load(context: MemoryContext): Promise { return [...(this.sessions.get(context.sessionId) ?? [])]; } async append(input: MemoryAppendInput): Promise { const current = this.sessions.get(input.context.sessionId) ?? []; this.sessions.set(input.context.sessionId, [...current, ...input.messages]); } async clear(context: MemoryContext): Promise { this.sessions.delete(context.sessionId); } } const memory = new AppMemoryStore(); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .memory(memory) .build(); await agent.session("thread_123", { userId: "user_456" }).prompt("Remember my plan.").send(); await agent.session("thread_123", { userId: "user_456" }).prompt("What is my plan?").send(); ``` ## Mental Model [#mental-model] Use `agent.prompt("...")` for stateless one-off requests. Use `agent.prompt([...messages])` when you already have an explicit transcript. The last message is the active prompt and earlier messages are temporary request history. Use `agent.session(id).prompt("...")` when Anvia should load and save durable conversation messages through the configured memory store. ## Save Policy [#save-policy] Memory defaults to `savePolicy: "message"`. This saves the user prompt, completed assistant messages, and completed tool result messages as soon as they are ready. ```ts new AgentBuilder("support", model).memory(memory, { savePolicy: "turn" }); ``` Available policies are: | Policy | Behavior | | ----------- | ---------------------------------------------------------------------------- | | `"message"` | Save each completed message immediately. Best recovery for long failed runs. | | `"turn"` | Save completed messages after each model/tool turn. Fewer writes. | | `"run"` | Save only after a successful final response. Simplest persistence behavior. | On failure, memory stores with `recordError(...)` receive the error and partial run messages. ## Migration From Explicit History [#migration-from-explicit-history] Old request history should become an explicit transcript: ```ts const history = await conversations.loadMessages(conversationId); const response = await agent .prompt([...history, Message.user(userInput)]) .send(); ``` Use sessions when you want core to own the load/save cycle: ```ts const response = await agent.session(conversationId).prompt(userInput).send(); ``` For storage adapter examples, see the dedicated [Memory](/docs/guides/memory) section. # Messages and History (/docs/guides/sdk-fundamentals/messages-and-history) Anvia history is a plain `Message[]`. For explicit stateless history, pass the whole transcript to `agent.prompt([...])`; the last message is the active prompt and earlier messages are history. ## Message Roles [#message-roles] | Role | Shape | Use for | | ----------- | ---------------------------------------------------- | ------------------------------------------------- | | `system` | `{ role: "system", content: string }` | System-level instructions in raw message history | | `user` | `{ role: "user", content: UserContent[] }` | User text, images, and documents | | `assistant` | `{ role: "assistant", content: AssistantContent[] }` | Assistant text, reasoning, images, and tool calls | | `tool` | `{ role: "tool", content: ToolContent[] }` | Tool results returned after assistant tool calls | Most application history contains user and assistant messages. Tool-using runs can also include tool messages. ## Text History [#text-history] ```ts import type { Message } from "@anvia/core"; const history: Message[] = [ { role: "user", content: [{ type: "text", text: "My checkout failed." }], }, { role: "assistant", content: [ { type: "text", text: "I can help. What error did you see?", }, ], }, ]; ``` You can create the same shape with helpers: ```ts import { Message } from "@anvia/core"; const history = [ Message.user("My checkout failed."), Message.assistant("I can help. What error did you see?"), ]; ``` ## Persisting History [#persisting-history] `response.messages` contains only the new messages created by one run. Append those messages to your stored conversation. ```ts const history = await conversations.loadMessages(conversationId); const currentPrompt = Message.user(userInput); const response = await agent.prompt([...history, currentPrompt]).send(); await conversations.saveMessages(conversationId, [ ...history, ...response.messages, ]); ``` Persist the plain message objects returned by Anvia. JSON storage is enough for most applications. ## Tool Calls In History [#tool-calls-in-history] Tool calls are assistant content. Tool results are tool content. ```ts import type { Message } from "@anvia/core"; const historyWithToolCall: Message[] = [ { role: "user", content: [{ type: "text", text: "Where is order A-100?" }], }, { role: "assistant", content: [ { type: "tool_call", id: "call_1", function: { name: "lookup_order", arguments: { orderId: "A-100" }, }, }, ], }, { role: "tool", content: [ { type: "tool_result", id: "call_1", content: [{ type: "text", text: "{\"status\":\"shipped\"}" }], }, ], }, { role: "assistant", content: [{ type: "text", text: "Order A-100 has shipped." }], }, ]; ``` Only create tool-call history yourself when importing an existing transcript or replaying a known run. For live runs, persist `response.messages`. ## Reasoning In History [#reasoning-in-history] Assistant messages can include provider-returned reasoning content. The stable display field is `reasoning.text`; adapters may also include structured `content` blocks for text, summaries, encrypted state, and redacted state. ```ts import { AssistantContent, Message } from "@anvia/core"; const assistant = Message.assistant([ AssistantContent.reasoningFromContent([ { type: "summary", text: "Checked the policy and the tool result." }, { type: "encrypted", data: "provider-opaque-state" }, ]), AssistantContent.text("Order A-100 has shipped."), ]); ``` Persist reasoning objects exactly as returned when you store history. Provider adapters use signatures, encrypted blocks, and redacted blocks to maintain reasoning continuity across tool calls. ## Attachments [#attachments] User messages can include text, images, and documents: ```ts import { Message, UserContent } from "@anvia/core"; const message = Message.user([ UserContent.text("Summarize this report."), UserContent.documentUrl("https://example.com/report.pdf", "application/pdf", { filename: "report.pdf", }), ]); ``` Provider support for attachments varies. Check the provider and model before building a workflow that depends on images or documents. ## Next [#next] Read [Prompt Responses](/docs/guides/sdk-fundamentals/prompt-responses) to see what an agent run returns. # Import Paths (/docs/guides/sdk-fundamentals/package-exports) The easiest import path is the root package: ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; ``` The root export includes the core public SDK. Use it for most applications and examples. Studio is a separate package: ```ts import { Studio } from "@anvia/studio"; ``` Use Studio when you want the local playground, sessions, approval UI, and trace browser around built agents. ## Focused Subpath Exports [#focused-subpath-exports] Anvia also exposes focused subpaths when you want narrower imports. | Export | Use for | | --------------------------- | ---------------------------------------------------------------- | | `@anvia/core/agent` | Agents, agent builders, prompt requests, hooks, and agent errors | | `@anvia/core/completion` | Completion request types, messages, content helpers, and usage | | `@anvia/core/embeddings` | Embedding interfaces and shared embedding types | | `@anvia/core/evals` | Eval suites, built-in metrics, agent eval targets, and reporters | | `@anvia/core/extractor` | Structured extraction builders and runtime | | `@anvia/core/mcp` | MCP connections, tool adapters, and result handling | | `@anvia/core/observability` | Observers, trace groups, and runtime observation types | | `@anvia/core/pipeline` | Pipeline builders and pipeline execution | | `@anvia/core/skills` | Local skill loading, skill instructions, and skill tools | | `@anvia/core/streaming` | Streaming helpers such as readable stream adapters | | `@anvia/core/tool` | Tool creation, tool sets, tool registries, and tool errors | | `@anvia/core/vector-store` | Vector store interfaces, filters, and local indexes | ## Root Import Example [#root-import-example] ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; ``` Root imports are clear for small apps, examples, and docs. ## Subpath Import Example [#subpath-import-example] ```ts import { AgentBuilder } from "@anvia/core/agent"; import { OpenAIClient } from "@anvia/openai"; import { createTool } from "@anvia/core/tool"; ``` Subpath imports can make ownership clearer in larger codebases, especially when one module should only work with tools, providers, or completion message types. ## Recommendation [#recommendation] Start with root imports while learning. Move to subpath imports when a file has a narrow responsibility or when your project style prefers explicit module boundaries. # Prompt Requests (/docs/guides/sdk-fundamentals/prompt-requests) Prompt requests are the boundary between your application and an agent run. Most workflows start with: ```ts const response = await agent.prompt("Summarize this support ticket.").send(); ``` That line creates a `PromptRequest`, builds a normalized completion request, sends it to the model, handles tools if needed, and returns a final prompt response. ## 1. Start With a Prompt [#1-start-with-a-prompt] The simplest prompt is a string: ```ts await agent.prompt("Write a short onboarding checklist.").send(); ``` Use a structured `Message` when you need explicit content parts: ```ts import { Message, UserContent } from "@anvia/core"; await agent .prompt( Message.user([ UserContent.text("Inspect this screenshot."), UserContent.imageUrl("https://example.com/screenshot.png", { detail: "auto", }), ]), ) .send(); ``` Provider support for images and documents depends on the provider model you choose. ## 2. Add History When Needed [#2-add-history-when-needed] ```ts const history = await conversations.loadMessages(conversationId); const response = await agent .prompt([...history, Message.user(userInput)]) .send(); ``` History is not global state inside stateless prompts. You load it from your application and pass an explicit transcript into the request. ## 3. Override Runtime Options Per Prompt [#3-override-runtime-options-per-prompt] Prompt requests can override run behavior without rebuilding the agent: ```ts const response = await agent .prompt("Check this order.") .maxTurns(2) .withToolConcurrency(2) .send(); ``` Use this when one request needs a tighter turn limit, different tool concurrency, approval handling, hooks, or tracing. ## 4. Request Flow [#4-request-flow] When you call `send()`, Anvia builds a normalized completion request in this order: 1. Start with the current prompt. 2. Add any earlier messages from `prompt(Message[])`. 3. Add agent instructions. 4. Add static context from `.context(...)`. 5. Fetch dynamic context if retrieval is configured. 6. Add tool definitions from tools, skills, and MCP servers. 7. Add runtime settings such as temperature, max tokens, tool choice, and output schema. 8. Send the request to the completion model. The model response can include text, reasoning, and tool calls. If tool calls are present, Anvia executes the tools and loops until the model returns final text or the turn limit is reached. ## 5. Stream Instead of Waiting [#5-stream-instead-of-waiting] Use `stream()` when the UI or job runner needs incremental events: ```ts for await (const event of agent.prompt("Draft a launch checklist.").stream()) { if (event.type === "text_delta") { process.stdout.write(event.delta); } if (event.type === "final") { console.log(event.output); } } ``` Streaming emits normalized events for text deltas, reasoning deltas, tool calls, tool results, turn boundaries, final output, and errors. ## Next [#next] Read [Messages and History](/docs/guides/sdk-fundamentals/messages-and-history) to understand the `Message[]` shape used by prompts, history, and responses. # Prompt Responses (/docs/guides/sdk-fundamentals/prompt-responses) `send()` returns a `PromptResponse`. ```ts const response = await agent.prompt("Explain cache invalidation.").send(); console.log(response.output); console.log(response.usage.totalTokens); ``` ## Response Shape [#response-shape] | Field | Type | Meaning | | | ---------- | ---------------- | ----------------------------------------------- | -------------------------------------- | | `output` | `string` | Text extracted from the final assistant message | | | `usage` | `Usage` | Aggregated token usage across model calls | | | `messages` | `Message[]` | New messages created during this prompt run | | | `trace` | \`AgentTraceInfo | undefined\` | Trace metadata when tracing is enabled | ## Output [#output] `output` is the convenient text result: ```ts const response = await agent.prompt("Write a concise summary.").send(); return response.output; ``` Use `output` when your workflow expects normal assistant text. Use structured output or extractors when your workflow needs schema-shaped data. ## Usage [#usage] Usage is normalized across providers: ```ts console.log(response.usage.inputTokens); console.log(response.usage.outputTokens); console.log(response.usage.totalTokens); console.log(response.usage.cachedInputTokens); ``` Use usage data for logs, analytics, budgets, and rate-limit decisions. ## Messages [#messages] `messages` contains only the new messages created during this prompt run: ```ts import { Message } from "@anvia/core"; const history = await conversations.loadMessages(conversationId); const response = await agent .prompt([...history, Message.user(userInput)]) .send(); await conversations.saveMessages(conversationId, [ ...history, ...response.messages, ]); ``` If the agent called tools, `messages` can include: * the user prompt * assistant tool calls * tool result messages * the final assistant message ## Trace [#trace] `trace` is present when tracing is enabled for the run: ```ts const response = await agent .prompt("Run the traced workflow.") .withTrace({ name: "support-check" }) .send(); console.log(response.trace); ``` Use observers and tracing when you need to inspect runs, generations, tool calls, and usage. ## Next [#next] Read [Errors and Cancellation](/docs/guides/sdk-fundamentals/errors) to understand common failure modes and runtime limits. # How Anvia Works (/docs/guides/sdk-fundamentals/runtime-boundaries) Anvia is designed around explicit runtime boundaries. Each object has a small job, and your application stays in control of data, permissions, persistence, and side effects. ## The Boundary Map [#the-boundary-map] | Layer | Example | Responsibility | | ----------- | ------------------------------------------------- | ------------------------------------------------------------------------ | | Application | Your server, worker, route, or job | Owns product logic, auth, database access, and deployment | | Client | `OpenAIClient`, `AnthropicClient`, `GeminiClient` | Owns provider credentials and provider SDK setup | | Model | `client.completionModel(...)` | Executes normalized completion requests | | Agent | `new AgentBuilder(...).build()` | Adds instructions, context, tools, hooks, output schema, and turn limits | | Tool | `createTool(...)` | Exposes a typed application action to the agent | | Request | `agent.prompt(...).send()` | Runs one prompt through the agent runtime | ## How The Pieces Connect [#how-the-pieces-connect] ```txt Application -> Client -> Model -> Agent -> Prompt Request -> Prompt Response | | | +-> Tools, context, history, hooks, tracing | +-> Extractors, pipelines, retrieval ``` The application creates and owns the runtime objects. Anvia coordinates the prompt run, but your app still decides what data is loaded, which tools are available, where history is stored, and how side effects are authorized. Built agents, extractors, pipelines, models, and tools are ordinary TypeScript objects. Create them during startup, request setup, or tests, then pass them to the code that needs them. Anvia does not require a global application runtime to discover your AI features. ## Step By Step [#step-by-step] ### 1. Keep Provider Setup In a Client [#1-keep-provider-setup-in-a-client] ```ts import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ apiKey }); ``` The client owns API keys, base URLs, default headers, and provider SDK construction. ### 2. Reuse Models [#2-reuse-models] ```ts const model = client.completionModel("gpt-5.5"); ``` A model is a reusable capability. You can pass the same model to multiple agents, extractors, and pipelines. ### 3. Put Behavior In the Agent [#3-put-behavior-in-the-agent] ```ts import { AgentBuilder } from "@anvia/core"; const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .defaultMaxTurns(3) .build(); ``` The agent owns runtime behavior around the model. The stable id, `support`, should not change casually because it can be used by tracing, Studio, and multi-agent workflows. ### 4. Keep Side Effects In Tools [#4-keep-side-effects-in-tools] ```ts import { createTool } from "@anvia/core"; import { z } from "zod"; const lookupCustomer = createTool({ name: "lookup_customer", description: "Look up a customer by email.", input: z.object({ email: z.string().email(), }), async execute({ email }) { return { email, plan: "pro" }; }, }); ``` Tools are ordinary application code with validation at the boundary. This is where you enforce product permissions and decide which side effects are allowed. ### 5. Run Work From Application Code [#5-run-work-from-application-code] ```ts const response = await agent .prompt("Can customer mira@example.com use priority support?") .send(); console.log(response.output); ``` The prompt request runs one agent workflow. It can use history, stream events, request tool approvals, attach traces, or override the turn limit. ## What Anvia Does Not Own [#what-anvia-does-not-own] Anvia does not need to own: * your database connection * user authentication * permission checks * durable message storage * queueing and retries * deployment topology Keep those in your application. Pass only the context, tools, and model capabilities the agent needs for one workflow. ## Practical Rule [#practical-rule] If a decision affects product correctness, security, or data ownership, keep it in application code or a tool. If a decision affects how a model is prompted, constrained, streamed, or observed, configure it on the agent or prompt request. ## Next [#next] Read [Provider Clients and Models](/docs/guides/sdk-fundamentals/clients-and-models) to configure provider access and reusable model capabilities. # Local Skills (/docs/guides/skills/local-skills) Skills are local capability packages. They let an agent discover focused instructions, reference files, and executable scripts only when that skill is relevant. ## 1. Create a Skills Directory [#1-create-a-skills-directory] ```txt skills/ code-review/ SKILL.md references/ checklist.md scripts/ lint.sh ``` Each skill lives in its own folder. The folder name must match the `name` field in `SKILL.md`. ## 2. Load Local Skills [#2-load-local-skills] ```ts import { loadSkills, skill } from "@anvia/core"; const skillSet = await loadSkills(skill.local("./skills")); ``` `skill.local("./skills")` scans child directories that contain `SKILL.md`. You can also point directly at one skill directory. ```ts const skillSet = await loadSkills(skill.local("./skills/code-review")); ``` ## 3. Attach Skills to an Agent [#3-attach-skills-to-an-agent] ```ts const agent = new AgentBuilder("developer-assistant", model) .instructions("Help with repository maintenance.") .skills(skillSet) .defaultMaxTurns(3) .build(); ``` `.skills(skillSet)` adds the skill summary instructions and registers the skill tools on the agent. ## 4. Let the Agent Load Details On Demand [#4-let-the-agent-load-details-on-demand] The agent receives a compact skill list first. It can then call skill tools to load the full instructions, reference files, or scripts only when needed. ```ts const response = await agent .prompt("Review this pull request for correctness.") .send(); ``` Use skills for optional capabilities. Put always-on behavior in normal agent instructions. # Skill Files (/docs/guides/skills/skill-files) Every local skill is a directory with a required `SKILL.md` file and optional `references/` and `scripts/` folders. ## 1. Write SKILL.md [#1-write-skillmd] ```md --- name: code-review description: Review code for correctness and maintainability. --- # Code Review Prioritize correctness, regressions, missing tests, and risky behavior. Return findings with file and line references when possible. ``` The frontmatter is required. | Field | Required | Notes | | ------------- | -------- | -------------------------------------------- | | `name` | yes | lowercase letters, numbers, and hyphens only | | `description` | yes | short summary shown in the skill list | | `license` | no | optional metadata | | `metadata` | no | optional object for your application | ## 2. Match the Directory Name [#2-match-the-directory-name] ```txt skills/ code-review/ SKILL.md ``` The directory must be named `code-review` when `SKILL.md` uses `name: code-review`. ## 3. Add Reference Files [#3-add-reference-files] ```txt skills/ code-review/ references/ checklist.md style-guide.md ``` Reference files are read-only context the agent can load with `get_skill_reference`. ## 4. Add Scripts [#4-add-scripts] ```txt skills/ code-review/ scripts/ lint.sh ``` Scripts are executable helpers the agent can run with `run_skill_script`. ```sh chmod +x skills/code-review/scripts/lint.sh ``` Keep scripts narrow and deterministic. They run from the skill directory and receive string arguments. ## Validation Rules [#validation-rules] Anvia validates skill files while loading: | Rule | Failure | | ---------------------------------------- | ---------------------- | | Missing frontmatter | `SkillValidationError` | | Invalid name | `SkillValidationError` | | Directory name does not match skill name | `SkillValidationError` | | Missing description | `SkillValidationError` | Next, load the directory with [Skill Loading](/docs/guides/skills/skill-loading). # Skill Instructions (/docs/guides/skills/skill-instructions) Skill instructions are split into two layers: a compact skill list that is always added to the agent, and full `SKILL.md` guidance that the model loads only when needed. ## 1. Inspect the Generated Summary [#1-inspect-the-generated-summary] ```ts const skillSet = await loadSkills(skill.local("./skills")); console.log(skillSet.instructions); ``` The summary tells the model: * which skills are available * each skill description * which reference files are available * which scripts are available * which skill tools can load details ## 2. Add Skills After Base Instructions [#2-add-skills-after-base-instructions] ```ts const agent = new AgentBuilder("assistant", model) .instructions("Be concise and verify repository facts before answering.") .skills(skillSet) .build(); ``` Anvia appends skill instructions after your base instructions. Keep base instructions for behavior that always applies. Keep skill details inside `SKILL.md`. ## 3. Load Full Instructions On Demand [#3-load-full-instructions-on-demand] During a run, the model can call: ```ts get_skill_instructions({ skillName: "code-review", }); ``` That returns the body of `skills/code-review/SKILL.md`, without the frontmatter. ## 4. Keep SKILL.md Focused [#4-keep-skillmd-focused] Good skill instructions explain: | Include | Avoid | | ---------------------------------- | ------------------------------------ | | when to use the skill | general agent behavior | | exact review or workflow rules | unrelated background | | required output shape | large reference content | | when to load references or scripts | secrets or environment-specific data | Put large supporting material in `references/` so it can be loaded separately. # Skill Loading (/docs/guides/skills/skill-loading) Use `loadSkills(...)` to resolve one or more skill loaders into a `SkillSet`. ## 1. Load One Directory [#1-load-one-directory] ```ts import { loadSkills, skill } from "@anvia/core"; const skillSet = await loadSkills(skill.local("./skills")); ``` The returned `SkillSet` contains: ```ts skillSet.skills; skillSet.instructions; skillSet.tools; ``` ## 2. Load Multiple Sources [#2-load-multiple-sources] ```ts const skillSet = await loadSkills([ skill.local("./skills/base"), skill.local("./skills/project"), ]); ``` When two loaders return the same skill name, the later loader wins. Use this to override shared skills with project-specific versions. ## 3. Handle Validation Errors [#3-handle-validation-errors] ```ts import { SkillValidationError } from "@anvia/core"; try { const skillSet = await loadSkills(skill.local("./skills")); return skillSet; } catch (error) { if (error instanceof SkillValidationError) { console.error(error.issues); } throw error; } ``` Each validation issue includes the file path and a message. ## 4. Attach the Result Once [#4-attach-the-result-once] ```ts const skillSet = await loadSkills(skill.local("./skills")); const agent = new AgentBuilder("assistant", model) .skills(skillSet) .build(); ``` Load skills during startup or request setup, then pass the `SkillSet` to the agent builder. ## Empty Skill Sets [#empty-skill-sets] If no skills are loaded, `skillSet.instructions` is an empty string and `skillSet.tools` is an empty array. # Skill Tools (/docs/guides/skills/skill-tools) When `loadSkills(...)` finds skills, Anvia creates four tools and includes them in `skillSet.tools`. ## Generated Tools [#generated-tools] | Tool | Purpose | | ------------------------ | --------------------------------------------------- | | `get_skill_instructions` | Load the full `SKILL.md` body for one skill | | `get_skill_reference` | Read a file from the skill's `references/` folder | | `get_skill_script` | Read a file from the skill's `scripts/` folder | | `run_skill_script` | Execute a script from the skill's `scripts/` folder | ## 1. Register Tools Through AgentBuilder [#1-register-tools-through-agentbuilder] ```ts const agent = new AgentBuilder("assistant", model) .skills(skillSet) .defaultMaxTurns(3) .build(); ``` This is the normal path. `.skills(skillSet)` adds both instructions and tools. ## 2. Call Skill Tools Directly [#2-call-skill-tools-directly] Direct calls are useful in tests and admin workflows. ```ts import { ToolSet } from "@anvia/core"; const toolSet = new ToolSet().addTools(skillSet.tools); const instructions = await toolSet.call( "get_skill_instructions", JSON.stringify({ skillName: "code-review" }), ); ``` ## 3. Read a Reference File [#3-read-a-reference-file] ```ts const checklist = await toolSet.call( "get_skill_reference", JSON.stringify({ skillName: "code-review", referencePath: "checklist.md", }), ); ``` Only files listed under the skill's `references/` directory can be read. ## 4. Run a Script [#4-run-a-script] ```ts const result = await toolSet.call( "run_skill_script", JSON.stringify({ skillName: "code-review", scriptPath: "lint.sh", args: ["packages/core/src"], timeoutMs: 30_000, }), ); ``` Scripts run with a timeout and return formatted stdout and stderr. Failed scripts throw tool errors. ## Security Boundary [#security-boundary] Skill tools reject absolute paths and path traversal. Still, treat scripts as application-owned code: review them, keep them narrow, and avoid passing secrets unless the script explicitly needs them. # Readable Streams (/docs/guides/streaming/readable-streams) Use `readableStream()` or `toReadableStream(...)` when an HTTP route should return newline-delimited JSON events. ## 1. Stream an Agent Request [#1-stream-an-agent-request] ```ts const stream = agent.prompt("Draft a reply.").readableStream(); ``` `readableStream()` converts the agent stream into a `ReadableStream`. ## 2. Return It From an HTTP Route [#2-return-it-from-an-http-route] ```ts return new Response(agent.prompt("Draft a reply.").readableStream(), { headers: { "Content-Type": "application/x-ndjson", }, }); ``` Each line is one JSON event. ```jsonl {"type":"text_delta","turn":1,"delta":"Hello"} {"type":"final","output":"Hello","usage":{"totalTokens":12}} ``` ## 3. Convert Any Async Iterable [#3-convert-any-async-iterable] ```ts import { toReadableStream } from "@anvia/core"; const stream = toReadableStream(agent.prompt("Draft a reply.").stream()); ``` Use this helper when you already have an async iterable of events. ## 4. Handle Stream Errors [#4-handle-stream-errors] If iteration fails, Anvia emits one final JSON line with `type: "error"` and then closes the stream. ```jsonl {"type":"error","error":{"name":"Error","message":"provider failed"}} ``` Clients should handle both `final` and `error` events. # Stream Accumulation (/docs/guides/streaming/stream-accumulation) Anvia streams deltas while also accumulating the final assistant message, tool calls, usage, and history. ## 1. Render Deltas Immediately [#1-render-deltas-immediately] ```ts let text = ""; for await (const event of agent.prompt("Write a reply.").stream()) { if (event.type === "text_delta") { text += event.delta; render(text); } } ``` Use deltas for live UI updates. ## 2. Use the Final Event for State [#2-use-the-final-event-for-state] ```ts for await (const event of agent.prompt("Write a reply.").stream()) { if (event.type === "final") { await saveMessages(event.messages); await saveUsage(event.usage); } } ``` Use the final event for durable application state. It contains the completed output and the message history created by the run. ## 3. Tool Calls Are Accumulated Across Deltas [#3-tool-calls-are-accumulated-across-deltas] Providers can stream tool call names and arguments in chunks. Anvia buffers those chunks and emits a normalized `tool_call` event when the call is complete. ```ts for await (const event of agent.prompt("Add 2 and 5.").stream()) { if (event.type === "tool_call") { console.log(event.toolCall.function.name); console.log(event.toolCall.function.arguments); } } ``` After tool execution, Anvia emits `tool_result` and continues the run with the next model turn. ## 4. Reasoning Deltas Are Preserved [#4-reasoning-deltas-are-preserved] When a provider exposes reasoning deltas, Anvia streams them as `reasoning_delta` events and includes the accumulated reasoning content in the final assistant message. Providers can mark deltas with `contentType: "summary"`, `"text"`, `"encrypted"`, or `"redacted"`; plain deltas still work as legacy reasoning text. ```ts for await (const event of agent.prompt("Think through this.").stream()) { if (event.type === "reasoning_delta") { if (event.contentType === undefined || event.contentType === "summary") { debugPanel.append(event.delta); } } } ``` Keep user-facing UI focused on `text_delta` unless your product intentionally exposes reasoning content. # Streaming Events (/docs/guides/streaming/streaming-events) Use `.stream()` when your UI, worker, or API route needs incremental updates instead of waiting for the final response. ## 1. Start a Stream [#1-start-a-stream] ```ts for await (const event of agent.prompt("Write a short reply.").stream()) { if (event.type === "text_delta") { process.stdout.write(event.delta); } } ``` The agent's completion model must support streaming. If the model does not support streaming, Anvia throws before the run starts. ## 2. Handle the Core Events [#2-handle-the-core-events] ```ts for await (const event of agent.prompt("Where is order A-100?").stream()) { switch (event.type) { case "text_delta": process.stdout.write(event.delta); break; case "tool_call": console.log("tool_call", event.toolCall.function.name); break; case "tool_result": console.log("tool_result", event.toolName, event.result); break; case "agent_tool_event": console.log("child agent", event.agentId, event.event.type); break; case "final": console.log("done", event.output); break; } } ``` ## 3. Know the Event Order [#3-know-the-event-order] Most text-only runs look like this: ```txt turn_start text_delta text_delta turn_end final ``` Tool runs can span multiple turns: ```txt turn_start tool_call turn_end tool_result turn_start text_delta turn_end final ``` Streaming agent-tools add nested child events between the parent `turn_end` and final parent `tool_result`: ```txt turn_start tool_call turn_end agent_tool_event turn_start agent_tool_event text_delta agent_tool_event tool_call agent_tool_event tool_result agent_tool_event final tool_result turn_start text_delta turn_end final ``` ## 4. Event Types [#4-event-types] | Event | Meaning | | ------------------ | ------------------------------------------------------------------------------- | | `turn_start` | A model turn is starting | | `text_delta` | Assistant text arrived | | `reasoning_delta` | Reasoning text or summary arrived from a provider that exposes it | | `tool_call` | The model requested a tool | | `tool_result` | Anvia ran a tool and produced a result | | `agent_tool_event` | A child agent exposed through `asTool({ stream: true })` emitted a stream event | | `turn_end` | A model turn ended | | `final` | The agent run completed | | `error` | The stream failed | The `final` event contains the same important data as `.send()`: `output`, `usage`, `messages`, and optional `trace`. `reasoning_delta` may include `contentType` and `signature` metadata. Render summaries or your own internal debug UI deliberately; encrypted and redacted blocks are opaque provider state for history continuity. `agent_tool_event` wraps the child event with `toolName`, `internalCallId`, optional provider `toolCallId`, and the child `agentId`/`agentName`. Use those fields to group nested progress in UIs. The parent model still receives only the final child output as the normal `tool_result`. The `final` event includes `runId`. Use it with [Event Store](/docs/guides/agents/event-store) when you need to load the persisted event log after the stream ends. # Agent Output (/docs/guides/structured-output/agent-output) Use agent output schemas when the agent's final answer should follow a JSON-shaped contract. ## 1. Define the Response Shape [#1-define-the-response-shape] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; const model = new OpenAIClient({ apiKey }).completionModel("gpt-5.5"); const classificationSchema = z.object({ category: z.enum(["billing", "technical", "account"]), confidence: z.number(), }); ``` ## 2. Attach It to the Agent [#2-attach-it-to-the-agent] ```ts const agent = new AgentBuilder("classifier", model) .instructions("Classify support messages.") .outputSchema(classificationSchema) .build(); ``` ## 3. Send the Prompt [#3-send-the-prompt] ```ts const response = await agent .prompt("I cannot update my payment method.") .send(); ``` `response.output` is the final assistant text. When your application needs typed data, parse and validate it at the boundary. ```ts const data = classificationSchema.parse(JSON.parse(response.output)); console.log(data.category); ``` ## Keep the Prompt Direct [#keep-the-prompt-direct] Structured output works best when the instruction names the expected object. ```ts const agent = new AgentBuilder("lead-qualifier", model) .instructions("Return only the lead qualification object.") .outputSchema( z.object({ qualified: z.boolean(), companySize: z.string().optional(), reason: z.string(), }), ) .build(); ``` Avoid asking for both a long explanation and a strict object in the same agent. If you need both, include fields for both. ## With Tools [#with-tools] Structured output can be combined with tools. The agent may call tools first, then return the final schema-shaped answer. ```ts const agent = new AgentBuilder("order-status", model) .instructions("Look up the order, then return the status object.") .tool(lookupOrder) .defaultMaxTurns(3) .outputSchema( z.object({ orderId: z.string(), status: z.string(), customerMessage: z.string(), }), ) .build(); ``` ## When to Use Agent Output [#when-to-use-agent-output] Use this when the agent is already doing the reasoning and should return a structured final answer. Use an [Extractor](/docs/guides/structured-output/extractors) when the job is only to convert existing text into data. # Extractors (/docs/guides/structured-output/extractors) Extractors are the easiest path when you have text and want validated data back. Internally, Anvia builds an agent with a required `submit` tool from your schema. ## 1. Create the Schema [#1-create-the-schema] ```ts import { ExtractorBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { z } from "zod"; const model = new OpenAIClient({ apiKey }).completionModel("gpt-5.5"); const ticketSchema = z.object({ customer: z.string(), priority: z.enum(["low", "medium", "high"]), summary: z.string(), }); ``` ## 2. Build the Extractor [#2-build-the-extractor] ```ts const extractor = new ExtractorBuilder(model, ticketSchema) .instructions("Extract support ticket fields.") .retries(1) .build(); ``` Use `retries(...)` when invalid or missing structured data should be retried before failing. ## 3. Extract Data [#3-extract-data] ```ts const ticket = await extractor.extract(` Acme Co. reports checkout failures for all enterprise users. The issue is urgent and blocking revenue. `); console.log(ticket.priority); ``` ## 4. Keep Usage and Messages When Needed [#4-keep-usage-and-messages-when-needed] ```ts const result = await extractor.extractWithUsage("Ada reports a billing issue."); console.log(result.data); console.log(result.usage.totalTokens); console.log(result.messages); ``` Use `extractWithHistory(...)` when extraction should consider previous messages. Next, read [Output Validation](/docs/guides/structured-output/output-validation) to understand where validation happens. # Failure Handling (/docs/guides/structured-output/failure-handling) Plan for structured output failures. Models can omit fields, produce invalid values, or return text when you expected data. ## 1. Retry Extraction Failures [#1-retry-extraction-failures] ```ts const extractor = new ExtractorBuilder(model, ticketSchema) .instructions("Return only fields present in the ticket.") .retries(2) .build(); ``` Use retries for extraction because the model has one focused job: call the generated `submit` tool with valid data. ## 2. Catch Validation Errors [#2-catch-validation-errors] ```ts try { const ticket = await extractor.extract(rawTicket); return ticket; } catch (error) { logger.warn(error, "ticket extraction failed"); return null; } ``` Keep the fallback explicit. Do not silently coerce invalid model output into valid application data. ## 3. Handle Agent Output Parsing [#3-handle-agent-output-parsing] ```ts const response = await agent.prompt("Classify this message.").send(); try { return classificationSchema.parse(JSON.parse(response.output)); } catch (error) { logger.warn({ output: response.output, error }, "invalid classifier output"); throw error; } ``` If the result drives user-visible or financial behavior, fail closed and ask for another attempt. ## 4. Make Optional Data Optional [#4-make-optional-data-optional] Do not require fields that might not exist in the source text. ```ts const invoiceSchema = z.object({ invoiceId: z.string(), dueDate: z.string().optional(), amountDue: z.number().optional(), }); ``` Use required fields for data your workflow cannot continue without. # Output Validation (/docs/guides/structured-output/output-validation) Structured output should become trusted application data only after validation. Anvia gives you two validation points: provider-side schema guidance and local Zod parsing. ## 1. Validate Extractor Results [#1-validate-extractor-results] Extractors validate submitted data before returning it. ```ts const ticket = await extractor.extract("Urgent checkout failure from Acme."); // ticket already passed the extractor schema. console.log(ticket.priority); ``` If the model submits invalid data, the extractor can retry when configured with `retries(...)`. ## 2. Validate Agent Output Locally [#2-validate-agent-output-locally] Agent output schemas constrain the provider request, but your app should still parse the final text. ```ts const response = await agent.prompt("Classify this billing issue.").send(); const data = classificationSchema.parse(JSON.parse(response.output)); ``` Keep parsing near the boundary where text becomes application data. ## 3. Validate Tool Results [#3-validate-tool-results] Tool output schemas validate values before they are sent back to the model. ```ts const lookupOrder = createTool({ name: "lookup_order", description: "Look up an order by id.", input: z.object({ orderId: z.string() }), output: z.object({ orderId: z.string(), status: z.string(), }), async execute({ orderId }) { return orders.find(orderId); }, }); ``` ## 4. Decide the Failure Policy [#4-decide-the-failure-policy] | Workflow | Common Policy | | ------------ | ------------------------------------------------ | | Extractors | retry once or twice, then fail the request | | Agent output | parse once, then ask the user or retry the agent | | Tool output | throw and let the agent receive the tool error | Next, read [Failure Handling](/docs/guides/structured-output/failure-handling) for concrete recovery patterns. # Schemas (/docs/guides/structured-output/schemas) Start every structured output workflow by writing the shape your application needs. Anvia uses Zod schemas for tool inputs, tool outputs, extractors, and agent output schemas. ## 1. Define the Shape [#1-define-the-shape] ```ts import { z } from "zod"; export const ticketSchema = z.object({ customer: z.string(), priority: z.enum(["low", "medium", "high"]), summary: z.string(), }); ``` Keep schemas small at first. Add nested fields only when your application actually needs them. ## 2. Choose Where the Schema Runs [#2-choose-where-the-schema-runs] | Workflow | Use | | -------------------------------------------------- | -------------------------------- | | The agent should produce a structured final answer | `AgentBuilder.outputSchema(...)` | | You have existing text and need typed data | `ExtractorBuilder` | | A tool receives model arguments | `createTool({ input })` | | A tool result must be validated | `createTool({ output })` | ## 3. Prefer Explicit Values [#3-prefer-explicit-values] Enums, required strings, booleans, and numbers are easier for models to satisfy than loose objects. ```ts const classificationSchema = z.object({ category: z.enum(["billing", "technical", "account"]), confidence: z.number(), needsFollowup: z.boolean(), }); ``` ## 4. Add Descriptions Where They Matter [#4-add-descriptions-where-they-matter] Descriptions are sent to the provider as schema metadata. Use them when a field could be interpreted in more than one way. ```ts const escalationSchema = z.object({ reason: z.string().describe("Short explanation for why escalation is needed."), severity: z.enum(["normal", "urgent"]), }); ``` Next, read [Zod Schema](/docs/guides/structured-output/zod-schema) to see how Anvia converts this shape for providers. # Zod Schema (/docs/guides/structured-output/zod-schema) Anvia accepts Zod schemas and converts them into provider-facing JSON Schema. Use Zod as the source of truth in your application code. ## 1. Create a Zod Schema [#1-create-a-zod-schema] ```ts import { z } from "zod"; const summarySchema = z .object({ title: z.string(), bullets: z.array(z.string()), }) .meta({ title: "summary_response" }); ``` The `.meta({ title })` value is preserved in the JSON Schema sent to the model provider. ## 2. Use It With an Agent [#2-use-it-with-an-agent] ```ts const agent = new AgentBuilder("summarizer", model) .instructions("Return a concise structured summary.") .outputSchema(summarySchema) .build(); ``` `outputSchema(...)` constrains the model response at the provider request layer. The prompt response still contains `response.output` as text, so parse or validate it before using it as application data. ## 3. Use It With an Extractor [#3-use-it-with-an-extractor] ```ts const extractor = new ExtractorBuilder(model, summarySchema) .retries(1) .build(); ``` Extractors validate the submitted data against the schema and return typed data from `extract(...)`. ## 4. Keep Provider Limits in Mind [#4-keep-provider-limits-in-mind] Use schema features that map cleanly to JSON Schema: | Prefer | Be Careful With | | ----------------------------------------------- | --------------------------------------- | | `z.object`, `z.string`, `z.number`, `z.boolean` | deeply recursive schemas | | `z.enum([...])` | broad `z.any()` fields | | `z.array(...)` | complex unions | | `.describe(...)` | descriptions that repeat the field name | Next, use the schema with [Agent Output](/docs/guides/structured-output/agent-output) or [Extractors](/docs/guides/structured-output/extractors). # Agents and Retrieval (/docs/guides/testing/agents-and-retrieval) Agent tests should focus on the application boundary around the agent. Retrieval tests should focus on indexing and filtering behavior before the model sees context. ## Test Agent Wrappers [#test-agent-wrappers] Put product policy around agent calls in a small wrapper. Then test that wrapper owns history, trace metadata, usage records, and error handling: ```ts import { Message, type Agent, type Message as MessageType } from "@anvia/core"; export async function runSupportAgent( agent: Agent, conversationId: string, input: string, history: MessageType[], ) { const response = await agent .prompt([...history, Message.user(input)]) .withTrace({ name: "support-agent" }) .maxTurns(3) .send(); await conversations.saveMessages(conversationId, [ ...history, ...response.messages, ]); return response.output; } ``` In tests, use a controlled model implementation or a narrow integration test against a provider. Keep broad provider calls out of fast unit tests. ## Test Known Runtime Errors [#test-known-runtime-errors] Application wrappers should classify known SDK errors where the product can make a decision: | Error | What to check | | ---------------------- | ----------------------------------------------------------------------------------- | | `MaxTurnsError` | the wrapper records the failure and returns or throws the intended product response | | `PromptCancelledError` | the wrapper treats cancellation like an interrupted or denied workflow | | provider errors | the wrapper logs request metadata and preserves the original error | | tool errors | expected product states are returned; unexpected failures are surfaced | ## Test Retrieval Boundaries [#test-retrieval-boundaries] Retrieval tests should cover: | Boundary | What to check | | ----------------------- | ------------------------------------------------- | | Embedding preprocessing | IDs, metadata, chunking, and content selectors | | Filters | Tenant, user, document status, and product scopes | | Search output | Result order, score thresholds, and empty states | | Dynamic context | Formatting and prompt budget behavior | Security-sensitive filters belong in application code. Test them without a model. ## Test Dynamic Context Formatting [#test-dynamic-context-formatting] When retrieval is attached as dynamic context, test the formatter separately: ```ts const formatted = formatSupportDoc({ title: "Refund policy", body: "Refunds are available within 30 days.", }); expect(formatted).toContain("Refund policy"); expect(formatted).toContain("30 days"); ``` The model should receive only the context shape your product intends to expose. # Evals (/docs/guides/testing/evals) Use evals when you want a repeatable signal for agent or workflow behavior. The eval runner lives in `@anvia/core`; integrations such as Langfuse can report scores without becoming part of the core runtime. ## Run a Suite [#run-a-suite] An eval suite has cases, a target, metrics, and optional reporters. ```ts import { contains, exactMatch, runEvalSuite } from "@anvia/core/evals"; const result = await runEvalSuite({ name: "support-answer-quality", cases: [ { id: "refund-window", input: "When can I request a refund?", expected: "Refunds are available for 30 days.", }, ], target: async (input) => answerSupportQuestion(input), metrics: [ contains({ expected: "30 days" }), exactMatch({ name: "not_blank", actual: ({ output }) => output.trim().length > 0, expected: true, }), ], }); console.log(result.passed, result.failed, result.invalid); ``` `runEvalSuite(...)` returns ordered case results. If the target throws, each metric for that case is marked invalid. ## Evaluate Agents [#evaluate-agents] Wrap an agent with `agentEvalTarget(...)` when the target should call `agent.prompt(...).send()`. ```ts import { agentEvalTarget, contains, runEvalSuite } from "@anvia/core/evals"; const result = await runEvalSuite({ name: "support-agent-regression", cases: [{ id: "refund-window", input: "How long are refunds available?", expected: "30 days" }], target: agentEvalTarget(agent), metrics: [contains()], }); ``` By default, string metrics evaluate the prompt response output. You can pass selectors when your target returns a custom object. ## Built-in Metrics [#built-in-metrics] | Metric | Use it for | | ------------------------- | ------------------------------------------------------ | | `exactMatch(...)` | deterministic strings, booleans, or JSON-shaped values | | `contains(...)` | required phrases or regular expressions | | `semanticSimilarity(...)` | embedding-based answer similarity | | `llmJudge(...)` | schema-shaped LLM judgments with a pass predicate | | `llmScore(...)` | 0 to 1 LLM scoring with feedback and a threshold | ## Report to Langfuse [#report-to-langfuse] Use `createLangfuseEvalReporter(...)` from `@anvia/langfuse` when you want metric outcomes sent as Langfuse scores. ```ts import { createLangfuseEvalReporter, langfuse } from "@anvia/langfuse"; const tracing = langfuse.create({ publicKey, secretKey, baseUrl }); await runEvalSuite({ name: "support-agent-regression", cases, target: agentEvalTarget(agent), metrics: [contains()], reporters: [createLangfuseEvalReporter(tracing)], }); ``` The reporter reads trace IDs from a prompt response trace or from case metadata: ```ts { id: "refund-window", input: "How long are refunds available?", expected: "30 days", metadata: { traceId: "trace-id", observationId: "observation-id", }, } ``` Invalid outcomes are skipped by default. Pass `{ publishInvalid: true }` if invalid outcomes should publish as score `0`. For runnable examples, see `examples/cookbook/08_evals` for deterministic, semantic, custom, agent, and LLM judge/score evals. Langfuse reporting remains in `examples/cookbook/10_integrations/04-langfuse-eval-reporting.ts`. # Testing (/docs/guides/testing) Anvia workflows are easiest to test when product code keeps clear boundaries: tools own side effects, agents own model behavior, pipelines own workflow shape, and the application wrapper owns persistence, logging, and error handling. Use focused tests at each boundary before adding broad provider integration tests. ## Testing Map [#testing-map] | Boundary | What to test | Page | | -------------- | ---------------------------------------------------------------------------------- | ----------------------------------------------------------------- | | Tools | input validation, expected product states, permission failures, output shape | [Tools and Pipelines](/docs/guides/testing/tools-and-pipelines) | | Pipelines | deterministic step output, composition, batch behavior, concurrency limits | [Tools and Pipelines](/docs/guides/testing/tools-and-pipelines) | | Agent wrappers | history persistence, trace metadata, usage records, runtime limits, known errors | [Agents and Retrieval](/docs/guides/testing/agents-and-retrieval) | | Retrieval | embedding selectors, metadata filters, result ordering, dynamic context formatting | [Agents and Retrieval](/docs/guides/testing/agents-and-retrieval) | | Evals | repeatable quality checks for functions, agents, and traced workflows | [Evals](/docs/guides/testing/evals) | | Studio | registration, HTTP run shape, sessions, traces, approvals, questions | [Studio and Providers](/docs/guides/testing/studio-and-providers) | | Providers | one narrow happy path per provider and the behavior your workflow depends on | [Studio and Providers](/docs/guides/testing/studio-and-providers) | ## Practical Rule [#practical-rule] Most application correctness should be covered before a provider call is made. Use provider tests to verify integration behavior, not every branch of product policy. ## Start Here [#start-here] 1. Test tools and deterministic pipeline steps first. 2. Test application wrappers around agents. 3. Test retrieval filters without a model. 4. Use Studio `.fetch(...)` for local runtime wiring. 5. Keep provider integration tests narrow and explicit. # Studio and Providers (/docs/guides/testing/studio-and-providers) Studio and provider tests are useful integration checks. Keep them narrow so they diagnose wiring and provider behavior without carrying all product correctness. ## Test Studio Without Opening a Port [#test-studio-without-opening-a-port] Studio exposes `.fetch(...)` for tests and internal tooling: ```ts const studio = new Studio([agent]); const response = await studio.fetch( new Request("http://studio.test/agents/support/runs", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ message: "Hello" }), }), ); expect(response.status).toBe(200); ``` Use this to verify agent registration, run request shape, sessions, traces, approval routes, and question routes. ## Test Sessions and Traces [#test-sessions-and-traces] When a workflow depends on Studio persistence, check the store-facing behavior: | Area | What to check | | --------- | -------------------------------------------------------------- | | Sessions | create, append, list, get, and delete behavior | | Traces | run status, observations, usage, metadata, and session linkage | | Approvals | pending approval creation and approved or rejected decisions | | Questions | pending question creation and answered question results | Keep product auth and external notification policy outside Studio tests unless your application wrapper adds those behaviors. ## Keep Provider Tests Focused [#keep-provider-tests-focused] Provider integration tests should cover the behavior your workflow depends on: 1. Test one happy path for each provider you support. 2. Test tools and structured output where provider behavior matters. 3. Test attachments only for provider models that support them. 4. Record usage and trace ids so failures are diagnosable. 5. Keep API keys in your normal secret system. ## Avoid Broad Provider Suites [#avoid-broad-provider-suites] Do not use provider calls to test every branch of tool code, retrieval filtering, or application policy. Those branches should already be covered with direct tool, pipeline, wrapper, and retrieval tests. Provider tests should answer a smaller question: does this model adapter behave the way this workflow requires? # Tools and Pipelines (/docs/guides/testing/tools-and-pipelines) Tools and pipelines usually contain the highest-value unit tests because they hold application behavior that should work before a model is involved. ## Test Tools Directly [#test-tools-directly] Tools are application functions with schemas. Test expected product states without involving a model: ```ts const result = await lookupOrder.call({ orderId: "A-100" }); expect(result).toEqual({ status: "shipped", carrier: "DHL", }); ``` Also test invalid input and permission failures at the tool boundary. Do not rely on prompts to enforce product permissions. ## Test Expected Product States [#test-expected-product-states] Prefer explicit results for expected conditions: ```ts await expect(lookupOrder.call({ orderId: "missing" })).resolves.toEqual({ status: "not_found", }); ``` Throw only when the workflow should fail or be retried. That keeps tool behavior easier to assert from agents, pipelines, and Studio. ## Test Pipelines as Workflow Code [#test-pipelines-as-workflow-code] Pipeline steps are ordinary functions. Keep deterministic steps small enough to test before adding model calls: ```ts const pipeline = new PipelineBuilder() .step((input) => input.trim()) .step((input) => ({ query: input, limit: 5 })) .build(); await expect(pipeline.run(" refund policy ")).resolves.toEqual({ query: "refund policy", limit: 5, }); ``` ## Test Batch Behavior [#test-batch-behavior] Use `batch(...)` tests when concurrency, ordering, or failure behavior matters: ```ts const results = await pipeline.batch(["one", "two"], { concurrency: 2, }); expect(results).toHaveLength(2); ``` Keep model-backed pipeline stages in narrower integration tests. Deterministic preprocessing, branching, and result formatting should stay covered by fast tests. # Creating Tools (/docs/guides/tools/creating-tools) Tools expose application-owned behavior to an agent. The model chooses when to call the tool, but your code owns the implementation. ## Tool APIs [#tool-apis] Use `createTool()` to define one tool. Use `ToolSet` to group reusable tools for sharing, direct calls, tests, and mutable runtime tool stores. ## Minimal Tool [#minimal-tool] ```ts import { createTool } from "@anvia/core"; import { z } from "zod"; export const lookupOrder = createTool({ name: "lookup_order", description: "Look up an order by order id.", input: z.object({ orderId: z.string().describe("The order id to inspect."), }), output: z.object({ orderId: z.string(), status: z.string(), }), async execute({ orderId }) { return { orderId, status: "shipped", }; }, }); ``` `input` validates arguments from the model before your handler runs. `output` validates the value returned by your handler before Anvia sends it back to the model. ## Add the Tool to an Agent [#add-the-tool-to-an-agent] ```ts const agent = new AgentBuilder("support", model) .instructions("Use tools when order status is needed.") .tool(lookupOrder) .defaultMaxTurns(3) .build(); ``` The turn limit gives the agent room to call the tool and then ask the model for the final answer. ## Naming [#naming] Use names that describe the action. | Good | Avoid | | --------------- | ------------- | | `lookup_order` | `order` | | `create_refund` | `refund_tool` | | `search_docs` | `search` | Descriptions should explain when to use the tool, not how the tool is implemented. ## Read-Only vs Side-Effect Tools [#read-only-vs-side-effect-tools] Read-only tools can usually run automatically. ```ts const searchDocs = createTool({ name: "search_docs", description: "Search public documentation.", input: z.object({ query: z.string() }), async execute({ query }) { return docs.search(query); }, }); ``` Side-effect tools should usually be paired with [Human in the Loop](/docs/guides/human-in-the-loop) approval. You can attach passive approval metadata for runtimes such as Studio; core stores this metadata and still leaves execution control to hooks or the runtime. ```ts const refundOrder = createTool({ name: "refund_order", description: "Refund an eligible order.", input: z.object({ orderId: z.string(), amount: z.number().positive(), }), output: z.object({ refunded: z.boolean() }), approval: { when: ({ args }) => args.amount > 100, reason: ({ args }) => `Review refund of $${args.amount} for ${args.orderId}.`, rejectMessage: "Refund was not approved.", }, async execute({ orderId, amount }) { return refunds.create(orderId, amount); }, }); ``` Keep the tool implementation explicit. Do not rely on the model prompt as the permission boundary. # Think Tool (/docs/guides/tools/think-tool) `createThinkTool(...)` creates a small tool that records a thought and returns it. It does not read data, store memory, or change external state. Use it when you want the model to create an explicit checkpoint during a complex tool workflow. ## Add the Default Think Tool [#add-the-default-think-tool] ```ts import { AgentBuilder, createThinkTool } from "@anvia/core"; const agent = new AgentBuilder("support", model) .instructions("Use the think tool before taking multi-step actions.") .tool(createThinkTool()) .tool(lookupOrder) .tool(createRefund) .defaultMaxTurns(5) .build(); ``` The default tool is named `think`. ## Override Name or Description [#override-name-or-description] ```ts const planTool = createThinkTool({ name: "plan", description: "Record a short plan before using another tool.", }); ``` Use a custom name when `think` conflicts with another tool or when you want a more domain-specific instruction. ## Behavior [#behavior] The tool accepts one field: ```ts { thought: "Check the order status before creating a refund." } ``` It returns the same string as the tool result. That result becomes part of the agent's tool-call history. ## When Not To Use It [#when-not-to-use-it] Do not add a think tool to every agent by default. Prefer it when: * the workflow has several tool calls * you want an explicit planning checkpoint * the provider does not already expose useful reasoning events * you can tolerate another tool turn # Tool Errors (/docs/guides/tools/tool-errors) Anvia distinguishes tool lookup, JSON parsing, and tool execution failures. ## Error Types [#error-types] | Error | When it happens | | ------------------- | --------------------------------------------------------------- | | `ToolNotFoundError` | A requested tool name is not registered | | `ToolJsonError` | Tool arguments are not valid JSON | | `ToolCallError` | Input validation, handler execution, or output validation fails | ## Direct Tool Set Calls [#direct-tool-set-calls] When you call a `ToolSet` directly, catch tool errors at the call boundary. ```ts import { ToolCallError, ToolJsonError, ToolNotFoundError } from "@anvia/core"; try { const result = await toolSet.call("lookup_order", rawArgs); return result; } catch (error) { if (error instanceof ToolJsonError) { return "Invalid tool arguments."; } if (error instanceof ToolNotFoundError) { return `Unknown tool: ${error.toolName}`; } if (error instanceof ToolCallError) { logger.warn(error, "tool call failed"); return "Tool failed."; } throw error; } ``` ## Agent Runs [#agent-runs] During agent runs, Anvia catches tool execution errors and sends the error string back as the tool result. The model can then recover, ask for clarification, or explain that the action failed. Use clear expected return values for product states the model can handle. ```ts return { found: false, reason: "Order does not exist.", }; ``` Throw for unexpected failures. ```ts throw new Error("Order service unavailable."); ``` ## Validation Errors [#validation-errors] Input and output validation failures are wrapped as `ToolCallError`. ```ts const updatePlan = createTool({ name: "update_plan", description: "Update a customer's plan.", input: z.object({ customerId: z.string(), plan: z.enum(["free", "pro", "enterprise"]), }), output: z.object({ updated: z.boolean(), }), async execute(input) { return billing.updatePlan(input); }, }); ``` Use schema constraints to catch bad model arguments before side effects happen. # Tool Handlers (/docs/guides/tools/tool-handlers) The `execute(...)` function is the tool handler. It receives parsed input and may return a value or a promise. ## Handler Signature [#handler-signature] ```ts const lookupOrder = createTool({ name: "lookup_order", description: "Look up an order by order id.", input: z.object({ orderId: z.string() }), output: z.object({ status: z.string() }), async execute({ orderId }) { return orders.findById(orderId); }, }); ``` The handler receives `z.output`. The model never calls this function directly; Anvia validates and parses the model arguments first. ## Capture App Dependencies [#capture-app-dependencies] Pass services into a factory when the tool needs application dependencies. ```ts export function createOrderTools(services: { orders: OrderRepository; logger: Logger; }) { return [ createTool({ name: "lookup_order", description: "Look up an order by order id.", input: z.object({ orderId: z.string() }), async execute({ orderId }) { services.logger.info({ orderId }, "lookup order"); return services.orders.findById(orderId); }, }), ]; } ``` This keeps auth, tenancy, logging, and database access in your application code. ## Return Product States [#return-product-states] Expected states should be normal return values. ```ts const lookupOrder = createTool({ name: "lookup_order", description: "Look up an order by order id.", input: z.object({ orderId: z.string() }), output: z.object({ found: z.boolean(), status: z.string().optional(), }), async execute({ orderId }) { const order = await orders.findById(orderId); return order === undefined ? { found: false } : { found: true, status: order.status }; }, }); ``` Throw only when the workflow should be treated as failed or retried. ## Side Effects [#side-effects] Side-effect tools should be explicit and narrow. ```ts const createRefund = createTool({ name: "create_refund", description: "Create a refund for an eligible order.", input: z.object({ orderId: z.string() }), output: z.object({ refundId: z.string() }), async execute({ orderId }) { return refunds.create({ orderId }); }, }); ``` For user approval before execution, use [Human in the Loop](/docs/guides/human-in-the-loop). # Tool Middleware (/docs/guides/tools/tool-middleware) Tool result middleware runs after a tool produces its serialized string result and before that result is sent back to the model. Use it for output gates, redaction, compression, or file references when a tool result is too large. ## Create Middleware [#create-middleware] ```ts import { createToolMiddleware } from "@anvia/core"; const outputGate = createToolMiddleware({ async onResult({ toolName, result, internalCallId }) { if (result.length <= 1_000) { return undefined; } const path = await files.write({ name: `${toolName}-${internalCallId}.txt`, content: result, }); return JSON.stringify({ type: "file_reference", reason: "tool_output_too_large", chars: result.length, path, }); }, }); ``` Return a string to replace the current tool result. Return `undefined` to keep the current result. ## Register On An Agent [#register-on-an-agent] ```ts const agent = new AgentBuilder("support", model) .tools([lookupOrder, exportReport]) .toolMiddleware(outputGate) .build(); ``` Middleware applies to tool results from local tools, MCP tools, dynamic tools, vector search tools, and agents exposed with `agent.asTool(...)`. ## Register For One Request [#register-for-one-request] ```ts const response = await agent .prompt("Summarize the large report.") .withToolMiddleware(outputGate) .send(); ``` Use request middleware when one caller needs stricter output policy than the agent default. ## Compose Middleware [#compose-middleware] ```ts const agent = new AgentBuilder("support", model) .toolMiddlewares([redactSecrets, outputGate]) .build(); ``` Middleware runs in registration order. Agent middleware runs before request middleware. Each middleware receives the latest `result` and the unchanged `originalResult`. ## Skill Tools [#skill-tools] Skill runtime tools are excluded from tool result middleware. Their behavior is owned by the skills runtime, including loading instructions, reading references, and running skill scripts. # Tool Results (/docs/guides/tools/tool-results) Tool results are serialized before they are sent back to the model. Use [tool middleware](/docs/guides/tools/tool-middleware) when an agent should transform serialized tool results, such as replacing large outputs with file references. ## String Results [#string-results] Strings are returned as-is. ```ts const echo = createTool({ name: "echo", description: "Echo a message.", input: z.object({ value: z.string() }), output: z.string(), execute: ({ value }) => value, }); ``` The model receives the plain string. ## Object Results [#object-results] Objects are JSON serialized. ```ts const lookupOrder = createTool({ name: "lookup_order", description: "Look up an order.", input: z.object({ orderId: z.string() }), output: z.object({ orderId: z.string(), status: z.string(), }), execute: ({ orderId }) => ({ orderId, status: "shipped", }), }); ``` The model receives: ```json {"orderId":"A-100","status":"shipped"} ``` ## Result Messages [#result-messages] During an agent run, tool results are appended as tool messages. ```ts { role: "tool", content: [ { type: "tool_result", id: "call_1", content: [{ type: "text", text: "{\"status\":\"shipped\"}" }] } ] } ``` You usually do not construct this message yourself. Store `response.messages` if you need conversation history. ## Display Values [#display-values] Return values should be useful to the model, not necessarily formatted for your UI. ```ts return { status: "shipped", carrier: "DHL", trackingNumber: "123", }; ``` Format UI-specific text outside the tool when possible. That keeps the tool reusable across agents, pipelines, and tests. # Tool Schemas (/docs/guides/tools/tool-schemas) Anvia tools use Zod schemas for runtime validation and provider tool definitions. ## Input Schema [#input-schema] ```ts const lookupCustomer = createTool({ name: "lookup_customer", description: "Look up a customer by email.", input: z.object({ email: z.string().email().describe("The customer email address."), }), async execute({ email }) { return customers.findByEmail(email); }, }); ``` The input schema is converted into JSON Schema and advertised to the model. ## Output Schema [#output-schema] ```ts const lookupCustomer = createTool({ name: "lookup_customer", description: "Look up a customer by email.", input: z.object({ email: z.string().email(), }), output: z.object({ id: z.string(), plan: z.enum(["free", "pro", "enterprise"]), }), async execute({ email }) { return customers.findByEmail(email); }, }); ``` If `output` is provided, Anvia validates the handler result before serializing it for the model. ## Schema Metadata [#schema-metadata] Use `.describe(...)` to make tool arguments clearer to the model. ```ts const createTicket = createTool({ name: "create_ticket", description: "Create a support ticket.", input: z.object({ title: z.string().describe("Short issue summary."), priority: z.enum(["low", "medium", "high"]).describe("Customer-visible priority."), }), output: z.object({ ticketId: z.string(), }), async execute(input) { return tickets.create(input); }, }); ``` Prefer small schemas. If a tool needs many fields, consider splitting the workflow into smaller tools or using a structured output step before the tool call. ## Validation Failures [#validation-failures] Invalid input or output becomes a tool call error. ```ts try { await toolSet.call("lookup_customer", JSON.stringify({ email: "not-an-email" })); } catch (error) { if (error instanceof ToolCallError) { logger.warn(error, "tool validation failed"); } } ``` In normal agent runs, Anvia catches tool execution errors and sends the error string back as the tool result so the model can recover or explain the failure. # Tool Sets (/docs/guides/tools/tool-sets) Use `ToolSet` when you want to group tools, inspect definitions, call tools directly, or share a collection across agents. ## Create a Tool Set [#create-a-tool-set] ```ts import { ToolSet } from "@anvia/core"; const supportTools = ToolSet.fromTools([ lookupOrder, searchDocs, createTicket, ]); ``` ## Register on an Agent [#register-on-an-agent] Use `.useToolSet(...)` when the agent should read from a shared mutable tool set. ```ts const agent = new AgentBuilder("support", model) .useToolSet(supportTools) .defaultMaxTurns(3) .build(); ``` Later updates to the same `ToolSet` are visible to future prompt runs. ```ts supportTools.addTool(refundOrder); ``` ## Call a Tool Directly [#call-a-tool-directly] This is useful for tests and internal tooling. ```ts const result = await supportTools.call( "lookup_order", JSON.stringify({ orderId: "A-100" }), ); ``` `ToolSet.call(...)` parses the JSON string, validates input, runs the tool, validates output, and returns the serialized result. ## Combine Tool Sets [#combine-tool-sets] ```ts const supportTools = ToolSet.fromTools([lookupOrder]); const billingTools = ToolSet.fromTools([lookupInvoice]); supportTools.addTools(billingTools); ``` When two tools have the same name, the later tool replaces the earlier one. ## Inspect Definitions [#inspect-definitions] ```ts const definitions = await supportTools.getToolDefinitions(); ``` Definitions are the provider-facing tool schemas sent to completion models. ## Select Tools Dynamically [#select-tools-dynamically] Use dynamic tools when a shared tool catalog is too large to send every turn. ```ts const toolIndex = await createToolIndex(embeddings, supportTools, { metadata: (tool) => ({ name: tool.name }), }); const agent = new AgentBuilder("support", model) .dynamicTools(toolIndex, { topK: 6, threshold: 0.7, }) .build(); ``` On each turn, Anvia searches the tool index with the prompt text and sends only matching dynamic tool definitions. Static tools registered with `.tools(...)`, `.tool(...)`, `.mcp(...)`, `.skills(...)`, or `.useToolSet(...)` are still sent every turn. # Cloudflare AI Gateway (/docs/models/compatible-gateways/cloudflare-ai-gateway) Cloudflare AI Gateway provides an OpenAI-compatible chat completions endpoint under `https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat`. ## Create the Client [#create-the-client] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const accountId = process.env.CLOUDFLARE_ACCOUNT_ID; const gatewayId = process.env.CLOUDFLARE_AI_GATEWAY_ID ?? "default"; const client = new OpenAIClient({ baseUrl: `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}/compat`, apiKey: process.env.CLOUDFLARE_AI_GATEWAY_API_KEY, }); const model = client.completionModel("anthropic/claude-sonnet-4.5"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); console.log(response.output); ``` Use Cloudflare's gateway model format, usually `provider/model`. ## Get the Model List [#get-the-model-list] If your Cloudflare AI Gateway configuration exposes the OpenAI-compatible models endpoint, `listModels()` reads it through the configured `baseUrl`. ```ts const models = await client.listModels(); console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy }))); ``` ## Notes [#notes] * Cloudflare supports provider-native routes and an OpenAI-compatible unified route. This page uses the OpenAI-compatible route. * Authentication depends on whether you use unified billing, stored keys, or request headers in Cloudflare. * Model availability and feature support depend on the selected upstream provider. For current Cloudflare AI Gateway details, see the [getting started guide](https://developers.cloudflare.com/ai-gateway/get-started/) and [OpenAI-compatible endpoint documentation](https://developers.cloudflare.com/ai-gateway/chat-completion/). # Helicone AI Gateway (/docs/models/compatible-gateways/helicone-ai-gateway) Helicone AI Gateway exposes a unified OpenAI-compatible API at `https://ai-gateway.helicone.ai`. ## Create the Client [#create-the-client] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: "https://ai-gateway.helicone.ai", apiKey: process.env.HELICONE_API_KEY, }); const model = client.completionModel("gpt-4o-mini"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); console.log(response.output); ``` Use the model ids and routing formats configured in Helicone. ## Get the Model List [#get-the-model-list] If your Helicone AI Gateway route exposes a model registry through the OpenAI-compatible models endpoint, call `listModels()`. ```ts const models = await client.listModels(); console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy }))); ``` ## Notes [#notes] * Helicone AI Gateway focuses on unified routing, fallbacks, and observability. Its gateway documentation currently describes the gateway as beta. * Model ids, routing, and fallback behavior are controlled by Helicone and the upstream provider. * Test specific model behavior before enabling tools, structured output, attachments, or multimodal features. For current Helicone details, see the [Helicone AI Gateway overview](https://docs.helicone.ai/gateway). # LangDB AI Gateway (/docs/models/compatible-gateways/langdb-ai-gateway) LangDB AI Gateway provides OpenAI-compatible APIs for routing to multiple LLM providers. The regional API base URL commonly includes your LangDB project id, such as `https://api.us-east-1.langdb.ai/{project_id}/v1`. ## Create the Client [#create-the-client] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const projectId = process.env.LANGDB_PROJECT_ID; const client = new OpenAIClient({ baseUrl: `https://api.us-east-1.langdb.ai/${projectId}/v1`, apiKey: process.env.LANGDB_API_KEY, }); const model = client.completionModel("anthropic/claude-sonnet-4"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); console.log(response.output); ``` Use the model ids supported by your LangDB project and region. ## Get the Model List [#get-the-model-list] If your LangDB project exposes model listing through its OpenAI-compatible API, `listModels()` calls the configured `/models` endpoint. ```ts const models = await client.listModels(); console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy }))); ``` ## Notes [#notes] * LangDB can also require project metadata headers depending on the API path and account setup. Keep those values in configuration and pass them with `headers` if needed. * Routing, tracing, guardrails, and model access are controlled by your LangDB project. * Confirm the region-specific base URL before deploying. For current LangDB details, see the [LangDB API guide](https://docs.langdb.ai/getting-started/working-with-api) and [AI Gateway API reference](https://docs.langdb.ai/api-reference/ai-gateway-api). # LiteLLM (/docs/models/compatible-gateways/litellm) LiteLLM can run as a proxy server that exposes OpenAI-compatible endpoints for many upstream providers. The local proxy base URL is commonly `http://localhost:4000/v1`. ## Create the Client [#create-the-client] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: process.env.LITELLM_BASE_URL ?? "http://localhost:4000/v1", apiKey: process.env.LITELLM_API_KEY ?? "local", }); const model = client.completionModel("gpt-5-mini"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); console.log(response.output); ``` The model id must match a model configured in your LiteLLM proxy. Depending on your proxy configuration, that might be a public provider model id or a local alias. ## Get the Model List [#get-the-model-list] When the LiteLLM proxy exposes `/models`, `listModels()` reads the configured model list through the same base URL. ```ts const models = await client.listModels(); console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy }))); ``` ## Notes [#notes] * Keep LiteLLM model aliases in your proxy configuration and pass those aliases to `completionModel(...)`. * Advanced features such as tools, structured output, images, and reasoning metadata depend on the upstream provider and proxy configuration. * If your proxy requires a master key or virtual key, pass it as `apiKey`. For current LiteLLM details, see the [LiteLLM documentation](https://docs.litellm.ai/). # MiniMax (/docs/models/compatible-gateways/minimax) MiniMax exposes OpenAI-compatible endpoints at `https://api.minimax.io/v1`. In Anvia, configure `OpenAIClient` with that `baseUrl`, then pass MiniMax model ids to `completionModel(...)`. ## Create the Client [#create-the-client] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: "https://api.minimax.io/v1", apiKey: process.env.MINIMAX_API_KEY, }); const model = client.completionModel("MiniMax-M2.5"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); console.log(response.output); ``` ## Get the Model List [#get-the-model-list] MiniMax provides an OpenAI-compatible model list endpoint. Because the client was created with `baseUrl`, `listModels()` calls MiniMax's `/models` endpoint. ```ts const models = await client.listModels(); console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy }))); ``` Use the `id` field directly with `completionModel(...)`. ## Notes [#notes] * MiniMax also appears in some tools as MiniMax coding plans or alternate domains. Prefer the official API base URL for production docs unless your product explicitly supports a plan-specific endpoint. * Compatible endpoints can differ in reasoning metadata, tool behavior, streaming chunks, and rate limits. Test the specific model id before enabling advanced features. For current MiniMax API details, see the [MiniMax model list reference](https://platform.minimax.io/docs/api-reference/models/openai/list-models). # Moonshot AI (/docs/models/compatible-gateways/moonshot-ai) Moonshot AI's Kimi API exposes OpenAI-compatible chat endpoints at `https://api.moonshot.ai/v1`. ## Create the Client [#create-the-client] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: "https://api.moonshot.ai/v1", apiKey: process.env.MOONSHOT_API_KEY, }); const model = client.completionModel("kimi-k2"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); console.log(response.output); ``` ## Get the Model List [#get-the-model-list] If your Moonshot account exposes the OpenAI-compatible models endpoint, `listModels()` reads available model ids from `/models`. ```ts const models = await client.listModels(); console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy }))); ``` ## Notes [#notes] * Kimi-specific options may need to be passed through provider-specific request fields. Keep those options close to the workflow that needs them. * OpenAI compatibility covers the transport shape, not identical behavior across all models. For current Moonshot API details, see the [Kimi API overview](https://platform.kimi.ai/docs/api/overview). # Novita AI (/docs/models/compatible-gateways/novita-ai) Novita AI exposes OpenAI-compatible LLM endpoints under `https://api.novita.ai/openai`. ## Create the Client [#create-the-client] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: "https://api.novita.ai/openai", apiKey: process.env.NOVITA_API_KEY, }); const model = client.completionModel("meta-llama/llama-3.1-8b-instruct"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); console.log(response.output); ``` ## Get the Model List [#get-the-model-list] Novita AI documents model listing as part of its OpenAI-compatible LLM API. Because the client was created with `baseUrl`, `listModels()` calls Novita AI's configured models endpoint. ```ts const models = await client.listModels(); console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy }))); ``` ## Notes [#notes] * Novita AI also offers image, audio, and video APIs. This page focuses only on the OpenAI-compatible LLM endpoint used by `OpenAIClient`. * Check the specific model for tool calling, structured output, streaming, and multimodal support before enabling those features. For current Novita AI API details, see the [Novita API reference](https://novita.ai/docs/api-reference) and [LLM API introduction](https://novita.ai/docs/api-reference/model-apis-introduction). # NVIDIA NIM (/docs/models/compatible-gateways/nvidia-nim) NVIDIA NIM for large language models exposes OpenAI-compatible inference endpoints. For local NIM containers, the default base URL is commonly the running NIM server, such as `http://localhost:8000/v1`. ## Create the Client [#create-the-client] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: "http://localhost:8000/v1", apiKey: process.env.NVIDIA_NIM_API_KEY ?? "local", }); const model = client.completionModel("meta/llama-3.1-8b-instruct"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); console.log(response.output); ``` For NVIDIA-hosted endpoints, use the base URL and credentials from your NVIDIA API catalog or deployment configuration. ## Get the Model List [#get-the-model-list] NIM exposes `GET /v1/models` for loaded and available inference models. Configure `OpenAIClient` with your NIM base URL, then call `listModels()`. ```ts const baseUrl = process.env.NVIDIA_NIM_BASE_URL ?? "http://localhost:8000/v1"; const client = new OpenAIClient({ baseUrl, apiKey: process.env.NVIDIA_NIM_API_KEY ?? "local", }); const models = await client.listModels(); console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy }))); ``` ## Notes [#notes] * NIM deployments can be local, private, or hosted. Keep `baseUrl` in your app configuration instead of hardcoding deployment-specific hosts. * NIM exposes additional management endpoints that are outside Anvia's normalized completion model surface. For current NVIDIA NIM API details, see the [NIM LLM API reference](https://docs.nvidia.com/nim/large-language-models/2.0.3/reference/api-reference.html). # Ollama Cloud (/docs/models/compatible-gateways/ollama-cloud) Ollama Cloud exposes hosted models through Ollama's cloud API. Treat it as a separate configuration from local Ollama because authentication, model availability, and base URLs differ. ## Create the Client [#create-the-client] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: "https://ollama.com/v1", apiKey: process.env.OLLAMA_API_KEY, }); const model = client.completionModel("gpt-oss:20b"); const agent = new AgentBuilder("cloud-support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); console.log(response.output); ``` ## Get the Model List [#get-the-model-list] If your Ollama Cloud account exposes the OpenAI-compatible models endpoint, use `listModels()`. ```ts const models = await client.listModels(); console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy }))); ``` ## Notes [#notes] * Keep local Ollama and Ollama Cloud as separate app configuration entries. * Model ids and availability can differ from your local Ollama runtime. For current Ollama Cloud details, see [Ollama Cloud](https://docs.ollama.com/cloud). # Ollama (/docs/models/compatible-gateways/ollama) Ollama provides partial OpenAI API compatibility for local models. The local OpenAI-compatible base URL is usually `http://localhost:11434/v1`. ## Create the Client [#create-the-client] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: "http://localhost:11434/v1", apiKey: "ollama", }); const model = client.completionModel("llama3.1"); const agent = new AgentBuilder("local-support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); console.log(response.output); ``` Ollama does not require an API key for local use, but the OpenAI SDK expects one. Use a placeholder value. ## Get the Model List [#get-the-model-list] Use the OpenAI-compatible models endpoint through `listModels()`: ```ts const models = await client.listModels(); console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy }))); ``` ## Notes [#notes] * Ollama compatibility covers common OpenAI-style chat flows, but not every OpenAI parameter or advanced capability. * Local model availability depends on which models are pulled into your Ollama runtime. For current Ollama API details, see [Ollama OpenAI compatibility](https://docs.ollama.com/openai). # OpenCode (/docs/models/compatible-gateways/opencode) OpenCode documents provider entries for OpenCode Go and OpenCode Zen. Treat them as OpenAI-compatible gateway-style endpoints when your OpenCode account exposes an API key and base URL. ## OpenCode Go [#opencode-go] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: "https://opencode.ai/zen/go/v1", apiKey: process.env.OPENCODE_API_KEY, }); const model = client.completionModel("opencode-go"); const agent = new AgentBuilder("coding-assistant", model) .instructions("Answer coding questions with concise, concrete steps.") .build(); const response = await agent.prompt("Explain this TypeScript error.").send(); console.log(response.output); ``` ## OpenCode Zen [#opencode-zen] ```ts const client = new OpenAIClient({ baseUrl: "https://opencode.ai/zen/v1", apiKey: process.env.OPENCODE_API_KEY, }); const model = client.completionModel("opencode-zen"); ``` ## Get the Model List [#get-the-model-list] If your OpenCode endpoint exposes an OpenAI-compatible model list, call `listModels()` on the client configured with that base URL. ```ts const client = new OpenAIClient({ baseUrl: "https://opencode.ai/zen/go/v1", apiKey: process.env.OPENCODE_API_KEY, }); const models = await client.listModels(); console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy }))); ``` ## Notes [#notes] * OpenCode Go and OpenCode Zen are OpenCode-specific provider entries. Keep them on one page unless you need separate setup flows. * Verify the exact base URL and model ids in your OpenCode account before deploying. For current OpenCode provider details, see the [OpenCode providers documentation](https://opencode.ai/docs/providers/). # OpenRouter (/docs/models/compatible-gateways/openrouter) OpenRouter exposes an OpenAI-compatible API at `https://openrouter.ai/api/v1`. In Anvia, use `OpenAIClient` with that `baseUrl`, then pass OpenRouter model ids to `completionModel(...)`. ## Create the Client [#create-the-client] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: "https://openrouter.ai/api/v1", apiKey: process.env.OPENROUTER_API_KEY, }); const model = client.completionModel("openai/gpt-5.5"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); console.log(response.output); ``` `baseUrl` makes Anvia use the OpenAI-compatible chat completion adapter. The model id is the OpenRouter id, not an Anvia-specific alias. ## Get the Model List [#get-the-model-list] OpenRouter's models API returns the model ids and metadata you can use when choosing a model. Because the client was created with `baseUrl`, `listModels()` calls OpenRouter's `/models` endpoint. ```ts const models = await client.listModels(); console.table( models.data.map((model) => ({ id: model.id, name: model.name, contextLength: model.contextLength, })), ); ``` Use the `id` field directly: ```ts const model = client.completionModel("anthropic/claude-opus-4.6"); ``` ## Filter Models [#filter-models] OpenRouter supports query parameters on the models endpoint. Use them when your UI or configuration screen only needs models with specific capabilities. `listModels()` does not expose gateway-specific filters. Use `fetch` directly when you need OpenRouter query parameters. ```ts const response = await fetch( "https://openrouter.ai/api/v1/models?supported_parameters=tools", { headers: { Authorization: `Bearer ${process.env.OPENROUTER_API_KEY}`, }, }, ); ``` You can also filter by output modality: ```ts const response = await fetch( "https://openrouter.ai/api/v1/models?output_modalities=text", { headers: { Authorization: `Bearer ${process.env.OPENROUTER_API_KEY}`, }, }, ); ``` ## Notes [#notes] * OpenRouter API keys are passed as bearer tokens. * Optional OpenRouter headers such as `HTTP-Referer` and `X-Title` are for app attribution on OpenRouter. Add them in your own HTTP layer or OpenAI SDK client when needed. * Model capabilities still vary by upstream model and route. Check OpenRouter model metadata before enabling tools, reasoning, structured output, images, or other model-specific behavior. For OpenRouter's current API details, see the [OpenRouter quickstart](https://openrouter.ai/docs/) and [models API reference](https://openrouter.ai/docs/api/api-reference/models/get-models). # Portkey (/docs/models/compatible-gateways/portkey) Portkey exposes an OpenAI-compatible gateway at `https://api.portkey.ai/v1`. In Anvia, configure `OpenAIClient` with Portkey's base URL and pass Portkey headers through `headers`. ## Create the Client [#create-the-client] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: "https://api.portkey.ai/v1", apiKey: process.env.OPENAI_API_KEY, headers: { "x-portkey-api-key": process.env.PORTKEY_API_KEY ?? "", "x-portkey-provider": "openai", }, }); const model = client.completionModel("gpt-5-mini"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); console.log(response.output); ``` For saved Portkey providers, use the provider slug header, such as `x-portkey-provider: @openai-prod`, instead of passing a raw upstream provider key. ## Get the Model List [#get-the-model-list] If your Portkey provider or config exposes model listing for the selected route, `listModels()` calls Portkey's configured `/models` endpoint. ```ts const models = await client.listModels(); console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy }))); ``` ## Notes [#notes] * Portkey supports provider headers, virtual providers, configs, retries, caching, and observability. Keep those choices in your application configuration. * The `apiKey` value is forwarded as the OpenAI SDK authorization header. For provider slugs stored in Portkey, use the provider slug headers documented by Portkey. * Model capabilities still depend on the upstream provider and Portkey route. For current Portkey details, see the [Portkey AI Gateway guide](https://portkey.ai/docs/guides/getting-started/getting-started-with-ai-gateway) and [headers reference](https://portkey.ai/docs/api-reference/inference-api/headers). # Vercel AI Gateway (/docs/models/compatible-gateways/vercel-ai-gateway) Vercel AI Gateway exposes an OpenAI-compatible API at `https://ai-gateway.vercel.sh/v1`. ## Create the Client [#create-the-client] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: "https://ai-gateway.vercel.sh/v1", apiKey: process.env.AI_GATEWAY_API_KEY, }); const model = client.completionModel("anthropic/claude-sonnet-4.6"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); console.log(response.output); ``` Use Vercel AI Gateway model ids directly, usually in `provider/model` form. ## Get the Model List [#get-the-model-list] Vercel AI Gateway supports `GET /models` on its OpenAI-compatible API. ```ts const models = await client.listModels(); console.table(models.data.map((model) => ({ id: model.id, owner: model.ownedBy }))); ``` ## Notes [#notes] * Vercel AI Gateway can route across providers and models behind one API key. * Provider options, fallbacks, attachments, and model-specific behavior still depend on the selected gateway model. * If you use Vercel OIDC instead of an API key, pass the token through your application configuration as `apiKey`. For current Vercel AI Gateway details, see the [OpenAI-compatible API documentation](https://vercel.com/docs/ai-gateway/openai-compat). # Embeddings (/docs/models/embeddings) Embedding models convert text into vectors. In Anvia, they are separate from completion models so retrieval code can swap providers without changing agent logic. ```ts import { OpenAIClient } from "@anvia/openai"; const openai = new OpenAIClient({ apiKey }); const embeddings = openai.embeddingModel("text-embedding-3-small"); const result = await embeddings.embedTexts(["Anvia carries structured context."]); ``` ## Provider Support [#provider-support] | Provider package | Embedding support | | --------------------- | ------------------------------------------------ | | `@anvia/openai` | OpenAI and OpenAI-compatible embedding endpoints | | `@anvia/gemini` | Gemini embedding models | | `@anvia/mistral` | Mistral embedding models | | `@anvia/fastembed` | Local FastEmbed embeddings | | `@anvia/transformers` | Local Transformers embeddings | Use embeddings with [Vector Stores](/docs/guides/retrieval/vector-stores) for retrieval workflows. Keep the embedding model used for indexing and searching consistent. If one workflow embeds documents with one provider and searches with another, vector dimensions and similarity behavior may not match. # Models (/docs/models) Anvia treats models as capabilities. A completion model can answer prompts and call tools. An embedding model can turn text into vectors for retrieval. Provider clients create those capabilities, then agents, extractors, pipelines, and retrieval code consume them through provider-neutral interfaces. ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const openai = new OpenAIClient({ apiKey }); const model = openai.completionModel("gpt-5.5"); const agent = new AgentBuilder("support", model) .instructions("Answer clearly and ask for missing details.") .build(); ``` ## Model Categories [#model-categories] | Category | Use it for | | -------------------- | ----------------------------------------------------------------------- | | Completion | Agents, extractors, prompt steps, streaming, and tool calls | | Embeddings | Retrieval, document search, semantic routing, and vector stores | | Model listing | Discovering provider-returned model ids and metadata | | Compatible gateways | OpenAI-compatible gateways, hosted model APIs, and local model backends | | Compatible providers | OpenAI-compatible APIs through `OpenAIClient({ baseUrl, apiKey })` | | Vertex AI | Gemini through `GeminiClient({ vertexai: true, project, location })` | ## Completion Capabilities [#completion-capabilities] Completion models expose `provider`, `defaultModel`, and `capabilities`. Anvia uses those capabilities to reject unsupported requests before making provider calls. Core capability checks cover the normalized Anvia surface: streaming, tools, tool choice, image input, document file input, output schemas, and reasoning events. They do not guarantee that every upstream model name supports every feature, and they do not validate provider-specific `additionalParams`. ## Configuration Boundary [#configuration-boundary] Provider clients only use explicit constructor values. They do not read environment variables directly. Load credentials through your application configuration layer, then pass them into the client once: ```ts const client = new OpenAIClient({ apiKey: config.openai.apiKey, baseUrl: config.openai.baseUrl, }); ``` Create clients and models once per application runtime when practical. Agents, extractors, pipelines, and retrieval code can reuse the model instances. ## Model Listing [#model-listing] Use `listModels()` when you need provider-returned model ids or metadata for configuration screens, diagnostics, or model selection. ```ts const models = await client.listModels(); ``` Model listing fetches live provider data. Anvia does not cache results or add hidden metadata. Beta or private model ids can still be passed directly to `completionModel(...)`, but they only appear in `listModels()` when the provider returns them. See [Model Listing](/docs/models/model-listing) for the normalized response shape and compatible-gateway behavior. ## Compatible Gateways [#compatible-gateways] | Gateway | Page | | --------------------- | ------------------------------------------------------------------------------- | | OpenRouter | [OpenRouter](/docs/models/compatible-gateways/openrouter) | | LiteLLM | [LiteLLM](/docs/models/compatible-gateways/litellm) | | Vercel AI Gateway | [Vercel AI Gateway](/docs/models/compatible-gateways/vercel-ai-gateway) | | Cloudflare AI Gateway | [Cloudflare AI Gateway](/docs/models/compatible-gateways/cloudflare-ai-gateway) | | Portkey | [Portkey](/docs/models/compatible-gateways/portkey) | | Helicone AI Gateway | [Helicone AI Gateway](/docs/models/compatible-gateways/helicone-ai-gateway) | | LangDB AI Gateway | [LangDB AI Gateway](/docs/models/compatible-gateways/langdb-ai-gateway) | | MiniMax | [MiniMax](/docs/models/compatible-gateways/minimax) | | Moonshot AI | [Moonshot AI](/docs/models/compatible-gateways/moonshot-ai) | | Novita AI | [Novita AI](/docs/models/compatible-gateways/novita-ai) | | NVIDIA NIM | [NVIDIA NIM](/docs/models/compatible-gateways/nvidia-nim) | | Ollama | [Ollama](/docs/models/compatible-gateways/ollama) | | Ollama Cloud | [Ollama Cloud](/docs/models/compatible-gateways/ollama-cloud) | | OpenCode | [OpenCode](/docs/models/compatible-gateways/opencode) | Use compatible gateways when a service exposes an OpenAI-compatible API shape. In Anvia, these endpoints use `OpenAIClient({ baseUrl, apiKey })`. ## Providers [#providers] | Provider | Page | | -------------------- | ------------------------------------------------------------------------- | | OpenAI | [OpenAI Models](/docs/models/providers/openai) | | Anthropic | [Anthropic Models](/docs/models/providers/anthropic) | | Google Gemini | [Google Gemini Models](/docs/models/providers/gemini) | | Mistral | [Mistral Models](/docs/models/providers/mistral) | | Compatible providers | [Compatible Provider Models](/docs/models/providers/compatible-providers) | Provider support varies for attachments, tool behavior, streaming metadata, structured output, and provider-specific parameters. Anvia normalizes the SDK surface and validates known adapter capabilities, but it does not make every provider model identical. # Model Listing (/docs/models/model-listing) Provider clients can list models through the normalized `listModels()` method. ```ts import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ apiKey }); const models = await client.listModels(); for (const model of models.data) { console.log(model.id, model.contextLength); } ``` `listModels()` returns a `ModelList` from `@anvia/core/model-listing`. ```ts type ListedModel = { id: string; name?: string; description?: string; type?: string; createdAt?: number; ownedBy?: string; contextLength?: number; }; type ModelList = { data: ListedModel[]; }; ``` Only `id` is guaranteed. Providers and compatible gateways expose different metadata, so unknown fields remain omitted. ## Supported Clients [#supported-clients] ```ts await new OpenAIClient({ apiKey }).listModels(); await new AnthropicClient({ apiKey }).listModels(); await new GeminiClient({ apiKey }).listModels(); await new MistralClient({ apiKey }).listModels(); ``` Each client fetches live provider data. Anvia does not add hidden model metadata, cache the result, or include beta/private model ids that the provider does not return. ## Compatible Gateways [#compatible-gateways] OpenAI-compatible gateways use the same `OpenAIClient` path. ```ts const client = new OpenAIClient({ baseUrl: "https://openrouter.ai/api/v1", apiKey: process.env.OPENROUTER_API_KEY, }); const models = await client.listModels(); ``` The request goes to the configured gateway models endpoint, usually `GET {baseUrl}/models`. If the gateway returns sparse OpenAI-shaped data, Anvia normalizes fields such as `id`, `createdAt`, and `ownedBy`. If the gateway returns richer fields such as `name` or `context_length`, Anvia includes those as `name` and `contextLength`. If the gateway does not expose a compatible model-list endpoint, `listModels()` rejects with `ModelListingError`. ## Manual Model Lists [#manual-model-lists] You do not need `listModels()` to use a model. If your app already knows the allowed model ids, define a manual `ModelList` and pass those ids to the provider client. ```ts import type { ModelList } from "@anvia/core/model-listing"; import { OpenAIClient } from "@anvia/openai"; const models: ModelList = { data: [ { id: "openai/gpt-5-mini", name: "GPT-5 Mini", ownedBy: "openai", contextLength: 400_000, }, { id: "anthropic/claude-sonnet-4.6", name: "Claude Sonnet 4.6", ownedBy: "anthropic", contextLength: 200_000, }, ], }; const client = new OpenAIClient({ baseUrl: process.env.OPENAI_COMPATIBLE_BASE_URL, apiKey: process.env.OPENAI_COMPATIBLE_API_KEY, }); const selectedModelId = models.data[0]?.id ?? "openai/gpt-5-mini"; const model = client.completionModel(selectedModelId); ``` Use this pattern for configuration screens, private deployments, beta models, or gateways that do not expose `GET /models`. If you want a manually defined source to match the same shape as provider clients, implement `ModelListingClient`: ```ts import type { ModelList, ModelListingClient } from "@anvia/core/model-listing"; function staticModelListing(models: ModelList): ModelListingClient { return { async listModels() { return models; }, }; } ``` You can also merge live provider data with manual entries: ```ts const liveModels = await client.listModels().catch((): ModelList => ({ data: [] })); const mergedModels: ModelList = { data: [ ...models.data, ...liveModels.data.filter( (liveModel) => !models.data.some((manualModel) => manualModel.id === liveModel.id), ), ], }; ``` ## Unlisted Models [#unlisted-models] You can still use a known beta or private model id directly: ```ts const model = client.completionModel("provider/beta-model"); ``` That model will not appear in `listModels()` unless the provider includes it in the model-list response. ## Cookbook [#cookbook] Run the model-listing cookbook example: ```sh pnpm cookbook:providers:10 ``` # Anthropic (/docs/models/providers/anthropic) Use `@anvia/anthropic` when you want Anthropic completion models through Anvia's normalized runtime. ## Completion Model [#completion-model] ```ts import { AgentBuilder } from "@anvia/core"; import { AnthropicClient } from "@anvia/anthropic"; const client = new AnthropicClient({ apiKey }); const model = client.completionModel("claude-opus-4-6"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); ``` Anthropic models can be used anywhere Anvia expects a completion model. ## Custom Client Options [#custom-client-options] ```ts const client = new AnthropicClient({ apiKey: config.anthropic.apiKey, baseUrl: config.anthropic.baseUrl, }); ``` Use this form when secrets are injected by your app framework instead of read directly from `process.env`. ## Capabilities [#capabilities] | Capability | Example | | ---------- | --------------------------------------------- | | Completion | `client.completionModel("claude-opus-4-6")` | | Tool use | Supported through Anvia tools | | Streaming | Supported through normalized streaming events | Provider support can vary for attachments, tool behavior, streaming fields, and model-specific parameters. Anvia normalizes the SDK surface, but it does not make every model capability identical. # Compatible Providers (/docs/models/providers/compatible-providers) Use the top-level provider clients for compatible endpoints. Pass the endpoint URL and credentials explicitly; Anvia does not read provider configuration from environment variables. ## OpenAI-Compatible [#openai-compatible] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ baseUrl: "https://provider.example.com/v1", apiKey, }); const model = client.completionModel("provider-chat-model"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); ``` The compatible client can also create embedding models when the endpoint supports them. ```ts const embeddings = client.embeddingModel("provider-embedding-model"); ``` The same client can list models when the endpoint exposes an OpenAI-compatible model-list endpoint. ```ts const models = await client.listModels(); ``` `listModels()` fetches from the configured endpoint, usually `GET {baseUrl}/models`, and returns provider-reported data only. Anvia does not add hidden model metadata or cache the result. Anvia does not need a package for every OpenAI-compatible provider. If the provider exposes OpenAI-compatible completion or embedding endpoints, use `@anvia/openai` and pass the endpoint directly. ## Anthropic-Compatible [#anthropic-compatible] ```ts import { AgentBuilder } from "@anvia/core"; import { AnthropicClient } from "@anvia/anthropic"; const client = new AnthropicClient({ baseUrl: "https://provider.example.com", apiKey, }); const model = client.completionModel("provider-claude-compatible-model"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); ``` ## Portability Notes [#portability-notes] | Capability | Example | | --------------- | --------------------------------------- | | Completion | `client.completionModel(modelName)` | | Embeddings | `client.embeddingModel(modelName)` | | Model listing | `client.listModels()` | | Custom endpoint | `new OpenAIClient({ baseUrl, apiKey })` | Compatible APIs still differ in model names, attachments, streaming metadata, tool-call behavior, and provider-specific parameters. Treat compatible clients as a shared transport shape, not a guarantee that every model behaves the same. # Google Gemini (/docs/models/providers/gemini) Use `@anvia/gemini` for Gemini API and Vertex AI completion and embedding models. ## Gemini API [#gemini-api] ```ts import { AgentBuilder } from "@anvia/core"; import { GeminiClient } from "@anvia/gemini"; const client = new GeminiClient({ apiKey }); const model = client.completionModel("gemini-3.1-flash-lite-preview"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); ``` ## Vertex AI [#vertex-ai] ```ts const client = new GeminiClient({ vertexai: true, project, location, }); ``` Anvia does not read Google environment variables. Pass values from your application configuration. ## Embedding Model [#embedding-model] ```ts const embeddings = client.embeddingModel("gemini-embedding-001", { taskType: "RETRIEVAL_DOCUMENT", dimensions: 768, }); ``` Use embedding models for document preprocessing and retrieval. ## Capabilities [#capabilities] | Capability | Example | | ---------- | --------------------------------------------------------- | | Completion | `client.completionModel("gemini-3.1-flash-lite-preview")` | | Embeddings | `client.embeddingModel("gemini-embedding-001")` | | Vertex AI | `new GeminiClient({ vertexai: true, project, location })` | Gemini v1 support covers text completions, image and document input, tools, structured output, streaming, Gemini embeddings, and Vertex AI client construction. # Mistral (/docs/models/providers/mistral) Use `@anvia/mistral` when you want Mistral chat completion and embedding models through Anvia's normalized runtime. ## Completion Model [#completion-model] ```ts import { AgentBuilder } from "@anvia/core"; import { MistralClient } from "@anvia/mistral"; const client = new MistralClient({ apiKey }); const model = client.completionModel("mistral-large-latest"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); ``` `completionModel(...)` returns a reusable streaming completion model. Pass it to agents, extractors, or pipeline prompt steps. ## Embedding Model [#embedding-model] ```ts import { embedDocuments } from "@anvia/core"; const embeddings = client.embeddingModel("mistral-embed"); const embedded = await embedDocuments(embeddings, documents, { id: (doc) => doc.id, content: (doc) => `${doc.title}\n${doc.body}`, }); ``` Use embedding models for document preprocessing and retrieval. ## Custom Client Options [#custom-client-options] Use the constructor when credentials or a custom Mistral server URL come from your own configuration system. ```ts const client = new MistralClient({ apiKey: config.mistral.apiKey, serverURL: config.mistral.serverURL, }); ``` You can also pass an existing Mistral SDK client when your app already owns provider setup. ## Capabilities [#capabilities] | Capability | Example | | ----------------- | ------------------------------------------------ | | Completion | `client.completionModel("mistral-large-latest")` | | Embeddings | `client.embeddingModel("mistral-embed")` | | Custom server URL | `new MistralClient({ serverURL, apiKey })` | Mistral v1 support covers text completions, tools, tool choice, structured output, streaming, and embeddings. Image and document file attachments are intentionally rejected until Anvia adds full multimodal mapping for Mistral. # OpenAI (/docs/models/providers/openai) Use `@anvia/openai` when you want OpenAI completion and embedding models through Anvia's normalized runtime. ## Completion Model [#completion-model] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); const agent = new AgentBuilder("support", model) .instructions("Answer support questions clearly.") .build(); const response = await agent.prompt("Hello!").send(); ``` `completionModel(...)` returns a reusable model instance. Pass it to agents, extractors, or pipeline prompt steps. ## Embedding Model [#embedding-model] ```ts import { embedDocuments } from "@anvia/core"; const embeddings = client.embeddingModel("text-embedding-3-small"); const embedded = await embedDocuments(embeddings, documents, { id: (doc) => doc.id, content: (doc) => `${doc.title}\n${doc.body}`, }); ``` Use embedding models for document preprocessing and retrieval. ## Model Listing [#model-listing] ```ts const models = await client.listModels(); ``` `listModels()` fetches OpenAI's `/models` endpoint and returns a normalized `ModelList`. When `baseUrl` is set, the request goes to that OpenAI-compatible endpoint instead. OpenAI's model list is usually sparse, so fields such as `contextLength` may be omitted unless the compatible gateway returns them. ## Custom Client Options [#custom-client-options] Use the constructor when credentials or base URL come from your own configuration system. ```ts const client = new OpenAIClient({ apiKey: config.openai.apiKey, baseUrl: config.openai.baseUrl, }); ``` You can also pass an existing OpenAI SDK client when your app already owns provider setup. ## Capabilities [#capabilities] | Capability | Example | | ------------------- | ------------------------------------------------- | | Completion | `client.completionModel("gpt-5.5")` | | Embeddings | `client.embeddingModel("text-embedding-3-small")` | | Model listing | `client.listModels()` | | Compatible endpoint | `new OpenAIClient({ baseUrl, apiKey })` | Credentials are passed explicitly to the constructor. Anvia does not read environment variables. # API Coverage (/docs/reference/api-coverage) This page records the coverage policy that keeps handwritten reference pages aligned with public package exports. ## Scope [#scope] The coverage check treats every package entry in `package.json#exports` as a public import path. It uses the TypeScript compiler to enumerate symbols exported from those entrypoints, then verifies that the mapped reference docs mention each import path and exported symbol. Internal source-file exports are outside this check unless they are re-exported by a package entrypoint. ## Current Coverage [#current-coverage] | Package | Public entrypoints | Public exports | Reference coverage | | --------------------- | -----------------: | -------------: | --------------------------------------------------------- | | `@anvia/core` | 19 | 250 | [Core](/docs/reference/core) | | `@anvia/openai` | 1 | 17 | [OpenAI Provider](/docs/reference/providers/openai) | | `@anvia/gemini` | 1 | 13 | [Gemini Provider](/docs/reference/providers/gemini) | | `@anvia/anthropic` | 1 | 4 | [Anthropic Provider](/docs/reference/providers/anthropic) | | `@anvia/mistral` | 1 | 10 | [Mistral Provider](/docs/reference/providers/mistral) | | `@anvia/fastembed` | 1 | 6 | [FastEmbed](/docs/reference/integrations/fastembed) | | `@anvia/transformers` | 1 | 6 | [Transformers](/docs/reference/integrations/transformers) | | `@anvia/chroma` | 1 | 4 | [Chroma](/docs/reference/integrations/chroma) | | `@anvia/pgvector` | 1 | 6 | [pgvector](/docs/reference/integrations/pgvector) | | `@anvia/qdrant` | 1 | 5 | [Qdrant](/docs/reference/integrations/qdrant) | | `@anvia/langfuse` | 1 | 6 | [Langfuse](/docs/reference/integrations/langfuse) | | `@anvia/otel` | 1 | 3 | [OpenTelemetry](/docs/reference/integrations/otel) | | `@anvia/studio` | 1 | 61 | [Studio](/docs/reference/studio) | The check must report: ```txt TOTAL_MISSING_ENTRYPOINTS=0 TOTAL_MISSING_EXPORTS=0 ``` ## Re-run Check [#re-run-check] Run this from the repository root: ```bash pnpm docs:reference-check ``` `pnpm docs:typecheck` also runs the same check before MDX generation and TypeScript validation. ## Documentation Standard [#documentation-standard] Each practical reference page should document the public surface with: | Requirement | Expected content | | --------------- | ---------------------------------------------------------------------- | | Import source | Package or subpath imports, such as `@anvia/core/memory` | | Signature | TypeScript class, function, interface, type, or constant shape | | Purpose | What the primitive owns or represents | | Return behavior | Resolved value, emitted events, side effects, or type-only behavior | | Notable errors | Validation, provider, transport, persistence, or capability failures | | Example | Minimal code showing ordinary use when the primitive is runtime-facing | | Related docs | Guides or cookbook entries for workflow-oriented usage | # Agent (/docs/reference/core/agent) Import from `@anvia/core` or `@anvia/core/agent`. ## Agent [#agent] ```ts class Agent { readonly id: string; readonly name?: string; readonly description?: string; readonly model: M; readonly instructions?: string; readonly staticContext: Document[]; readonly temperature?: number; readonly maxTokens?: number; readonly additionalParams?: JsonValue; readonly toolSet: ToolSet; readonly toolChoice?: ToolChoice; readonly defaultMaxTurns?: number; readonly hook?: PromptHook; readonly outputSchema?: JsonObject; readonly observers: AgentObserverRegistration[]; readonly dynamicContexts: DynamicContextRegistration[]; readonly dynamicTools: DynamicToolRegistration[]; readonly toolMiddlewares: ToolMiddleware[]; readonly memory?: MemoryRegistration; readonly eventStore?: AgentEventStoreRegistration; constructor(options: AgentOptions); prompt(prompt: string | Message | Message[]): PromptRequest; session(sessionId: string, options?: SessionOptions): AgentSession; asTool(options: AgentToolOptions): Tool<{ prompt: string }, string>; getTool(toolName: string): Tool | undefined; callTool(toolName: string, args: string, context?: ToolCallContext): Promise; } ``` Purpose: immutable runnable agent configuration around one completion model. Return behavior: `prompt(...)` creates a mutable `PromptRequest`; `prompt(Message[])` treats the last message as the active prompt and earlier messages as stateless history; `session(...)` creates a durable memory-backed session; `asTool(...)` exposes the agent as a tool that returns the nested agent output string. `asTool({ stream: true })` forwards child stream events when the parent run uses `.stream()`. Notable errors: the constructor throws `TypeError` when `id` is not a non-empty string. `asTool(...)` forwards errors from the nested prompt run. ## AgentOptions [#agentoptions] ```ts type AgentOptions = { id: string; name?: string; description?: string; model: M; instructions?: string; staticContext?: Document[]; temperature?: number; maxTokens?: number; additionalParams?: JsonValue; toolSet?: ToolSet; toolChoice?: ToolChoice; defaultMaxTurns?: number; hook?: PromptHook; outputSchema?: JsonObject; observers?: AgentObserverRegistration[]; dynamicContexts?: DynamicContextRegistration[]; dynamicTools?: DynamicToolRegistration[]; toolMiddlewares?: ToolMiddleware[]; memory?: MemoryRegistration; eventStore?: AgentEventStoreRegistration; }; ``` Purpose: constructor contract for `Agent`. Prefer `AgentBuilder` for application code. Return behavior: used only as input to `new Agent(...)`. Notable errors: invalid `id` is rejected by the `Agent` constructor. ## AgentBuilder [#agentbuilder] ```ts class AgentBuilder { constructor(agentId: string, completionModel: M); name(name: string): this; description(description: string): this; instructions(instructions: string): this; context(text: string, id?: string): this; dynamicContext(index: VectorSearchIndex, options: DynamicContextOptions): this; dynamicTools(index: VectorSearchIndex, options: DynamicToolOptions): this; tool(tool: Tool): this; tools(tools: Tool[]): this; useToolSet(toolSet: ToolSet): this; mcp(servers: McpServer[]): this; skills(skillSet: SkillSet): this; temperature(temperature: number): this; maxTokens(maxTokens: number): this; additionalParams(params: JsonValue): this; toolChoice(toolChoice: ToolChoice): this; defaultMaxTurns(defaultMaxTurns: number): this; hook(hook: PromptHook): this; toolMiddleware(middleware: ToolMiddleware): this; toolMiddlewares(middlewares: ToolMiddleware[]): this; observe(observer: AgentObserver, options?: ObserveOptions): this; memory(store: MemoryStore, options?: MemoryOptions): this; eventStore(store: AgentEventStore, options?: AgentEventStoreOptions): this; outputSchema(schema: ZodSchema): this; build(): Agent; } ``` Purpose: fluent builder for agent configuration. Return behavior: all mutator methods return `this`; `build()` returns an `Agent`. Notable errors: the constructor rejects an empty agent id. `outputSchema(...)` can throw if the schema cannot be converted to provider JSON schema. ## AgentSession [#agentsession] ```ts class AgentSession { prompt(prompt: string | Message): PromptRequest; messages(): Promise; clear(): Promise; } ``` Purpose: durable conversation scope created by `agent.session(sessionId, options?)`. Return behavior: `prompt(...)` loads messages from the configured memory store before the run and appends new messages according to the agent memory policy. Session prompts do not accept `Message[]`; use `agent.prompt(Message[])` for explicit stateless transcripts. Notable errors: `agent.session(...)` throws when no memory store is configured or when the session id is empty. ## Memory [#memory] ```ts type MemorySavePolicy = "message" | "turn" | "run"; type MemoryContext = { sessionId: string; userId?: string; metadata?: JsonObject; }; interface MemoryStore { load(context: MemoryContext): Promise; append(input: { context: MemoryContext; runId: string; turn: number; messages: Message[]; }): Promise; clear(context: MemoryContext): Promise; recordError?(input: { context: MemoryContext; runId: string; error: unknown; messages: Message[]; }): Promise; } type MemoryOptions = { savePolicy?: MemorySavePolicy; }; ``` Purpose: configure durable conversation storage for `agent.session(...)`. Return behavior: `savePolicy` defaults to `"message"`. Core provides the interface; applications provide the storage implementation. ## AgentEventStore [#agenteventstore] ```ts interface AgentEventStore { append(input: AgentEventAppendInput): Promise; load(runId: string): Promise; clear?(runId: string): Promise; } type AgentEventStoreInclude = "all" | "agent_tool_events"; type AgentEventStoreOptions = { include?: AgentEventStoreInclude; }; type AgentEventAppendInput = { runId: string; agentId: string; agentName?: string; turn?: number; toolName?: string; toolCallId?: string; internalCallId?: string; event: unknown; }; type AgentEventRecord = AgentEventAppendInput & { createdAt?: Date; }; ``` Purpose: persist runtime stream events for replay, debugging, or local inspection. Return behavior: `include: "all"` stores parent and child stream events. `include: "agent_tool_events"` stores only nested child-agent events from streaming agent-tools. Event storage is separate from `MemoryStore`, which remains transcript-oriented. ## Dynamic Tools [#dynamic-tools] ```ts type DynamicToolOptions = { topK: number; threshold?: number; filter?: VectorFilter; }; type DynamicToolRegistration = { index: VectorSearchIndex; options: DynamicToolOptions; }; ``` Purpose: retrieve relevant tool definitions before each model turn. Return behavior: static tools are always sent; matching dynamic tools are added after static tools and deduped by name. Notable errors: errors from the vector index surface before the model request is sent. ## PromptRequest [#promptrequest] ```ts class PromptRequest { static fromAgent( agent: Agent, prompt: string | Message | Message[], ): PromptRequest; maxTurns(maxTurns: number): this; requestHook(hook: PromptHook): this; withToolConcurrency(concurrency: number): this; withToolMiddleware(middleware: ToolMiddleware): this; withToolMiddlewares(middlewares: ToolMiddleware[]): this; withTrace(trace: AgentTraceOptions): this; send(): Promise; stream(): AsyncIterable; readableStream(options?: ReadableStreamOptions): ReadableStream; } ``` Purpose: per-run request state for an agent prompt. Return behavior: `send()` resolves a final `PromptResponse`; `stream()` yields run events and ends with a `final` event; `readableStream()` wraps the stream in a web stream. Notable errors: throws `MaxTurnsError` when the tool loop exceeds the configured turn limit, `PromptCancelledError` when a hook terminates the run, and a plain `Error` when streaming is requested for a non-streaming model. ## PromptResponse [#promptresponse] ```ts type PromptResponse = { output: string; usage: Usage; messages: Message[]; trace?: AgentTraceInfo; }; ``` Purpose: final non-streaming agent result. Return behavior: `messages` contains the new run messages, not the full prior history unless history was manually included. Notable errors: none directly. ## AgentStreamEvent [#agentstreamevent] ```ts type AgentChildStreamEvent = Exclude; type AgentStreamEvent = | { type: "turn_start"; turn: number; prompt: Message; history: Message[] } | { type: "text_delta"; turn: number; delta: string } | { type: "reasoning_delta"; turn: number; delta: string; id?: string; contentType?: "text" | "summary" | "encrypted" | "redacted"; signature?: string } | { type: "tool_call"; turn: number; toolCall: ToolCall } | { type: "tool_result"; turn: number; toolName: string; toolCallId?: string; internalCallId: string; args: string; result: string } | { type: "agent_tool_event"; turn: number; toolName: string; toolCallId?: string; internalCallId: string; agentId: string; agentName?: string; event: AgentChildStreamEvent } | { type: "turn_end"; turn: number; response: CompletionResponse } | { type: "final"; runId: string; output: string; usage: Usage; messages: Message[]; trace?: AgentTraceInfo } | { type: "error"; error: unknown }; ``` Purpose: streaming event union for observing agent execution. Return behavior: emitted by `PromptRequest.stream()` and `readableStream()`. `agent_tool_event` appears when a child agent is exposed with `asTool({ stream: true })`. The terminal `final` event includes `runId`, which can be used with `AgentEventStore.load(...)`. Notable errors: terminal failures are yielded as `{ type: "error" }` and also originate from the same conditions as `send()`. ## Hooks [#hooks] ```ts type HookAction = { type: "continue" } | { type: "terminate"; reason: string }; type ToolApprovalRequestOptions = { reason?: string; rejectMessage?: string; }; type ToolCallHookAction = | { type: "continue" } | { type: "skip"; reason: string } | { type: "terminate"; reason: string } | ({ type: "approval_request" } & ToolApprovalRequestOptions); type RunControl = { continue(): HookAction; cancel(reason: string): HookAction; }; type ToolCallControl = { run(): ToolCallHookAction; skip(reason: string): ToolCallHookAction; cancel(reason: string): ToolCallHookAction; requestApproval(options?: ToolApprovalRequestOptions): ToolCallHookAction; }; type HookResult = HookAction | undefined; type ToolCallHookResult = ToolCallHookAction | undefined; type CompletionCallHookArgs = { prompt: Message; history: Message[]; run: RunControl; }; type CompletionResponseHookArgs = { prompt: Message; response: CompletionResponse; run: RunControl; }; type ToolHookArgs = { toolName: string; toolCallId?: string; internalCallId: string; args: string; }; type ToolCallHookArgs = ToolHookArgs & { tool: ToolCallControl; }; type ToolResultHookArgs = ToolHookArgs & { result: string; run: RunControl; }; const runControl: RunControl; const toolCallControl: ToolCallControl; function createHook(hook: PromptHook): PromptHook; function cancelPrompt(reason: string): HookAction; function skipTool(reason: string): ToolCallHookAction; function requestToolApproval(options?: ToolApprovalRequestOptions): ToolCallHookAction; ``` Purpose: intercept completion calls, completion responses, tool calls, and tool results. Return behavior: callback controls such as `tool.run()`, `tool.skip(...)`, `tool.cancel(...)`, `tool.requestApproval(...)`, `run.continue()`, and `run.cancel(...)` create actions consumed by `PromptRequest`. The low-level `cancelPrompt(...)`, `skipTool(...)`, and `requestToolApproval(...)` helpers are also available. Notable errors: a terminating hook produces `PromptCancelledError`. If `tool.requestApproval(...)` reaches core without Studio or another approval handler, the request cancels with `PromptCancelledError`. ## Error Classes [#error-classes] ```ts class MaxTurnsError extends Error { readonly maxTurns: number; readonly chatHistory: Message[]; readonly prompt: Message; } class PromptCancelledError extends Error { readonly chatHistory: Message[]; readonly reason: string; } ``` Purpose: typed agent-run failures. Return behavior: thrown by `PromptRequest`. Notable errors: these are the notable agent errors. ## Constants and Small Types [#constants-and-small-types] ```ts const DEFAULT_MAX_TURNS = 20; type AgentToolOptions = { name: string; description?: string; maxTurns?: number; stream?: boolean; }; type DynamicContextOptions = { topK: number; threshold?: number; filter?: VectorFilter; format?: (result: VectorSearchResult) => Document; }; type DynamicContextRegistration = { index: VectorSearchIndex; options: DynamicContextOptions; }; ``` Purpose: defaults and supporting agent configuration types. Return behavior: used as builder or constructor inputs. Notable errors: none directly. For workflow guidance, see [Creating Agents](/docs/guides/agents/creating-agents). # Audio Generation (/docs/reference/core/audio-generation) Import from `@anvia/core` or `@anvia/core/audio-generation`. ## AudioGenerationModel [#audiogenerationmodel] ```ts interface AudioGenerationModel { readonly provider?: string; readonly defaultModel?: string; audioGeneration(request: AudioGenerationRequest): Promise>; } ``` Purpose: provider-neutral audio generation and text-to-speech contract. Return behavior: providers return generated `audio` bytes, optional `mediaType`, and the provider response as `rawResponse`. ## Request Builder [#request-builder] ```ts type AudioGenerationRequest = { text: string; voice: string; speed: number; additionalParams?: JsonValue; }; type AudioGenerationResponse = { audio: Uint8Array; mediaType?: string; rawResponse: RawResponse; }; class AudioGenerationRequestBuilder { text(text: string): this; voice(voice: string): this; speed(speed: number): this; additionalParams(additionalParams: JsonValue): this; build(): AudioGenerationRequest; send(): Promise; } const response = await audioGenerationRequest(model) .text("Welcome to Anvia.") .voice("alloy") .speed(1) .additionalParams({ response_format: "mp3" }) .send(); ``` Purpose: Chainable builder for audio generation requests. Defaults: `text: ""`, `voice: ""`, and `speed: 1`. Notable errors: provider adapters reject on SDK or provider errors. # Completion (/docs/reference/core/completion) Import from `@anvia/core` or `@anvia/core/completion`. ## JSON Types [#json-types] ```ts type JsonPrimitive = string | number | boolean | null; type JsonValue = JsonPrimitive | JsonObject | JsonValue[]; type JsonObject = { [key: string]: JsonValue | undefined }; ``` Purpose: provider-safe JSON contracts used by tool arguments, model parameters, schemas, and metadata. Return behavior: type-only exports. Notable errors: none directly. ## Document [#document] ```ts type Document = { id: string; text: string; additionalProps?: Record; }; ``` Purpose: static or retrieved context document attached to a completion request. Return behavior: sent through `CompletionRequest.documents`. Notable errors: none directly. ## Content Types and Factories [#content-types-and-factories] ```ts type Text = { type: "text"; text: string; signature?: string }; type ImageDetail = "auto" | "low" | "high"; type ImageContent = { type: "image"; source: { type: "url"; url: string } | { type: "base64"; data: string; mediaType: string }; detail?: ImageDetail }; type DocumentContent = { type: "document"; source: { type: "url"; url: string; mediaType: string; filename?: string } | { type: "base64"; data: string; mediaType: string; filename?: string } | { type: "text"; text: string; mediaType?: string; filename?: string } }; type ReasoningContent = | { type: "text"; text: string; signature?: string } | { type: "summary"; text: string } | { type: "encrypted"; data: string } | { type: "redacted"; data: string }; type ReasoningContentType = ReasoningContent["type"]; type Reasoning = { type: "reasoning"; text: string; id?: string; content?: ReasoningContent[] }; type ToolFunction = { name: string; arguments: JsonValue }; type ToolCall = { type: "tool_call"; id: string; callId?: string; function: ToolFunction; signature?: string; additionalParams?: JsonValue }; type ToolResultContent = { type: "text"; text: string } | { type: "image"; data: string; mediaType?: string }; type ToolResult = { type: "tool_result"; id: string; callId?: string; content: ToolResultContent[] }; type ToolContent = ToolResult; ``` Purpose: normalized message content parts across providers. Return behavior: use the `UserContent`, `AssistantContent`, and `Message` factory objects to create valid values. Notable errors: none directly. ## Message [#message] ```ts type SystemMessage = { role: "system"; content: string }; type UserMessage = { role: "user"; content: UserContent[] }; type AssistantMessage = { role: "assistant"; id?: string; content: AssistantContent[] }; type ToolMessage = { role: "tool"; content: ToolContent[] }; type Message = SystemMessage | UserMessage | AssistantMessage | ToolMessage; const Message: { system(content: string): Message; user(content: string | UserContent[]): Message; assistant(content: string | AssistantContent[], id?: string): Message; tool(content: ToolContent | ToolContent[]): Message; }; ``` Purpose: normalized chat history and prompt shape. Return behavior: factory methods return message objects with normalized content arrays. Notable errors: none directly. ## UserContent, AssistantContent, and ToolContent [#usercontent-assistantcontent-and-toolcontent] ```ts const UserContent: { text(text: string): Text; imageUrl(url: string, options?: { detail?: ImageDetail }): ImageContent; imageBase64(data: string, mediaType: string, options?: { detail?: ImageDetail }): ImageContent; documentUrl(url: string, mediaType: string, options?: { filename?: string }): DocumentContent; documentBase64(data: string, mediaType: string, options?: { filename?: string }): DocumentContent; documentText(text: string): Text; }; const AssistantContent: { text(text: string): Text; imageUrl(url: string, options?: { detail?: ImageDetail }): ImageContent; imageBase64(data: string, mediaType: string, options?: { detail?: ImageDetail }): ImageContent; reasoning(text: string, id?: string): Reasoning; reasoningFromContent(content: ReasoningContent[], id?: string): Reasoning; reasoningSummary(text: string, id?: string): Reasoning; reasoningEncrypted(data: string, id?: string): Reasoning; reasoningRedacted(data: string, id?: string): Reasoning; toolCall(id: string, name: string, args: JsonValue, callId?: string): ToolCall; }; const ToolContent: { toolResult(id: string, content: string | ToolResultContent[], callId?: string): ToolResult; }; ``` Purpose: helper factories for user, assistant, and tool content. Return behavior: returns normalized content values. `AssistantContent.reasoning(text, id?)` keeps the legacy shape. Provider adapters can populate `reasoning.content` with structured text, summary, encrypted, or redacted blocks; `reasoning.text` remains the display-safe text made from text and summary blocks. Notable errors: none directly. ## ToolChoice and ToolDefinition [#toolchoice-and-tooldefinition] ```ts type ToolChoice = | "auto" | "required" | "none" | { type: "function"; name: string }; type ToolDefinition = { name: string; description: string; parameters: JsonObject; }; ``` Purpose: provider-facing tool selection and JSON schema definitions. Return behavior: passed through completion requests. Notable errors: providers can reject unsupported tool choice modes. ## Usage [#usage] ```ts type Usage = { inputTokens: number; outputTokens: number; totalTokens: number; cachedInputTokens: number; cacheCreationInputTokens: number; }; const Usage: { empty(): Usage; add(left: Usage, right: Usage): Usage; }; ``` Purpose: normalized token accounting. Return behavior: `empty()` returns zeroed usage; `add(...)` sums matching fields. Notable errors: none directly. ## CompletionRequest and CompletionResponse [#completionrequest-and-completionresponse] ```ts type CompletionRequest = { model?: string; instructions?: string; chatHistory: Message[]; documents: Document[]; tools: ToolDefinition[]; temperature?: number; maxTokens?: number; toolChoice?: ToolChoice; additionalParams?: JsonValue; outputSchema?: JsonObject; }; type CompletionResponse = { choice: AssistantContent[]; usage: Usage; rawResponse: RawResponse; messageId?: string; }; ``` Purpose: normalized model request and response contracts implemented by providers. Return behavior: models return one `CompletionResponse` per non-streaming completion. Notable errors: provider adapters can throw transport, authentication, or provider validation errors. ## Completion Models [#completion-models] ```ts type CompletionModelCapabilities = { streaming: boolean; tools: boolean; toolChoice: boolean; imageInput: boolean; documentInput: boolean; outputSchema: boolean; reasoning: boolean; }; interface CompletionModel { readonly provider: string; readonly defaultModel: string; readonly capabilities: CompletionModelCapabilities; completion(request: CompletionRequest): Promise>; } type CompletionStreamEvent = | { type: "text_delta"; delta: string } | { type: "reasoning_delta"; delta: string; id?: string; contentType?: ReasoningContent["type"]; signature?: string } | { type: "tool_call_delta"; id: string; callId?: string; name?: string; argumentsDelta?: string; signature?: string } | { type: "tool_call"; toolCall: ToolCall } | { type: "message_id"; id: string } | { type: "final"; response: CompletionResponse } | { type: "error"; error: unknown }; interface StreamingCompletionModel extends CompletionModel { streamCompletion(request: CompletionRequest): AsyncIterable>; } ``` Purpose: provider adapter interfaces. The metadata fields identify the adapter, its default model name, and the normalized request features the adapter supports. Return behavior: streaming models yield deltas and finish with a `final` event. Notable errors: provider adapters can throw transport, authentication, provider validation, stream, or capability validation errors. ## Capability Validation [#capability-validation] ```ts class CompletionCapabilityError extends Error {} function assertCompletionRequestSupported( model: CompletionModel, request: CompletionRequest, options?: { streaming?: boolean }, ): void; ``` Purpose: validates a normalized request against `model.capabilities`. Return behavior: returns `void` when the request is supported. Notable errors: throws `CompletionCapabilityError` for unsupported tools, tool choice, image input, file document input, output schema, or streaming. Static `CompletionRequest.documents` context and `UserContent.documentText(...)` are treated as text, not file document input. ## CompletionRequestBuilder [#completionrequestbuilder] ```ts class CompletionRequestBuilder { constructor(model: M, promptMessage: Message); modelOverride(model: string | undefined): this; instructions(instructions: string | undefined): this; messages(messages: Message[]): this; documents(documents: Document[]): this; tools(tools: ToolDefinition[]): this; temperature(temperature: number | undefined): this; maxTokens(maxTokens: number | undefined): this; toolChoice(toolChoice: ToolChoice | undefined): this; additionalParams(additionalParams: JsonValue | undefined): this; outputSchema(outputSchema: JsonObject | undefined): this; build(): CompletionRequest; send(): Promise; } ``` Purpose: fluent construction of provider requests. Return behavior: `build()` returns a `CompletionRequest`; `send()` validates capabilities and calls the configured model. Notable errors: `send()` throws `CompletionCapabilityError` before provider transport when the request uses unsupported features, and otherwise forwards model errors. ## Document Helpers [#document-helpers] ```ts function normalizeDocuments(documents: Document[]): Message | undefined; function formatDocument(document: Document): string; function textFromAssistantContent(content: AssistantContent[]): string; function reasoningDisplayText(content: ReasoningContent[]): string; ``` Purpose: convert documents and assistant content into text-oriented formats. Return behavior: `normalizeDocuments([])` returns `undefined`; `textFromAssistantContent(...)` joins text parts with newlines; `reasoningDisplayText(...)` joins text and summary reasoning blocks. Notable errors: none directly. # Embeddings (/docs/reference/core/embeddings) Import from `@anvia/core` or `@anvia/core/embeddings`. ## EmbeddingModel and Embedding [#embeddingmodel-and-embedding] ```ts type Embedding = { document: string; vector: number[]; }; interface EmbeddingModel { readonly dimensions?: number; readonly maxBatchSize?: number; embedTexts(texts: string[]): Promise; } ``` Purpose: provider-neutral embedding contract. Return behavior: `embedTexts(...)` must return one embedding per input text. Notable errors: provider implementations may throw; helpers throw when the returned count does not match the input count. ## EmbeddedDocument and Metadata [#embeddeddocument-and-metadata] ```ts type VectorMetadataValue = string | number | boolean | null; type VectorMetadata = Record; type EmbeddedDocument = { id: string; document: T; metadata?: Metadata; embeddings: Embedding[]; }; ``` Purpose: document plus one or more embeddings for vector stores. Return behavior: produced by `embedDocuments(...)`. Notable errors: none directly. ## EmbedDocumentsOptions [#embeddocumentsoptions] ```ts type EmbedDocumentsOptions = { id?: (document: T, index: number) => string; content(document: T, index: number): string | string[]; metadata?: (document: T, index: number) => Metadata | undefined; concurrency?: number; }; ``` Purpose: controls how typed documents become embedding inputs and metadata. Return behavior: used by `embedDocuments(...)`. Notable errors: invalid content callbacks can throw and fail embedding. ## Embedding Helpers [#embedding-helpers] ```ts function embedText(model: EmbeddingModel, text: string): Promise; function embedTexts(model: EmbeddingModel, texts: string[]): Promise; function embedDocuments( model: EmbeddingModel, documents: T[], options: EmbedDocumentsOptions, ): Promise>>; ``` Purpose: normalize batching, concurrency, and count validation. Return behavior: `embedTexts([])` returns `[]`; `embedText(...)` returns the first embedding from a single-item call. Notable errors: throws when the embedding model returns no embedding or a mismatched embedding count. ## Vector Math [#vector-math] ```ts function dotProduct(left: number[], right: number[]): number; function cosineSimilarity(left: number[], right: number[]): number; function angularDistance(left: number[], right: number[]): number; function euclideanDistance(left: number[], right: number[]): number; function manhattanDistance(left: number[], right: number[]): number; function chebyshevDistance(left: number[], right: number[]): number; ``` Purpose: distance and similarity utilities for embedding vectors. Return behavior: returns numeric scores or distances. Notable errors: throws when vectors have different dimensions. For workflow guidance, see [Embeddings](/docs/guides/retrieval/embeddings). # Evals (/docs/reference/core/evals) Import from `@anvia/core` or `@anvia/core/evals`. ## EvalCase [#evalcase] ```ts type EvalCase = { id: string; input: Input; expected?: Expected; metadata?: Record; }; ``` Purpose: one input and optional expected value for an eval suite. ## EvalOutcome [#evaloutcome] ```ts type EvalOutcomeStatus = "pass" | "fail" | "invalid"; type EvalOutcome = | { outcome: "pass"; score?: Score; comment?: string; metadata?: EvalMetadata } | { outcome: "fail"; score?: Score; comment?: string; metadata?: EvalMetadata } | { outcome: "invalid"; reason: string; score?: Score; comment?: string; metadata?: EvalMetadata }; ``` Purpose: normalized metric result. Use `EvalOutcome.pass(...)`, `EvalOutcome.fail(...)`, and `EvalOutcome.invalid(...)` to construct outcomes. ## EvalMetric [#evalmetric] ```ts type EvalMetric = { name: string; evaluate(args: EvalMetricArgs): EvalOutcome | Promise>; }; type EvalMetricArgs = { suiteName: string; case: EvalCase; output: Output; }; type EvalMetricResult = { metricName: string; outcome: EvalOutcome; reporterErrors: unknown[]; }; type EvalCaseResult = { case: EvalCase; output?: Output; targetError?: unknown; metrics: EvalMetricResult[]; }; ``` Purpose: evaluates one case output and records normalized metric results. ## EvalReporter [#evalreporter] ```ts type EvalReportArgs = { suiteName: string; case: EvalCase; output?: Output; targetError?: unknown; metric: EvalMetric; outcome: EvalOutcome; }; type EvalReporter = { report(args: EvalReportArgs): void | Promise; }; ``` Purpose: receives each metric outcome for persistence or external reporting. Return behavior: reporter errors are collected on metric results unless `failOnReporterError` is true. ## runEvalSuite [#runevalsuite] ```ts type RunEvalSuiteOptions = { name: string; cases: Array>; target: EvalTarget; metrics: Array>; concurrency?: number; reporters?: Array>; failOnReporterError?: boolean; }; type EvalSuiteResult = { name: string; results: Array>; passed: number; failed: number; invalid: number; durationMs: number; }; function runEvalSuite( options: RunEvalSuiteOptions, ): Promise>; ``` Purpose: runs cases through a target, evaluates each metric, calls optional reporters, and returns ordered results. Return behavior: target errors become invalid metric outcomes. Reporter errors are collected unless `failOnReporterError` is true. ## Built-in Metrics [#built-in-metrics] ```ts type ValueSelector = ( args: EvalMetricArgs, ) => Value | Promise; type SelectorOrValue = | Value | ValueSelector; type ExactMatchOptions = { name?: string; actual?: ValueSelector; expected?: SelectorOrValue; }; type ContainsOptions = { name?: string; actual?: ValueSelector; expected?: SelectorOrValue; }; type SemanticSimilarityOptions = { name?: string; model: EmbeddingModel; threshold: number; actual?: ValueSelector; expected?: SelectorOrValue; }; type LlmJudgeOptions = { name?: string; model: CompletionModel; schema: ZodSchema; passes(value: SchemaOutput): boolean; instructions?: string; retries?: number; prompt?: ValueSelector; }; type LlmScoreMetricScore = { score: number; feedback: string; }; type LlmScoreOptions = { name?: string; model: CompletionModel; threshold: number; criteria: string | string[]; instructions?: string; retries?: number; prompt?: ValueSelector; }; exactMatch(options?: ExactMatchOptions); contains(options?: ContainsOptions); semanticSimilarity(options: SemanticSimilarityOptions); llmJudge(options: LlmJudgeOptions); llmScore(options: LlmScoreOptions); ``` Purpose: common deterministic, embedding, and LLM-as-judge eval checks. ## agentEvalTarget [#agentevaltarget] ```ts type AgentEvalTargetOptions = { prompt?: (input: Input, testCase: EvalCase) => string | Message; output?: (response: PromptResponse, testCase: EvalCase) => Output; }; function agentEvalTarget( agent: Agent, options?: AgentEvalTargetOptions, ): EvalTarget; function agentEvalTarget( agent: Agent, options: AgentEvalTargetOptions, ): EvalTarget; ``` Purpose: adapts an `Agent` to an eval target by calling `agent.prompt(input).send()`. Return behavior: returns the full prompt response by default, or a selected value when `options.output` is provided. For workflow guidance, see [Evals](/docs/guides/testing/evals). # Extractor (/docs/reference/core/extractor) Import from `@anvia/core` or `@anvia/core/extractor`. ## ExtractionResponse [#extractionresponse] ```ts type ExtractionResponse = { data: T; usage: Usage; messages: Message[]; }; ``` Purpose: extraction result with usage and generated messages. Return behavior: returned by `Extractor.extractWithUsage(...)`. Notable errors: none directly. ## Extractor [#extractor] ```ts class Extractor { extract(text: string | Message): Promise; extractWithUsage(text: string | Message): Promise>; extractWithHistory(text: string | Message, history: Message[]): Promise; getInner(): Agent; } ``` Purpose: run a model with a required `submit` tool and parse the submitted arguments against a schema. Return behavior: `extract(...)` resolves schema-typed data; `getInner()` returns the underlying agent. Notable errors: throws `ExtractionError` when the model does not submit data, schema validation fails across all retries, or the model call fails. ## ExtractorBuilder [#extractorbuilder] ```ts class ExtractorBuilder { constructor(model: M, schema: ZodSchema); instructions(instructions: string): this; context(text: string, id?: string): this; temperature(temperature: number): this; maxTokens(maxTokens: number): this; additionalParams(params: JsonValue): this; toolChoice(toolChoice: ToolChoice): this; retries(retries: number): this; build(): Extractor; } ``` Purpose: configure an extractor around a completion model and Zod schema. Return behavior: mutators return `this`; `build()` returns an `Extractor`. Notable errors: schema conversion can throw; extraction errors are raised by the built extractor. ## ExtractionError [#extractionerror] ```ts class ExtractionError extends Error { readonly cause?: unknown; } ``` Purpose: wraps extraction failures. Return behavior: thrown by extraction methods. Notable errors: this is the notable extractor error. For workflow guidance, see [Extractors](/docs/guides/structured-output/extractors). # Image Generation (/docs/reference/core/image-generation) Import from `@anvia/core` or `@anvia/core/image-generation`. ## ImageGenerationModel [#imagegenerationmodel] ```ts interface ImageGenerationModel { readonly provider?: string; readonly defaultModel?: string; imageGeneration(request: ImageGenerationRequest): Promise>; } ``` Purpose: provider-neutral image generation contract. Return behavior: providers return the first image as `image`, all images as `images`, optional `mediaType`, and the provider response as `rawResponse`. ## Request Builder [#request-builder] ```ts type ImageGenerationRequest = { prompt: string; width: number; height: number; additionalParams?: JsonValue; }; type GeneratedImage = { data: Uint8Array; mediaType?: string; }; type ImageGenerationResponse = { image: Uint8Array; images: GeneratedImage[]; mediaType?: string; rawResponse: RawResponse; }; class ImageGenerationRequestBuilder { prompt(prompt: string): this; width(width: number): this; height(height: number): this; additionalParams(additionalParams: JsonValue): this; build(): ImageGenerationRequest; send(): Promise; } const response = await imageGenerationRequest(model) .prompt("A product diagram") .width(1024) .height(1024) .additionalParams({ output_format: "png" }) .send(); ``` Purpose: Chainable builder for image generation requests. Defaults: `prompt: ""`, `width: 1024`, and `height: 1024`. Notable errors: provider adapters may reject when the provider response contains no image bytes. # Core Reference (/docs/reference/core) `@anvia/core` is the provider-neutral runtime package. The root entry point re-exports the public runtime APIs from most core subpaths. ## Import Paths [#import-paths] | Import path | Area | | ------------------------------ | ------------------------------------------------------------------------------------ | | `@anvia/core` | Root export that re-exports the public core subpaths | | `@anvia/core/agent` | Agents, prompt requests, hooks, and run events | | `@anvia/core/completion` | Provider-facing completion messages, requests, responses, usage, and model contracts | | `@anvia/core/image-generation` | Provider-neutral image generation contracts and request builders | | `@anvia/core/audio-generation` | Provider-neutral audio generation contracts and request builders | | `@anvia/core/transcription` | Provider-neutral audio transcription contracts and request builders | | `@anvia/core/tool` | Tool definitions, registries, tool sets, serialization, and tool errors | | `@anvia/core/pipeline` | Typed pipelines and batch execution | | `@anvia/core/extractor` | Structured extraction helpers | | `@anvia/core/evals` | Eval suites, metrics, agent targets, and reporters | | `@anvia/core/loaders` | Node file and PDF loaders for ingestion preprocessing | | `@anvia/core/embeddings` | Embedding models, documents, and vector math | | `@anvia/core/model-listing` | Provider-neutral model listing contracts and errors | | `@anvia/core/vector-store` | In-memory vector store, vector filters, and vector search tools | | `@anvia/core/memory` | Durable session memory interfaces and in-memory session store | | `@anvia/core/mcp` | MCP connection helpers and normalized MCP types | | `@anvia/core/observability` | Observer interfaces, trace options, and score contracts | | `@anvia/core/skills` | Skill loading, local skill discovery, validation, and generated skill tools | | `@anvia/core/streaming` | Conversion from async iterables to web `ReadableStream` | ## Root Export Notes [#root-export-notes] The root `@anvia/core` export is the convenient application import path. Subpaths are useful when provider packages or libraries need tighter import boundaries. `@anvia/core/loaders` is subpath-only so normal core imports do not load Node filesystem and PDF extraction dependencies. ```ts import { AgentBuilder, createTool, Message, PipelineBuilder } from "@anvia/core"; import type { CompletionModel } from "@anvia/core/completion"; ``` For workflow guidance, start with [SDK Fundamentals](/docs/guides/sdk-fundamentals/runtime-boundaries). # Loaders (/docs/reference/core/loaders) Import from `@anvia/core/loaders`. Loaders are async iterables for ingestion pipelines. They read source material and yield `LoaderResult` values by default. Call `.ignoreErrors()` when a batch should skip unreadable files instead of returning failed results. Loaders are intentionally not exported from the root `@anvia/core` entry point because they depend on Node filesystem and PDF extraction packages. ## LoaderResult [#loaderresult] ```ts type LoaderResult = | { ok: true; value: T } | { ok: false; error: unknown }; type FileSource = { path: string } | { path: ""; bytes: Uint8Array }; type FileReadWithPath = { path: string; text: string }; type PdfSource = { path: string } | { path: ""; bytes: Uint8Array }; type PdfReadWithPath = { path: string; text: string }; type PdfPage = { pageNumber: number; text: string }; type PdfPageWithPath = { path: string; pageNumber: number; text: string }; ``` Purpose: keeps batch ingestion from throwing on the first failed item and names the file/PDF records yielded by loader modes. Return behavior: `.ignoreErrors()` unwraps successful values and filters failures. ## FileLoader [#fileloader] ```ts FileLoader.withGlob(pattern); FileLoader.withDir(directory); FileLoader.fromBytes(bytes); FileLoader.fromBytesMany(bytesArray); loader.read(); loader.readWithPath(); loader.ignoreErrors(); ``` Purpose: read UTF-8 text files from globs, directories, or memory. Return behavior: | Method | Output | | ----------------- | ----------------------------- | | `.read()` | `string` text | | `.readWithPath()` | `{ path, text }` | | `fromBytes(...)` | uses `""` as the path | `withDir(...)` reads direct files only; subdirectories are ignored. ## PdfFileLoader [#pdffileloader] ```ts PdfFileLoader.withGlob(pattern); PdfFileLoader.withDir(directory); PdfFileLoader.fromBytes(bytes); PdfFileLoader.fromBytesMany(bytesArray); loader.read(); loader.readWithPath(); loader.byPage(); loader.ignoreErrors(); ``` Purpose: extract text from PDF files or PDF bytes. Return behavior: | Method | Output | | -------------------------- | --------------------------------------------------- | | `.read()` | full PDF text | | `.readWithPath()` | `{ path, text }` | | `.byPage()` | `{ pageNumber, text }` with zero-based page numbers | | `.readWithPath().byPage()` | `{ path, pageNumber, text }` | PDF page splitting is the only built-in loader chunking behavior. ## Document Adapters [#document-adapters] ```ts fileToDocument(file); fileLoaderToDocuments(loader); pdfToDocument(pdf); pdfLoaderToDocuments(loader); pdfPageToDocument(page); pdfPageLoaderToDocuments(loader); ``` Purpose: convert successful loader outputs into Anvia `Document[]` for retrieval preprocessing. Text documents include `source` and `mediaType: "text/plain"` metadata. PDF documents include `source` and `mediaType: "application/pdf"` metadata. PDF page documents also include a string `pageNumber` metadata value. For workflow guidance, see [Embed Documents](/docs/guides/retrieval/embed-documents). # MCP (/docs/reference/core/mcp) Import from `@anvia/core` or `@anvia/core/mcp`. ## connectMcp [#connectmcp] ```ts function connectMcp(connection: McpConnection): Promise; ``` Purpose: connect to an MCP server and adapt its listed tools into Anvia tools. Return behavior: resolves an `McpServer` with `tools` and `close()`. Notable errors: rejects when the connection fails, tool listing fails, or the MCP SDK throws. ## mcp [#mcp] ```ts const mcp: { stdio(options: McpStdioOptions): McpConnection; http(options: McpHttpOptions): McpConnection; sse(options: McpSseOptions): McpConnection; }; ``` Purpose: factories for stdio, streamable HTTP, and SSE MCP connections. Return behavior: returns lazy connection objects; network or process work starts when `connectMcp(...)` calls `connect()`. Notable errors: connection errors are raised during `connect()`. ## MCP Types [#mcp-types] ```ts type McpToolDefinition = { name: string; description?: string; inputSchema: JsonObject; }; type McpToolCallContent = | { type: "text"; text: string } | { type: "image"; data: string; mimeType: string } | { type: "resource"; resource: { uri: string; text: string; mimeType?: string } | { uri: string; blob: string; mimeType?: string } }; type McpToolCallResult = | { content: McpToolCallContent[]; isError?: boolean } | { toolResult: unknown }; ``` Purpose: normalized subset of MCP tool metadata and results. Return behavior: used by MCP clients and adapters. Notable errors: none directly. ## Client and Server Types [#client-and-server-types] ```ts type McpClient = { listTools(): Promise<{ tools: McpToolDefinition[] }>; callTool(params: { name: string; arguments?: Record }): Promise; close(): Promise; }; type McpConnection = { readonly name: string; connect(): Promise; }; type McpServer = { readonly name: string; readonly tools: Tool[]; close(): Promise; }; type McpStdioOptions = StdioServerParameters & { name: string }; type McpHttpOptions = { name: string; url: string | URL; transport?: StreamableHTTPClientTransportOptions; }; type McpSseOptions = { name: string; url: string | URL; transport?: SSEClientTransportOptions; }; ``` Purpose: connection and lifecycle contracts. Return behavior: `McpServer.close()` closes the underlying client. Notable errors: `close()` may reject when the underlying client rejects. For workflow guidance, see [MCP Connections](/docs/guides/mcp/connections). # Memory (/docs/reference/core/memory) Import from `@anvia/core` or `@anvia/core/memory`. ## MemoryStore [#memorystore] ```ts interface MemoryStore { load(context: MemoryContext): Promise; append(input: MemoryAppendInput): Promise; clear(context: MemoryContext): Promise; recordError?(input: MemoryErrorInput): Promise; } ``` Purpose: application-owned persistence adapter for durable agent sessions. Return behavior: `load(...)` returns prior transcript messages for a session; `append(...)` persists new run messages; `clear(...)` deletes the session transcript; `recordError(...)` optionally receives partial run messages when a prompt run fails. Notable errors: store implementations should reject when persistence fails. Rejections from `load(...)`, `append(...)`, `clear(...)`, or `recordError(...)` surface through session prompt calls. ## MemoryContext [#memorycontext] ```ts type MemoryContext = { sessionId: string; userId?: string | undefined; metadata?: JsonObject | undefined; }; ``` Purpose: identifies the conversation scope loaded and saved by a `MemoryStore`. Return behavior: passed to every store method. `sessionId` comes from `agent.session(sessionId, options?)`; `userId` and `metadata` come from `SessionOptions`. Notable errors: `agent.session(...)` rejects empty session ids before creating a `MemoryContext`. ## MemoryAppendInput and MemoryErrorInput [#memoryappendinput-and-memoryerrorinput] ```ts type MemoryAppendInput = { context: MemoryContext; runId: string; turn: number; messages: Message[]; }; type MemoryErrorInput = { context: MemoryContext; runId: string; error: unknown; messages: Message[]; }; ``` Purpose: structured inputs for normal message persistence and failure recording. Return behavior: `messages` contains the transcript messages Anvia is asking the store to persist for that save point. `runId` and `turn` let stores group messages by run or model/tool loop turn. Notable errors: none directly; store implementations decide how to handle duplicate or partially persisted messages. ## MemoryOptions [#memoryoptions] ```ts type MemorySavePolicy = "message" | "turn" | "run"; type MemoryOptions = { savePolicy?: MemorySavePolicy | undefined; }; type ResolvedMemoryOptions = { savePolicy: MemorySavePolicy; }; function resolveMemoryOptions(options?: MemoryOptions): ResolvedMemoryOptions; ``` Purpose: configures when `AgentSession` appends messages to the configured store. Return behavior: `resolveMemoryOptions(...)` fills the default `savePolicy: "message"`. `AgentBuilder.memory(store, options?)` stores the resolved policy in `MemoryRegistration`. Notable errors: none directly. | Policy | Behavior | | ----------- | ---------------------------------------------------------------------------------- | | `"message"` | Save completed user, assistant, and tool-result messages as they become available. | | `"turn"` | Save completed messages after each model/tool loop turn. | | `"run"` | Save only after a successful final response. | ## Registration and Session Types [#registration-and-session-types] ```ts type MemoryRegistration = { store: MemoryStore; options: ResolvedMemoryOptions; }; type SessionOptions = { userId?: string | undefined; metadata?: JsonObject | undefined; }; ``` Purpose: internal agent configuration and per-session metadata contracts. Return behavior: `MemoryRegistration` is created by `AgentBuilder.memory(...)`; `SessionOptions` is passed to `agent.session(sessionId, options?)` and becomes part of `MemoryContext`. Notable errors: `agent.session(...)` throws when the agent has no memory store configured. ## Example [#example] ```ts import { AgentBuilder, type MemoryStore, type Message } from "@anvia/core"; class InProcessMemoryStore implements MemoryStore { private readonly sessions = new Map(); async load({ sessionId }) { return this.sessions.get(sessionId) ?? []; } async append({ context, messages }) { this.sessions.set(context.sessionId, [...(this.sessions.get(context.sessionId) ?? []), ...messages]); } async clear({ sessionId }) { this.sessions.delete(sessionId); } } const agent = new AgentBuilder("support", model) .memory(new InProcessMemoryStore(), { savePolicy: "turn" }) .build(); const response = await agent .session("thread_123", { userId: "user_456" }) .prompt("Continue from the previous answer.") .send(); ``` ## Related Guides [#related-guides] | Topic | Guide | | -------------------- | ----------------------------------------------------- | | Memory overview | [Memory](/docs/guides/memory) | | Raw SQL storage | [Raw SQL](/docs/guides/memory/raw-sql) | | Prisma storage | [Prisma](/docs/guides/memory/prisma) | | Drizzle storage | [Drizzle](/docs/guides/memory/drizzle) | | Multi-agent sessions | [Multi-Agent Memory](/docs/guides/memory/multi-agent) | # Model Listing (/docs/reference/core/model-listing) Import from `@anvia/core/model-listing` or `@anvia/core`. ## Model Listing Types [#model-listing-types] ```ts type ListedModel = { id: string; name?: string; description?: string; type?: string; createdAt?: number; ownedBy?: string; contextLength?: number; }; type ModelList = { data: ListedModel[]; }; interface ModelListingClient { listModels(): Promise; } ``` Purpose: normalized model-listing contracts shared by provider clients. Return behavior: provider clients fetch live provider model data and normalize known fields. Unknown fields remain omitted. ## ModelListingError [#modellistingerror] ```ts class ModelListingError extends Error { readonly provider?: string; readonly statusCode?: number; readonly cause?: unknown; } ``` Purpose: standard error wrapper for provider model-listing failures. Return behavior: thrown by provider `listModels()` implementations when SDK or provider requests fail. # Observability (/docs/reference/core/observability) Import from `@anvia/core` or `@anvia/core/observability`. ## Trace Types [#trace-types] ```ts type AgentTraceInfo = { traceId?: string; observationId?: string; }; type AgentTraceOptions = { name?: string; userId?: string; sessionId?: string; metadata?: JsonObject; tags?: string[]; version?: string; traceId?: string; failOnObserverError?: boolean; }; ``` Purpose: trace identity and metadata passed into runs. Return behavior: trace info can appear on prompt responses and final stream events. Notable errors: none directly. ## Run Observer Types [#run-observer-types] ```ts type AgentRunStartArgs = { agentName?: string; agentDescription?: string; instructions?: string; trace?: AgentTraceOptions; prompt: Message; history: Message[]; maxTurns: number; }; type AgentRunEndArgs = { output: string; usage: Usage; messages: Message[]; }; type AgentRunErrorArgs = { error: unknown; usage: Usage; messages: Message[] }; interface AgentRunObserver { readonly trace?: AgentTraceInfo; startGeneration?( args: AgentGenerationStartArgs, ): AgentGenerationObserver | undefined | Promise; startTool?( args: AgentToolStartArgs, ): AgentToolObserver | undefined | Promise; end(args: AgentRunEndArgs): void | Promise; error?(args: AgentRunErrorArgs): void | Promise; } ``` Purpose: observe one agent run. Return behavior: created by `AgentObserver.startRun(...)`. Notable errors: observer errors are ignored unless `failOnObserverError` is enabled. ## Generation and Tool Observer Types [#generation-and-tool-observer-types] ```ts type AgentGenerationStartArgs = { turn: number; request: CompletionRequest }; type AgentGenerationEndArgs = { turn: number; response: CompletionResponse; firstDeltaMs?: number; }; type AgentGenerationErrorArgs = { turn: number; error: unknown }; interface AgentGenerationObserver { end(args: AgentGenerationEndArgs): void | Promise; error?(args: AgentGenerationErrorArgs): void | Promise; } type AgentToolStartArgs = { turn: number; toolCall: ToolCall; toolName: string; args: string; internalCallId: string; toolCallId?: string; }; type AgentToolEndArgs = AgentToolStartArgs & { result: string; skipped: boolean }; type AgentToolErrorArgs = AgentToolStartArgs & { error: unknown }; type AgentToolStreamEventArgs = AgentToolStartArgs & { event: ToolCallStreamEvent; }; interface AgentToolObserver { streamEvent?(args: AgentToolStreamEventArgs): void | Promise; end(args: AgentToolEndArgs): void | Promise; error?(args: AgentToolErrorArgs): void | Promise; } ``` Purpose: observe model calls and tool calls inside a run. Return behavior: called by the agent runtime as events stream, complete, or fail. `streamEvent(...)` receives nested child-agent stream events emitted by agent tools. Notable errors: observer errors follow the registration error policy. ## AgentObserver and Registration [#agentobserver-and-registration] ```ts interface AgentObserver { startRun( args: AgentRunStartArgs, ): AgentRunObserver | undefined | Promise; flush?(): void | Promise; shutdown?(): void | Promise; } type AgentObserverRegistration = { observer: AgentObserver; failOnObserverError?: boolean; }; type ObserveOptions = { failOnObserverError?: boolean; }; function createObserver(observer: AgentObserver): AgentObserver; ``` Purpose: top-level observer plugin contract. Return behavior: `createObserver(...)` returns the observer unchanged, mainly for typing. Notable errors: observer errors are ignored unless `failOnObserverError` is enabled. For workflow guidance, see [Tracing](/docs/guides/observability/tracing). # Pipeline (/docs/reference/core/pipeline) Import from `@anvia/core` or `@anvia/core/pipeline`. ## PipelineOp [#pipelineop] ```ts interface PipelineOp { run(input: Input): Output | Promise; } ``` Purpose: minimal interface for anything runnable as a pipeline stage. Return behavior: returns or resolves one output for one input. Notable errors: implementations can throw arbitrary errors. ## PipelineBatchOptions [#pipelinebatchoptions] ```ts interface PipelineBatchOptions { concurrency: number; } ``` Purpose: controls bounded parallelism for `Pipeline.batch(...)`. Return behavior: used as input. Notable errors: invalid values are normalized to at least `1`. ## Pipeline [#pipeline] ```ts class Pipeline implements PipelineOp> { readonly id: string; readonly name: string | undefined; readonly description: string | undefined; readonly metadata: JsonObject | undefined; run(input: Input, options?: PipelineRunOptions): Promise>; batch>( inputs: I, options: PipelineBatchOptions, ): Promise>>; graph(): PipelineGraph; } ``` Purpose: runnable pipeline returned by `PipelineBuilder.build()`. Return behavior: `run(...)` resolves the final stage output; `batch(...)` preserves input order; `graph()` returns inspectable pipeline metadata, nodes, and edges. Notable errors: forwards stage errors. ## Pipeline Graph Types [#pipeline-graph-types] ```ts type PipelineMetadata = { id?: string; name?: string; description?: string; metadata?: JsonObject; }; type PipelineStageMetadata = { id?: string; name?: string; description?: string; metadata?: JsonObject; }; type PipelineStageKind = | "input" | "step" | "pipeline" | "parallel" | "branch" | "agent" | "extractor" | "output"; type PipelineGraphNode = { id: string; kind: PipelineStageKind; label: string; description?: string; metadata?: JsonObject; agentId?: string; agentName?: string; pipelineId?: string; branchKey?: string; }; type PipelineGraphEdge = { id: string; source: string; target: string; label?: string; }; type PipelineGraph = PipelineMetadata & { id: string; nodes: PipelineGraphNode[]; edges: PipelineGraphEdge[]; }; ``` Purpose: automatic graph metadata for Studio and other inspectors. Return behavior: stage ids and labels are generated from build order unless optional metadata supplies better values. Notable errors: none directly. ## Pipeline Run Events [#pipeline-run-events] ```ts type PipelineRunEvent = | { type: "stage_started"; node: PipelineGraphNode } | { type: "stage_completed"; node: PipelineGraphNode; durationMs: number } | { type: "stage_failed"; node: PipelineGraphNode; durationMs: number; error: unknown }; type PipelineRunObserver = { onEvent(event: PipelineRunEvent): void | Promise; }; type PipelineRunOptions = { observer?: PipelineRunObserver; }; ``` Purpose: metadata-only execution events for runtimes such as Studio. Return behavior: pass an observer to `pipeline.run(input, { observer })` to receive stage status changes. Notable errors: observer errors propagate to the run. ## PipelineBuilder [#pipelinebuilder] ```ts class PipelineBuilder { constructor(); constructor(metadata: PipelineMetadata); step( fn: (input: Awaited) => Next | Promise, metadata?: PipelineStageMetadata, ): PipelineBuilder>; use( op: PipelineOp, Next>, metadata?: PipelineStageMetadata, ): PipelineBuilder>; parallel, unknown>>>( branches: Branches, metadata?: PipelineStageMetadata, ): PipelineBuilder>; prompt(agent: Agent, metadata?: PipelineStageMetadata): PipelineBuilder; extract(extractor: Extractor, metadata?: PipelineStageMetadata): PipelineBuilder; build(): Pipeline>; } ``` Purpose: typed composition of transform functions, operations, agents, and extractors. Return behavior: each composition method returns a new builder with the inferred output type. Notable errors: forwards errors from transform functions, nested operations, agents, or extractors. For workflow guidance, see [Pipeline Builder](/docs/guides/pipelines/pipeline-builder). # Schema (/docs/reference/core/schema) Import `ZodSchema` from `@anvia/core`. ## ZodSchema [#zodschema] ```ts type ZodSchema = z.ZodType; ``` Purpose: shared type alias for schema inputs accepted by tools, agents, and extractors. Return behavior: type-only export. Notable errors: schema parsing errors are thrown by the APIs that use the schema. The internal JSON schema conversion helper is not a public package export. Use `outputSchema(...)`, `createTool(...)`, or `ExtractorBuilder` instead of calling conversion code directly. # Skills (/docs/reference/core/skills) Import from `@anvia/core` or `@anvia/core/skills`. ## Skill Types [#skill-types] ```ts type Skill = { readonly name: string; readonly description: string; readonly instructions: string; readonly directory: string; readonly references: string[]; readonly scripts: string[]; readonly license?: string; readonly metadata?: Record; }; type SkillLoader = { load(): Promise; }; type SkillSet = { readonly skills: Skill[]; readonly tools: Tool[]; readonly instructions: string; }; ``` Purpose: loaded skill metadata and instruction/tool bundle. Return behavior: `SkillSet` is consumed by `AgentBuilder.skills(...)`. Notable errors: loaders can reject. ## loadSkills [#loadskills] ```ts function loadSkills(loaders: SkillLoader | SkillLoader[]): Promise; ``` Purpose: load one or more skill sources, merge by skill name, generate instructions, and create skill tools. Return behavior: later loaders override earlier skills with the same name. Notable errors: rejects with loader errors or `SkillValidationError` from local loaders. ## skill.local [#skilllocal] ```ts const skill: { local(path: string): SkillLoader; }; ``` Purpose: create a loader for one skill directory or a directory containing multiple skill directories. Return behavior: `load()` discovers `SKILL.md`, `references/`, and `scripts/`. Notable errors: throws `SkillValidationError` for invalid frontmatter, invalid names, missing descriptions, or directory/name mismatches. ## Validation Types [#validation-types] ```ts type SkillValidationIssue = { path: string; message: string; }; class SkillValidationError extends Error { readonly issues: SkillValidationIssue[]; } ``` Purpose: structured local skill validation failures. Return behavior: thrown by local loading. Notable errors: this is the notable skills error. For workflow guidance, see [Skill Files](/docs/guides/skills/skill-files). # Streaming (/docs/reference/core/streaming) Import from `@anvia/core` or `@anvia/core/streaming`. ## ReadableStreamOptions [#readablestreamoptions] ```ts type ReadableStreamOptions = { signal?: AbortSignal; }; ``` Purpose: cancellation options for stream conversion. Return behavior: passed to `toReadableStream(...)`. Notable errors: aborting the signal cancels the stream. ## toReadableStream [#toreadablestream] ```ts function toReadableStream( iterable: AsyncIterable, options?: ReadableStreamOptions, ): ReadableStream; ``` Purpose: expose async iterables through the standard web stream interface. Return behavior: pulls values from the iterable and enqueues them in a `ReadableStream`. Notable errors: errors thrown by the async iterable error the stream. For workflow guidance, see [Readable Streams](/docs/guides/streaming/readable-streams). # Tools (/docs/reference/core/tools) Import from `@anvia/core` or `@anvia/core/tool`. ## Tool [#tool] ```ts interface Tool { readonly name: string; readonly approval?: ToolApprovalPolicy; definition(prompt: string): ToolDefinition | Promise; call(args: Args, context?: ToolCallContext): Output | Promise; parseApprovalArgs?(args: unknown): Args; } type AnyTool = Omit, "approval"> & { readonly approval?: unknown; }; type ToolCallStreamEvent = { agentId: string; agentName?: string; event: unknown; }; type ToolCallContext = { emitStreamEvent?(event: ToolCallStreamEvent): void | Promise; }; ``` Purpose: normalized callable tool contract. Return behavior: `definition(...)` exposes provider JSON schema; `call(...)` executes local logic. The optional context is used by runtime-managed tools such as streaming agent-tools. Approval metadata is passive and is not included in provider tool definitions. Notable errors: tool implementations can throw arbitrary errors. ## createTool and CreateToolOptions [#createtool-and-createtooloptions] ```ts type CreateToolOptions = { name: string; description: string; input: InputSchema; output?: OutputSchema; approval?: ToolApprovalPolicy>; execute(args: z.output, context: ToolCallContext): unknown | Promise; }; function createTool( options: CreateToolOptions, ): Tool, ToolOutput>; ``` Purpose: create a typed tool from Zod input and optional output schemas. Return behavior: parses arguments before execution and parses output when an output schema is supplied. Notable errors: input or output validation errors throw from `call(...)`; schema conversion can throw during creation. ## ToolApprovalPolicy [#toolapprovalpolicy] ```ts type ToolApprovalContext = { toolName: string; args: Args; rawArgs: string; toolCallId?: string; internalCallId: string; run: ToolApprovalRunContext; }; type ToolApprovalRunContext = { agentId: string; runId: string; sessionId?: string; metadata?: JsonObject; }; type ToolApprovalPolicy = { when(ctx: ToolApprovalContext): boolean | Promise; reason?: string | ((ctx: ToolApprovalContext) => string | Promise); rejectMessage?: string | ((ctx: ToolApprovalContext) => string | Promise); }; ``` Purpose: attach portable approval metadata to a tool for runtimes such as Studio. Return behavior: core stores this metadata only. Core prompt execution ignores it unless a hook or runtime interprets it. Notable errors: runtime-specific approval evaluators can surface errors thrown by `when(...)`, `reason(...)`, or `rejectMessage(...)`. ## ToolMiddleware [#toolmiddleware] ```ts type ToolResultMiddlewareArgs = { toolName: string; args: string; result: string; originalResult: string; turn: number; toolCallId?: string; internalCallId: string; }; interface ToolMiddleware { onResult?(args: ToolResultMiddlewareArgs): string | undefined | Promise; } function createToolMiddleware(middleware: ToolMiddleware): ToolMiddleware; ``` Purpose: transform serialized tool results during agent runs before the model, stream events, hooks, observers, and messages receive the result. Return behavior: returning a string replaces the current result; returning `undefined` keeps it. Multiple middleware callbacks run in registration order. Skill runtime tools are excluded. Notable errors: errors thrown by middleware surface as prompt run errors. ## ToolSet [#toolset] ```ts class ToolSet { static fromTools(tools: Tool[]): ToolSet; addTool(tool: Tool): this; addTools(tools: Tool[] | ToolSet): this; deleteTool(toolName: string): boolean; contains(toolName: string): boolean; get(toolName: string): Tool | undefined; values(): Tool[]; getToolDefinitions(prompt?: string): Promise; call(toolName: string, args: string, context?: ToolCallContext): Promise; } ``` Purpose: storage and execution boundary for concrete tool implementations. Return behavior: `call(...)` parses JSON args, executes the matching tool, and serializes the output as a string. Notable errors: throws `ToolNotFoundError`, `ToolJsonError`, or `ToolCallError`. ## Dynamic Tool Selection [#dynamic-tool-selection] ```ts type ToolSearchDocument = { toolName: string; definition: ToolDefinition; text: string; metadata?: VectorMetadata; }; type EmbedToolsOptions = { content?: (tool: Tool, definition: ToolDefinition) => string | string[]; metadata?: (tool: Tool, definition: ToolDefinition) => VectorMetadata | undefined; concurrency?: number; }; interface DynamicToolIndex extends VectorSearchIndex { readonly toolSet: ToolSet; } function embedTools(model: EmbeddingModel, tools: Tool[] | ToolSet, options?: EmbedToolsOptions): Promise[]>; function createToolIndex(model: EmbeddingModel, tools: Tool[] | ToolSet, options?: EmbedToolsOptions): Promise; function isDynamicToolIndex(value: unknown): value is DynamicToolIndex; ``` Purpose: build a searchable index of tool capabilities for `AgentBuilder.dynamicTools(...)`. Return behavior: `embedTools(...)` returns embedded tool records; `createToolIndex(...)` returns an in-memory vector index that also carries the executable `ToolSet`; `isDynamicToolIndex(...)` checks for that `ToolSet` marker. Notable errors: tool definition or embedding failures reject the returned promise. ## createThinkTool [#createthinktool] ```ts type CreateThinkToolOptions = { name?: string; description?: string; }; function createThinkTool(options?: CreateThinkToolOptions): Tool<{ thought: string }, string>; ``` Purpose: create a built-in private reasoning tool that echoes a thought string. Return behavior: returns a tool named `think` unless overridden. Notable errors: validation errors can occur if the model calls it with invalid arguments. ## Serialization Helpers [#serialization-helpers] ```ts function serializeToolOutput(output: unknown): string; function parseToolArgs(args: string): JsonValue; ``` Purpose: convert tool outputs and model-supplied argument strings. Return behavior: `parseToolArgs("")` returns `{}`; `serializeToolOutput(...)` returns strings unchanged and JSON-stringifies other values. Notable errors: `parseToolArgs(...)` throws `SyntaxError` for invalid JSON. ## Error Classes [#error-classes] ```ts class ToolCallError extends Error { readonly cause?: unknown; } class ToolNotFoundError extends Error { readonly toolName: string; } class ToolJsonError extends Error { readonly cause?: unknown; } ``` Purpose: typed failures from `ToolSet.call(...)`. Return behavior: thrown errors. Notable errors: these are the notable tool errors. For workflow guidance, see [Creating Tools](/docs/guides/tools/creating-tools). # Transcription (/docs/reference/core/transcription) Import from `@anvia/core` or `@anvia/core/transcription`. ## TranscriptionModel [#transcriptionmodel] ```ts interface TranscriptionModel { readonly provider?: string; readonly defaultModel?: string; transcription(request: TranscriptionRequest): Promise>; } ``` Purpose: provider-neutral audio transcription contract. Return behavior: providers return normalized `text` and the provider response as `rawResponse`. ## Request Builder [#request-builder] ```ts type TranscriptionRequest = { data: Uint8Array; filename: string; language?: string; prompt?: string; temperature?: number; additionalParams?: JsonValue; }; type TranscriptionResponse = { text: string; rawResponse: RawResponse; }; class TranscriptionRequestBuilder { data(data: Uint8Array | ArrayBuffer): this; filename(filename: string): this; language(language: string): this; prompt(prompt: string): this; temperature(temperature: number): this; additionalParams(additionalParams: JsonValue): this; build(): TranscriptionRequest; send(): Promise; } const response = await transcriptionRequest(model) .data(audioBytes) .filename("meeting.mp3") .language("en") .prompt("Transcribe exactly.") .temperature(0) .send(); ``` Purpose: Chainable builder for transcription requests. Defaults: `filename: "file"` and empty data. Notable errors: `.build()` and `.send()` throw when data is empty. V1 does not include a filesystem `loadFile()` helper; callers pass bytes. # Vector Store (/docs/reference/core/vector-store) Import from `@anvia/core` or `@anvia/core/vector-store`. ## VectorFilter and vectorFilter [#vectorfilter-and-vectorfilter] ```ts type VectorFilter = | { type: "eq"; key: string; value: VectorMetadataValue } | { type: "gt"; key: string; value: VectorMetadataValue } | { type: "lt"; key: string; value: VectorMetadataValue } | { type: "and"; filters: [VectorFilter, VectorFilter] } | { type: "or"; filters: [VectorFilter, VectorFilter] }; const vectorFilter: { eq(key: string, value: VectorMetadataValue): VectorFilter; gt(key: string, value: VectorMetadataValue): VectorFilter; lt(key: string, value: VectorMetadataValue): VectorFilter; and(left: VectorFilter, right: VectorFilter): VectorFilter; or(left: VectorFilter, right: VectorFilter): VectorFilter; }; ``` Purpose: composable metadata filters for vector search. Return behavior: factory methods return serializable filter objects. Notable errors: unsupported metadata comparison types do not match results. ## Search Types [#search-types] ```ts type IndexStrategy = { type: "bruteForce" } | { type: "lsh"; numTables: number; numHyperplanes: number; seed?: number }; type VectorSearchRequest = { query: string; topK: number; threshold?: number; filter?: VectorFilter; }; type VectorSearchResult = { score: number; id: string; document: T; metadata?: Metadata; }; type VectorInspectRequest = { limit: number; cursor?: string; filter?: VectorFilter; }; type VectorInspectItem = { id: string; document: T; metadata?: Metadata; }; type VectorInspectPage = { items: Array>; nextCursor?: string; totalCount?: number; }; type VectorSearchToolOptions = { name: string; description?: string; topK?: number; threshold?: number; filter?: VectorFilter; }; ``` Purpose: vector indexing, search, and tool option contracts. Return behavior: used as inputs and outputs by vector indexes. Notable errors: invalid `topK` values are normalized by concrete indexes. `VectorStore` classes own documents. `VectorSearchIndex` is the query-time interface bound to an embedding model. `IndexStrategy` configures local candidate selection for the in-memory store. ## VectorSearchIndex [#vectorsearchindex] ```ts interface VectorSearchIndex { search(request: VectorSearchRequest): Promise>>; searchIds(request: VectorSearchRequest): Promise>; asTool(options: VectorSearchToolOptions): Tool<{ query: string; topK?: number }, unknown>; inspect?(request: VectorInspectRequest): Promise>; } ``` Purpose: search interface shared by in-memory and integration-backed vector stores. Return behavior: `searchIds(...)` strips documents and metadata; `asTool(...)` wraps search as a tool. `inspect(...)` is optional and returns a cursor page for UIs that need to browse indexed documents. Notable errors: model or store failures reject the returned promises. ## InMemoryVectorStore [#inmemoryvectorstore] ```ts class InMemoryVectorStore { constructor(options?: { index?: IndexStrategy }); static fromDocuments( documents: Array>, options?: { index?: IndexStrategy }, ): InMemoryVectorStore; addDocuments(documents: Array>): this; get(id: string): EmbeddedDocument | undefined; values(): Array>; len(): number; isEmpty(): boolean; index(model: EmbeddingModel): InMemoryVectorIndex; } ``` Purpose: local vector document store with brute force or LSH candidate selection. Return behavior: `index(...)` binds the store to an embedding model for searching. Notable errors: LSH index setup can fail if vectors have inconsistent dimensions. ## InMemoryVectorIndex [#inmemoryvectorindex] ```ts class InMemoryVectorIndex implements VectorSearchIndex { search(request: VectorSearchRequest): Promise>>; searchIds(request: VectorSearchRequest): Promise>; asTool(options: VectorSearchToolOptions): Tool<{ query: string; topK?: number }, unknown>; } ``` Purpose: query-time embedding and scoring over an `InMemoryVectorStore`. Return behavior: returns top results sorted by descending cosine score. Notable errors: embedding model failures reject search calls. ## createVectorSearchTool [#createvectorsearchtool] ```ts function createVectorSearchTool( index: VectorSearchIndex, options: VectorSearchToolOptions, ): Tool<{ query: string; topK?: number }, Array>>; ``` Purpose: expose any vector index as an agent tool. Return behavior: returns a typed tool whose output is vector search results. Notable errors: tool execution rejects when search rejects. For workflow guidance, see [Vector Stores](/docs/guides/retrieval/vector-stores). # SDK Reference (/docs/reference) The reference section documents public package exports, constructor options, return values, and TypeScript contracts. Use guides when you want workflow guidance. Use reference pages when you already know the primitive you need and want its public shape. ## Public Packages [#public-packages] | Package | Purpose | | --------------------- | ------------------------------------------------------------------------------------------------------- | | `@anvia/core` | Agent runtime, tools, context, workflows, streaming, retrieval primitives, and observability interfaces | | `@anvia/openai` | OpenAI completion and embedding client | | `@anvia/anthropic` | Anthropic completion client | | `@anvia/gemini` | Gemini API and Vertex AI completion and embedding client | | `@anvia/mistral` | Mistral completion and embedding client | | `@anvia/studio` | Local Studio runtime and inspection UI | | `@anvia/langfuse` | Langfuse observer integration | | `@anvia/otel` | OpenTelemetry observer integration | | `@anvia/chroma` | Chroma vector store integration | | `@anvia/qdrant` | Qdrant vector store integration | | `@anvia/pgvector` | Postgres pgvector store integration | | `@anvia/fastembed` | Local FastEmbed embedding integration | | `@anvia/transformers` | Local Transformers embedding integration | ## Package Entry Points [#package-entry-points] | Package | Public import paths | | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `@anvia/core` | `@anvia/core`, `@anvia/core/agent`, `@anvia/core/audio-generation`, `@anvia/core/completion`, `@anvia/core/embeddings`, `@anvia/core/evals`, `@anvia/core/extractor`, `@anvia/core/image-generation`, `@anvia/core/loaders`, `@anvia/core/mcp`, `@anvia/core/memory`, `@anvia/core/model-listing`, `@anvia/core/observability`, `@anvia/core/pipeline`, `@anvia/core/skills`, `@anvia/core/streaming`, `@anvia/core/tool`, `@anvia/core/transcription`, `@anvia/core/vector-store` | | `@anvia/openai` | `@anvia/openai` | | `@anvia/anthropic` | `@anvia/anthropic` | | `@anvia/gemini` | `@anvia/gemini` | | `@anvia/mistral` | `@anvia/mistral` | | `@anvia/studio` | `@anvia/studio` | | `@anvia/chroma` | `@anvia/chroma` | | `@anvia/qdrant` | `@anvia/qdrant` | | `@anvia/pgvector` | `@anvia/pgvector` | | `@anvia/fastembed` | `@anvia/fastembed` | | `@anvia/transformers` | `@anvia/transformers` | | `@anvia/langfuse` | `@anvia/langfuse` | | `@anvia/otel` | `@anvia/otel` | ## Sections [#sections] | Section | Contains | | -------------------------------------------- | --------------------------------------------------------------------------------------- | | [Core](/docs/reference/core) | Public `@anvia/core` exports and subpaths | | [Providers](/docs/reference/providers) | OpenAI, Anthropic, Gemini, Mistral, and compatible model adapters | | [Integrations](/docs/reference/integrations) | Chroma, Qdrant, pgvector, Transformers.js, Langfuse, and OpenTelemetry adapters | | [Studio](/docs/reference/studio) | Studio runtime, HTTP contracts, sessions, traces, approvals, stores, and exported types | # Chroma (/docs/reference/integrations/chroma) Import from `@anvia/chroma`. ## ChromaVectorStoreConnectOptions [#chromavectorstoreconnectoptions] ```ts type ChromaVectorStoreConnectOptions = { client?: ChromaClientLike; collectionName: string; createIfMissing?: boolean; metadata?: Record; configuration?: Record; }; ``` Purpose: connection options for a Chroma collection. Return behavior: consumed by `ChromaVectorStore.connect(...)`. Notable errors: missing collections reject when `createIfMissing` is `false`. Design note: `connect(...)` performs async collection lookup or creation before returning a store. This keeps constructors synchronous and side-effect free while making connection and configuration failures happen before ingestion or search. ## ChromaVectorStore [#chromavectorstore] ```ts class ChromaVectorStore { static connect( options: ChromaVectorStoreConnectOptions, ): Promise>; upsertDocuments(documents: Array>): Promise; index(model: EmbeddingModel): ChromaVectorIndex; } ``` Purpose: Chroma-backed document storage. Return behavior: `connect(...)` resolves a store; `index(...)` binds it to an embedding model. Notable errors: connection and upsert calls reject on Chroma errors; `upsertDocuments(...)` throws when a document has no embeddings. ## ChromaVectorIndex [#chromavectorindex] ```ts class ChromaVectorIndex implements VectorSearchIndex { search(request: VectorSearchRequest): Promise>>; searchIds(request: VectorSearchRequest): Promise>; asTool(options: VectorSearchToolOptions): Tool<{ query: string; topK?: number }, unknown>; } ``` Purpose: query-time Chroma search adapter. Return behavior: embeds the query, calls Chroma, deduplicates multi-embedding document IDs, and returns normalized results. Notable errors: embedding or Chroma query failures reject. ## filterToChromaWhere [#filtertochromawhere] ```ts function filterToChromaWhere(filter: VectorFilter | undefined): unknown; ``` Purpose: convert Anvia vector filters to Chroma `where` filters. Return behavior: returns `undefined` when no filter is supplied. Notable errors: none directly. # FastEmbed (/docs/reference/integrations/fastembed) Import from `@anvia/fastembed`. ## DEFAULT\_FASTEMBED\_EMBEDDING\_MODEL [#default_fastembed_embedding_model] ```ts const DEFAULT_FASTEMBED_EMBEDDING_MODEL = "fast-bge-small-en-v1.5"; ``` Purpose: default FastEmbed model id used by `FastEmbedEmbeddingModel.create(...)`. Return behavior: constant string. Notable errors: none directly. ## FastEmbed Types [#fastembed-types] ```ts type FastEmbedEmbeddingModelName = | "fast-all-MiniLM-L6-v2" | "fast-bge-base-en" | "fast-bge-base-en-v1.5" | "fast-bge-small-en" | "fast-bge-small-en-v1.5" | "fast-bge-small-zh-v1.5" | "fast-multilingual-e5-large"; type FastEmbedRuntime = { embed(texts: string[], batchSize?: number): AsyncIterable; }; type FastEmbedEmbeddingModelOptions = { model?: FastEmbedEmbeddingModelName; maxBatchSize?: number; initOptions?: { executionProviders?: ExecutionProvider[]; maxLength?: number; cacheDir?: string; showDownloadProgress?: boolean; modelName?: string; }; }; ``` Purpose: local FastEmbed model configuration and runtime contract. Return behavior: consumed by the embedding model. Notable errors: invalid runtime output causes embedding calls to throw. ## FastEmbedEmbeddingModel [#fastembedembeddingmodel] ```ts class FastEmbedEmbeddingModel implements EmbeddingModel { readonly model: string; readonly maxBatchSize: number; constructor(runtime: FastEmbedRuntime, options?: FastEmbedEmbeddingModelOptions); static create(options?: FastEmbedEmbeddingModelOptions): Promise; embedTexts(texts: string[]): Promise; } ``` Purpose: local FastEmbed adapter for Anvia embeddings. Return behavior: `create(...)` initializes FastEmbed; `embedTexts(...)` returns one embedding per text. Notable errors: rejects if model loading fails, runtime output shape is invalid, or the embedding count does not match input length. ## createFastEmbedEmbeddingModel [#createfastembedembeddingmodel] ```ts function createFastEmbedEmbeddingModel( options?: FastEmbedEmbeddingModelOptions, ): Promise; ``` Purpose: convenience wrapper around `FastEmbedEmbeddingModel.create(...)`. Return behavior: resolves a ready embedding model. Notable errors: same as `FastEmbedEmbeddingModel.create(...)`. # Integration Reference (/docs/reference/integrations) Integration packages connect Anvia core contracts to external systems. ## Packages [#packages] | Package | Purpose | | --------------------- | --------------------------------------------- | | `@anvia/chroma` | Vector store adapter for ChromaDB | | `@anvia/qdrant` | Vector store adapter for Qdrant | | `@anvia/pgvector` | Vector store adapter for Postgres pgvector | | `@anvia/fastembed` | Local FastEmbed embedding model adapter | | `@anvia/transformers` | Local Transformers.js embedding model adapter | | `@anvia/langfuse` | Langfuse tracing and scoring observer | | `@anvia/otel` | OpenTelemetry tracing observer | # Langfuse (/docs/reference/integrations/langfuse) Import from `@anvia/langfuse`. ## LangfuseTracingOptions [#langfusetracingoptions] ```ts type LangfuseTracingOptions = { publicKey?: string; secretKey?: string; baseUrl?: string; environment?: string; release?: string; }; ``` Purpose: configure Langfuse tracing and scoring. Return behavior: consumed by `langfuse.create(...)`. Notable errors: scoring requires both `publicKey` and `secretKey`. ## LangfuseTracing [#langfusetracing] ```ts type LangfuseTracing = AgentObserver & { flush(): Promise; shutdown(): Promise; score(args: LangfuseScoreArgs): Promise; }; ``` Purpose: Agent observer with Langfuse lifecycle and scoring methods. Return behavior: can be passed to `AgentBuilder.observe(...)`. Notable errors: `score(...)` throws without a trace id or credentials and rejects on Langfuse API failures. ## LangfuseScoreArgs [#langfusescoreargs] ```ts type LangfuseScoreArgs = { traceId?: string; observationId?: string; name: string; value: number; comment?: string; metadata?: Record; }; ``` Purpose: submit Langfuse scores for a trace or observation. Return behavior: consumed by `LangfuseTracing.score(...)`. Notable errors: score submission requires a trace id and Langfuse credentials. ## langfuse [#langfuse] ```ts const langfuse: { create(options?: LangfuseTracingOptions): LangfuseTracing; }; ``` Purpose: factory for Langfuse tracing observers. Return behavior: starts an OpenTelemetry SDK with a Langfuse span processor and returns an observer. Notable errors: construction or later flush/shutdown can fail through OpenTelemetry or Langfuse dependencies. ## Eval Reporter [#eval-reporter] ```ts type LangfuseEvalReporterOptions = { publishInvalid?: boolean; strict?: boolean; }; function createLangfuseEvalReporter( tracing: Pick, options?: LangfuseEvalReporterOptions, ): EvalReporter; ``` Purpose: bridge core eval metric results to Langfuse scores. Return behavior: returns an `EvalReporter` for `runEvalSuite(...)`. Invalid outcomes are skipped unless `publishInvalid` is true. Notable errors: with `strict: true`, missing trace ids reject the reporter call; scoring errors reject through the supplied tracing object. # OpenTelemetry (/docs/reference/integrations/otel) Import from `@anvia/otel`. ## OtelTracingOptions [#oteltracingoptions] ```ts type OtelTracingOptions = { tracer?: Tracer; tracerName?: string; tracerVersion?: string; serviceName?: string; }; ``` Purpose: configure the OpenTelemetry tracer used by the adapter. Return behavior: consumed by `otel.create(...)`. Notable behavior: when `tracer` is omitted, the adapter calls `trace.getTracer(tracerName ?? "@anvia/otel", tracerVersion)`. ## OtelTracing [#oteltracing] ```ts type OtelTracing = AgentObserver; ``` Purpose: Agent observer that emits OpenTelemetry spans. Return behavior: can be passed to `AgentBuilder.observe(...)`. Notable behavior: the adapter does not start, flush, or shut down an OpenTelemetry SDK. ## otel [#otel] ```ts const otel: { create(options?: OtelTracingOptions): OtelTracing; }; ``` Purpose: factory for OpenTelemetry tracing observers. Return behavior: creates an observer that emits root run spans, generation spans, and tool spans through the configured tracer. Notable behavior: if an Anvia trace contains a valid 32-character hex `traceId`, the root span is parented under a synthetic remote parent so emitted spans join that trace. # pgvector (/docs/reference/integrations/pgvector) Import from `@anvia/pgvector`. ## PgVectorStoreConnectOptions [#pgvectorstoreconnectoptions] ```ts type PgVectorDistance = "cosine" | "l2" | "innerProduct"; type PgVectorStoreConnectOptions = { client?: PgClientLike; connectionString?: string; tableName: string; vectorSize: number; createIfMissing?: boolean; distance?: PgVectorDistance; }; ``` Purpose: connection options for a Postgres table backed by the pgvector extension. Return behavior: consumed by `PgVectorStore.connect(...)`. Notable errors: missing tables reject when `createIfMissing` is `false`; table creation requires `vectorSize`; metadata keys starting with `__anvia_` are reserved. Design note: `connect(...)` performs async extension setup, table creation or validation, and vector dimension validation before returning a store. This keeps constructors synchronous and side-effect free while making connection and configuration failures happen before ingestion or search. ## PgVectorStore [#pgvectorstore] ```ts class PgVectorStore { static connect( options: PgVectorStoreConnectOptions, ): Promise>; upsertDocuments(documents: Array>): Promise; index(model: EmbeddingModel): PgVectorIndex; } ``` Purpose: Postgres pgvector-backed document storage. Return behavior: `connect(...)` resolves a store; `index(...)` binds it to an embedding model. Notable errors: connection and upsert calls reject on Postgres errors; `upsertDocuments(...)` throws when a document has no embeddings or metadata uses reserved `__anvia_*` keys. ## PgVectorIndex [#pgvectorindex] ```ts class PgVectorIndex implements VectorSearchIndex { search(request: VectorSearchRequest): Promise>>; searchIds(request: VectorSearchRequest): Promise>; asTool(options: VectorSearchToolOptions): Tool<{ query: string; topK?: number }, unknown>; } ``` Purpose: query-time pgvector search adapter. Return behavior: embeds the query, runs a parameterized vector search query, deduplicates multi-embedding document IDs, and returns normalized results. Notable errors: embedding or Postgres query failures reject. ## filterToPgVectorWhere [#filtertopgvectorwhere] ```ts function filterToPgVectorWhere( filter: VectorFilter | undefined, startIndex?: number, ): { sql: string; values: unknown[] } | undefined; ``` Purpose: convert Anvia vector filters to parameterized SQL over the `metadata jsonb` column. Return behavior: returns `undefined` when no filter is supplied. Notable errors: `gt` and `lt` filters require numeric metadata values. # Qdrant (/docs/reference/integrations/qdrant) Import from `@anvia/qdrant`. ## QdrantVectorStoreConnectOptions [#qdrantvectorstoreconnectoptions] ```ts type QdrantDistance = "Cosine" | "Dot" | "Euclid"; type QdrantVectorStoreConnectOptions = { client?: QdrantClientLike; collectionName: string; vectorSize: number; createIfMissing?: boolean; distance?: QdrantDistance; }; ``` Purpose: connection options for a Qdrant collection. Return behavior: consumed by `QdrantVectorStore.connect(...)`. Notable errors: missing collections reject when `createIfMissing` is `false`; collection creation requires `vectorSize`. Design note: `connect(...)` performs async collection lookup or creation before returning a store. This keeps constructors synchronous and side-effect free while making connection and configuration failures happen before ingestion or search. ## QdrantVectorStore [#qdrantvectorstore] ```ts class QdrantVectorStore { static connect( options: QdrantVectorStoreConnectOptions, ): Promise>; upsertDocuments(documents: Array>): Promise; index(model: EmbeddingModel): QdrantVectorIndex; } ``` Purpose: Qdrant-backed document storage. Return behavior: `connect(...)` resolves a store; `index(...)` binds it to an embedding model. Notable errors: connection and upsert calls reject on Qdrant errors; `upsertDocuments(...)` throws when a document has no embeddings or metadata uses reserved `__anvia_*` keys. ## QdrantVectorIndex [#qdrantvectorindex] ```ts class QdrantVectorIndex implements VectorSearchIndex { search(request: VectorSearchRequest): Promise>>; searchIds(request: VectorSearchRequest): Promise>; asTool(options: VectorSearchToolOptions): Tool<{ query: string; topK?: number }, unknown>; } ``` Purpose: query-time Qdrant search adapter. Return behavior: embeds the query, calls Qdrant, deduplicates multi-embedding document IDs, and returns normalized results. Notable errors: embedding or Qdrant query failures reject. ## filterToQdrantFilter [#filtertoqdrantfilter] ```ts function filterToQdrantFilter(filter: VectorFilter | undefined): unknown; ``` Purpose: convert Anvia vector filters to Qdrant payload filters. Return behavior: returns `undefined` when no filter is supplied. Notable errors: none directly. # Transformers (/docs/reference/integrations/transformers) Import from `@anvia/transformers`. ## DEFAULT\_TRANSFORMERS\_EMBEDDING\_MODEL [#default_transformers_embedding_model] ```ts const DEFAULT_TRANSFORMERS_EMBEDDING_MODEL = "Xenova/all-MiniLM-L6-v2"; ``` Purpose: default Hugging Face model id used by `TransformersEmbeddingModel.create(...)`. Return behavior: constant string. Notable errors: none directly. ## Transformers Types [#transformers-types] ```ts type TransformersPooling = "mean" | "cls"; type TransformersFeatureExtractionPipeline = ( texts: string[], options: { pooling: TransformersPooling; normalize: boolean }, ) => Promise<{ tolist(): unknown }>; type TransformersEmbeddingModelOptions = { model?: string; pooling?: TransformersPooling; normalize?: boolean; maxBatchSize?: number; }; ``` Purpose: local embedding model configuration and pipeline contract. Return behavior: consumed by the embedding model. Notable errors: invalid pipeline output causes embedding calls to throw. ## TransformersEmbeddingModel [#transformersembeddingmodel] ```ts class TransformersEmbeddingModel implements EmbeddingModel { readonly model: string; readonly maxBatchSize: number; constructor( extractor: TransformersFeatureExtractionPipeline, options?: TransformersEmbeddingModelOptions, ); static create(options?: TransformersEmbeddingModelOptions): Promise; embedTexts(texts: string[]): Promise; } ``` Purpose: local Transformers.js feature-extraction adapter. Return behavior: `create(...)` loads the feature-extraction pipeline; `embedTexts(...)` returns one embedding per text. Notable errors: rejects if model loading fails, the extractor output shape is invalid, or the embedding count does not match input length. ## createTransformersEmbeddingModel [#createtransformersembeddingmodel] ```ts function createTransformersEmbeddingModel( options?: TransformersEmbeddingModelOptions, ): Promise; ``` Purpose: convenience wrapper around `TransformersEmbeddingModel.create(...)`. Return behavior: resolves a ready embedding model. Notable errors: same as `TransformersEmbeddingModel.create(...)`. # Anthropic Provider (/docs/reference/providers/anthropic) Import from `@anvia/anthropic`. ## AnthropicClient [#anthropicclient] ```ts type AnthropicClientOptions = { apiKey?: string; baseUrl?: string; client?: Anthropic; }; class AnthropicClient { readonly client: Anthropic; constructor(options?: AnthropicClientOptions); listModels(): Promise; completionModel(model?: string): AnthropicCompletionModel; } ``` Purpose: factory for Anthropic completion models and model listing. Return behavior: `completionModel(...)` returns a streaming Anvia completion model. `listModels()` fetches Anthropic's model list and returns a normalized `ModelList`. Notable errors: constructor throws when neither `client` nor `apiKey` is supplied; `listModels()` rejects with `ModelListingError` when the provider request fails. ## AnthropicCompletionModel [#anthropiccompletionmodel] ```ts class AnthropicCompletionModel implements StreamingCompletionModel { constructor(client: Anthropic, defaultModel?: string); completion(request: CompletionRequest): Promise; streamCompletion(request: CompletionRequest): AsyncIterable; } ``` Purpose: adapter for Anthropic Messages API. Return behavior: maps Anvia completion requests to Anthropic params and returns normalized responses/events. Notable errors: rejects or yields SDK/provider errors. ## Helper Namespaces [#helper-namespaces] ```ts namespace anthropic { AnthropicClient; AnthropicCompletionModel; } ``` Purpose: namespaced access to the same public model/client exports. Return behavior: export namespaces only. Notable errors: none directly. # Gemini Provider (/docs/reference/providers/gemini) Import from `@anvia/gemini`. ## GeminiClient [#geminiclient] ```ts type GeminiClientOptions = | { apiKey?: string; vertexai?: false; client?: GoogleGenAI } | { vertexai: true; project?: string; location?: string; client?: GoogleGenAI }; class GeminiClient { readonly client: GoogleGenAI; constructor(options?: GeminiClientOptions); listModels(): Promise; completionModel(model?: string): GeminiCompletionModel; embeddingModel(model?: string, options?: GeminiEmbeddingModelOptions): GeminiEmbeddingModel; imageGenerationModel(model?: string): GeminiImageGenerationModel; imagenGenerationModel(model?: string): GeminiImagenGenerationModel; transcriptionModel(model?: string): GeminiTranscriptionModel; } ``` Purpose: factory for Gemini API or Vertex AI-backed completion, embedding, image generation, transcription, and model listing. Return behavior: creates or uses a `GoogleGenAI` client, then returns Gemini completion and embedding models. `listModels()` fetches the Gemini model list and returns a normalized `ModelList`. Notable errors: underlying SDK calls can fail for missing credentials, invalid project/location, or API errors; `listModels()` rejects with `ModelListingError` when the provider request fails. ## Multimodal Models [#multimodal-models] ```ts class GeminiImageGenerationModel implements ImageGenerationModel {} class GeminiImagenGenerationModel implements ImageGenerationModel {} class GeminiTranscriptionModel implements TranscriptionModel {} ``` Purpose: Gemini adapters for `@anvia/core` image generation and transcription. Return behavior: `imageGenerationModel()` uses Gemini native image generation through `generateContent` and maps inline image parts to `Uint8Array`. `imagenGenerationModel()` uses Imagen through `generateImages` and maps `generatedImages[].image.imageBytes` to `Uint8Array`. Transcription sends inline audio through `generateContent` and returns normalized text. Notable errors: rejects on Gemini SDK errors or when image/transcription responses do not contain expected content. Model constants: `GEMINI_2_5_FLASH_IMAGE`, `GEMINI_3_PRO_IMAGE_PREVIEW`, and `IMAGEN_4_GENERATE`. Gemini audio generation is not implemented in v1. ## GeminiCompletionModel [#geminicompletionmodel] ```ts class GeminiCompletionModel implements StreamingCompletionModel { constructor(client: GoogleGenAI, defaultModel?: string); completion(request: CompletionRequest): Promise; streamCompletion(request: CompletionRequest): AsyncIterable; } ``` Purpose: adapter for Gemini content generation. Return behavior: returns normalized completion responses and stream events. Notable errors: rejects or yields SDK/provider errors. ## GeminiEmbeddingModel [#geminiembeddingmodel] ```ts type GeminiEmbeddingTaskType = | "TASK_TYPE_UNSPECIFIED" | "RETRIEVAL_QUERY" | "RETRIEVAL_DOCUMENT" | "SEMANTIC_SIMILARITY" | "CLASSIFICATION" | "CLUSTERING" | "QUESTION_ANSWERING" | "FACT_VERIFICATION" | "CODE_RETRIEVAL_QUERY"; type GeminiEmbeddingModelOptions = { dimensions?: number; maxBatchSize?: number; taskType?: GeminiEmbeddingTaskType; title?: string; }; class GeminiEmbeddingModel implements EmbeddingModel { readonly dimensions?: number; readonly maxBatchSize: number; constructor(client: GoogleGenAI, model: string, options?: GeminiEmbeddingModelOptions); embedTexts(texts: string[]): Promise; } ``` Purpose: Gemini embedding adapter. Return behavior: returns one `Embedding` per input text. Notable errors: rejects on SDK errors or mismatched embedding counts. ## gemini Namespace [#gemini-namespace] ```ts namespace gemini { GeminiClient; GeminiCompletionModel; GeminiEmbeddingModel; GeminiImageGenerationModel; GeminiImagenGenerationModel; GeminiTranscriptionModel; } ``` Purpose: namespaced access to the same public Gemini exports. Return behavior: export namespace only. Notable errors: none directly. # Provider Reference (/docs/reference/providers) Provider packages adapt external model SDKs into Anvia completion and embedding interfaces. ## Packages [#packages] | Package | Main exports | | ------------------ | ------------------------------------------------------------------------- | | `@anvia/openai` | `OpenAIClient`, OpenAI Responses/chat completion models, embedding models | | `@anvia/anthropic` | `AnthropicClient`, Anthropic messages model | | `@anvia/gemini` | `GeminiClient`, Gemini completion and embedding models | | `@anvia/mistral` | `MistralClient`, Mistral completion and embedding models | Provider clients only use explicit constructor values. They do not read environment variables and do not expose `.fromEnv()`. For model-focused usage, see [Models](/docs/models). # Mistral Provider (/docs/reference/providers/mistral) Import from `@anvia/mistral`. ## MistralClient [#mistralclient] ```ts type MistralClientOptions = { apiKey?: string; serverURL?: string; client?: Mistral; }; class MistralClient { readonly client: Mistral; constructor(options?: MistralClientOptions); listModels(): Promise; completionModel(model?: string): MistralCompletionModel; embeddingModel(model?: string, options?: MistralEmbeddingModelOptions): MistralEmbeddingModel; } ``` Purpose: factory for Mistral completion, embedding, and model listing. Return behavior: creates or uses a Mistral SDK client, then returns normalized Anvia model adapters. `listModels()` fetches Mistral's model list and returns a normalized `ModelList`. Notable errors: constructor throws when neither `client` nor `apiKey` is supplied; `listModels()` rejects with `ModelListingError` when the provider request fails. ## MistralCompletionModel [#mistralcompletionmodel] ```ts class MistralCompletionModel implements StreamingCompletionModel { constructor(client: Mistral, defaultModel?: string); completion(request: CompletionRequest): Promise; streamCompletion(request: CompletionRequest): AsyncIterable; } ``` Purpose: adapter for Mistral chat completions. Return behavior: returns normalized completion responses and stream events. Notable errors: rejects unsupported image inputs and document file inputs before provider calls; rejects or yields Mistral SDK/provider errors for transport failures. ## MistralEmbeddingModel [#mistralembeddingmodel] ```ts type MistralEmbeddingModelOptions = { dimensions?: number; maxBatchSize?: number; }; class MistralEmbeddingModel implements EmbeddingModel { readonly dimensions?: number; readonly maxBatchSize: number; constructor(client: Mistral, model: string, options?: MistralEmbeddingModelOptions); embedTexts(texts: string[]): Promise; } ``` Purpose: Mistral embedding adapter. Return behavior: returns one `Embedding` per input text. Notable errors: rejects on Mistral SDK errors or mismatched embedding counts. ## mistral Namespace [#mistral-namespace] ```ts namespace mistral { MistralClient; MistralCompletionModel; MistralEmbeddingModel; toMistralChatParams; fromMistralChatResponse; fromMistralChatStreamChunk; mistralMessageHelpers; } ``` Purpose: namespaced access to the same public Mistral exports. Return behavior: export namespace only. Notable errors: none directly. ## Mapping Helpers [#mapping-helpers] ```ts function toMistralChatParams(defaultModel: string, request: CompletionRequest): unknown; function fromMistralChatResponse(response: unknown): CompletionResponse; function fromMistralChatStreamChunk(chunk: unknown): CompletionStreamEvent[]; const mistralMessageHelpers: { messageToMistralMessages(message: Message): unknown[]; toolDefinitionToMistral(tool: ToolDefinition): unknown; }; ``` Purpose: low-level request and response mappers used by `MistralCompletionModel`. Return behavior: exported for tests, custom adapters, and compatibility layers that need the same normalized mapping. Notable errors: malformed provider payloads can produce empty normalized content or throw while parsing tool arguments. # OpenAI Provider (/docs/reference/providers/openai) Import from `@anvia/openai`. ## OpenAIClient [#openaiclient] ```ts type OpenAIClientOptions = { apiKey?: string; baseUrl?: string; headers?: Record; completionApi?: "responses" | "chat"; client?: OpenAI; }; class OpenAIClient { readonly client: OpenAI; constructor(options?: OpenAIClientOptions); listModels(): Promise; completionModel(model?: string): StreamingCompletionModel; embeddingModel(model?: string, options?: ProviderEmbeddingModelOptions): OpenAIEmbeddingModel; imageGenerationModel(model?: string): OpenAIImageGenerationModel; audioGenerationModel(model?: string): OpenAIAudioGenerationModel; transcriptionModel(model?: string): OpenAITranscriptionModel; } ``` Purpose: factory for OpenAI completion, embedding, image generation, audio generation, transcription, and model listing. Return behavior: `completionModel(...)` returns a streaming model backed by Responses API by default or chat completions when `completionApi: "chat"` is set. `listModels()` fetches the configured OpenAI or OpenAI-compatible `/models` endpoint and returns a normalized `ModelList`. Notable errors: constructor throws when neither `client` nor `apiKey` is supplied; `listModels()` rejects with `ModelListingError` when the provider request fails. ## Multimodal Models [#multimodal-models] ```ts class OpenAIImageGenerationModel implements ImageGenerationModel {} class OpenAIAudioGenerationModel implements AudioGenerationModel {} class OpenAITranscriptionModel implements TranscriptionModel {} ``` Purpose: OpenAI adapters for `@anvia/core` image generation, text-to-speech, and transcription. Return behavior: image generation maps base64 image outputs to `Uint8Array`; audio generation maps speech responses to `Uint8Array`; transcription returns normalized text. Notable errors: rejects on OpenAI SDK errors or when image/transcription responses do not contain expected content. Model constants: `GPT_IMAGE_1`, `GPT_IMAGE_2`, `DALL_E_2`, `DALL_E_3`, `TTS_1`, `TTS_1_HD`, and `WHISPER_1`. ## OpenAIEmbeddingModel [#openaiembeddingmodel] ```ts type ProviderEmbeddingModelOptions = { dimensions?: number; user?: string; maxBatchSize?: number; }; class OpenAIEmbeddingModel implements EmbeddingModel { readonly dimensions?: number; readonly maxBatchSize: number; constructor(client: OpenAI, model: string, options?: ProviderEmbeddingModelOptions); embedTexts(texts: string[]): Promise; } ``` Purpose: OpenAI embedding adapter. Return behavior: returns one `Embedding` per input text. Notable errors: rejects on OpenAI SDK errors or mismatched embedding counts. ## OpenAIResponsesCompletionModel [#openairesponsescompletionmodel] ```ts class OpenAIResponsesCompletionModel implements StreamingCompletionModel { constructor(client: OpenAI, defaultModel?: string); completion(request: CompletionRequest): Promise; streamCompletion(request: CompletionRequest): AsyncIterable; } ``` Purpose: completion adapter for OpenAI Responses API. Return behavior: non-streaming calls return normalized `CompletionResponse`; streaming calls yield normalized completion events. Notable errors: rejects or yields errors from OpenAI Responses API calls. ## OpenAIChatCompletionModel [#openaichatcompletionmodel] ```ts class OpenAIChatCompletionModel implements StreamingCompletionModel { constructor(client: OpenAI, defaultModel?: string); completion(request: CompletionRequest): Promise; streamCompletion(request: CompletionRequest): AsyncIterable; } ``` Purpose: chat-completions adapter used by `OpenAIClient` when `completionApi: "chat"` is set or a custom `baseUrl` is provided. Return behavior: maps Anvia requests to chat completion params and maps responses back to Anvia content. Notable errors: rejects or yields provider SDK errors. ## Helper Namespaces [#helper-namespaces] ```ts namespace openai { OpenAIClient; OpenAIEmbeddingModel; OpenAIImageGenerationModel; OpenAIAudioGenerationModel; OpenAITranscriptionModel; OpenAIResponsesCompletionModel; OpenAIChatCompletionModel; } ``` Purpose: namespaced access to the same public model/client exports. Return behavior: export namespaces only. Notable errors: none directly. # Studio Approvals (/docs/reference/studio/approvals) Import from `@anvia/studio`. ## Approval Types [#approval-types] ```ts type StudioToolApprovalDecision = { approved: boolean; reason?: string; }; type StudioToolApprovalStatus = "pending" | "approved" | "rejected" | "timed_out"; ``` Purpose: Studio-owned approval contracts for HTTP decisions, records, and UI state. Return behavior: type-only exports. Notable errors: none directly. ## StudioToolApproval [#studiotoolapproval] ```ts type StudioToolApproval = { id: string; runId: string; agentId: string; sessionId?: string; toolName: string; callId?: string; internalCallId: string; args: string; status: StudioToolApprovalStatus; requestedAt: string; resolvedAt?: string; reason?: string; }; ``` Purpose: persisted approval request/result record for Studio. Return behavior: returned by approval routes and stream events. Notable errors: none directly. ## Transcript and Stream Events [#transcript-and-stream-events] ```ts type StudioToolApprovalTranscript = { id: string; status: StudioToolApprovalStatus; requestedAt: string; resolvedAt?: string; reason?: string; }; type StudioToolApprovalRequestEvent = { type: "tool_approval_request"; approval: StudioToolApproval; }; type StudioToolApprovalResultEvent = { type: "tool_approval_result"; approval: StudioToolApproval; }; ``` Purpose: represent approval state in transcripts and run streams. Return behavior: events are included in `AgentRunStreamEvent`. Notable errors: invalid approval decisions return Studio HTTP errors from approval routes. ## Question Types [#question-types] ```ts type StudioToolQuestionChoice = { label: string; value: string; }; type StudioToolQuestionPrompt = { id: string; question: string; choices: StudioToolQuestionChoice[]; }; type StudioToolQuestionAnswer = { questionId: string; answer: string; choice?: string; custom?: boolean; }; type StudioToolQuestionStatus = "pending" | "answered"; ``` Purpose: represent a human question requested by a tool such as `ask_question`. Return behavior: type-only exports used by Studio question records, transcripts, and stream events. Notable errors: invalid answers return Studio HTTP errors from question routes. ## StudioToolQuestion [#studiotoolquestion] ```ts type StudioToolQuestion = { id: string; runId: string; agentId: string; sessionId?: string; toolName: string; callId?: string; internalCallId: string; args: string; questions: StudioToolQuestionPrompt[]; status: StudioToolQuestionStatus; requestedAt: string; answeredAt?: string; answers?: StudioToolQuestionAnswer[]; }; ``` Purpose: persisted question request/result record for Studio. Return behavior: returned by question routes and stream events. Notable errors: none directly. ## Question Transcript and Stream Events [#question-transcript-and-stream-events] ```ts type StudioToolQuestionTranscript = { id: string; status: StudioToolQuestionStatus; requestedAt: string; answeredAt?: string; questions: StudioToolQuestionPrompt[]; answers?: StudioToolQuestionAnswer[]; }; type StudioToolQuestionRequestEvent = { type: "tool_question_request"; question: StudioToolQuestion; }; type StudioToolQuestionResultEvent = { type: "tool_question_result"; question: StudioToolQuestion; }; ``` Purpose: represent pending and answered human questions in transcripts and run streams. Return behavior: events are included in `AgentRunStreamEvent`. Notable errors: unanswered questions keep the run waiting until a Studio answer route resolves them. # Studio Reference (/docs/reference/studio) `@anvia/studio` exports the local inspection runtime, HTTP contracts, session and trace stores, approval events, and Studio trace observer. ## Public Imports [#public-imports] ```ts import { Studio, createSqliteSessionStore, StudioTraceObserver } from "@anvia/studio"; import type { StudioConfig, StudioSessionStore, StudioTraceStore } from "@anvia/studio"; ``` For setup and usage, start with [Run Studio](/docs/studio/run-studio). # Studio Runtime (/docs/reference/studio/runtime) Import from `@anvia/studio`. ## Studio [#studio] ```ts class Studio implements AnviaStudio { constructor(targets?: StudioTarget[], options?: StudioOptions); get app(): Hono; fetch(request: Request): Response | Promise; config(): StudioConfig; traceObserver(): StudioTraceObserver; start(serveOptions?: StudioServeOptions): this; close(): void; } ``` Purpose: local Studio HTTP runtime and UI/API host. Return behavior: pass built agents, built pipelines, or both in the first array. `start(...)` starts an HTTP server and returns `this`; `fetch(...)` delegates to the Hono app. Notable errors: server startup can fail when the port is unavailable; route handlers return structured `StudioErrorResponse` values for request errors. ## AnviaStudio [#anviastudio] ```ts type AnviaStudio = { readonly app: Hono; fetch(request: Request): Response | Promise; config(): StudioConfig; close(): void; }; ``` Purpose: minimal Studio runtime interface. Return behavior: implemented by `Studio`. Notable errors: none directly. ## Options [#options] ```ts type StudioOptions = { quickPrompts?: Record; }; type StudioTarget = Agent | Pipeline; type StudioServeOptions = { port?: number; hostname?: string; log?: boolean; }; type StudioUiOptions = { path?: string; rootRoutes?: boolean; title?: string; redirectRoot?: boolean; clientScript?: string; protectShell?: boolean; }; ``` Purpose: configure Studio agents, server binding, and UI mounting. Return behavior: options are constructor or start inputs. Notable errors: invalid server options can fail at the Hono server layer. ## Run Contracts [#run-contracts] Studio run request, response, and stream event contracts are documented in [Studio Types](/docs/reference/studio/types#run-types). Purpose: route-level runtime behavior for Studio agent runs. Return behavior: non-streaming runs return `PromptResponse`; streaming runs emit newline-delimited Studio run events. Notable errors: invalid request bodies return `StudioErrorResponse` with `bad_request`. # Studio Sessions (/docs/reference/studio/sessions) Import from `@anvia/studio`. ## Transcript Entries [#transcript-entries] ```ts type StudioTranscriptChatEntry = { entryId: number; kind: "message"; role: "user" | "assistant"; text: string; traceId?: string; }; type StudioTranscriptReasoningEntry = { entryId: number; kind: "reasoning"; reasoningId?: string; text: string; }; type StudioTranscriptToolEntry = { entryId: number; kind: "tool"; toolName: string; callId?: string; args?: string; result?: string; childEvents?: StudioTranscriptChildAgentEvent[]; approval?: StudioToolApprovalTranscript; question?: StudioToolQuestionTranscript; }; type StudioTranscriptChildAgentEvent = | { kind: "message"; agentId: string; agentName?: string; text: string; } | { kind: "reasoning"; agentId: string; agentName?: string; reasoningId?: string; text: string; } | { kind: "tool"; agentId: string; agentName?: string; toolName: string; callId?: string; args?: string; result?: string; }; type StudioTranscriptEntry = | StudioTranscriptChatEntry | StudioTranscriptReasoningEntry | StudioTranscriptToolEntry; ``` Purpose: UI-friendly transcript model derived from messages and stream events. Return behavior: persisted inside `StudioSession.transcript`. Notable errors: none directly. ## Session Types [#session-types] ```ts type StudioSessionSummary = { id: string; agentId: string; title?: string; createdAt: string; updatedAt: string; messageCount: number; metadata?: JsonObject; }; type StudioSession = StudioSessionSummary & { messages: Message[]; transcript: StudioTranscriptEntry[]; }; ``` Purpose: session list and detail contracts. Return behavior: stores return summaries for lists and full sessions for detail. Notable errors: none directly. ## Session Store Inputs [#session-store-inputs] ```ts type StudioSessionCreateInput = { id: string; agentId: string; title?: string; metadata?: JsonObject; }; type StudioSessionListOptions = { agentId?: string; limit: number; }; type StudioSessionRunStatus = "running" | "success" | "error"; type StudioSessionRunTranscriptInput = { id: string; runId: string; title?: string; transcript: StudioTranscriptEntry[]; status: StudioSessionRunStatus; error?: JsonValue; }; type StudioSessionLogLevel = "debug" | "info" | "warn" | "error"; type StudioSessionLogCategory = | "session" | "run" | "memory" | "prompt" | "model" | "tool" | "approval" | "question" | "api"; type StudioSessionLogEntry = { id: string; sessionId: string; runId?: string; sequence: number; timestamp: string; level: StudioSessionLogLevel; category: StudioSessionLogCategory; event: string; message: string; metadata?: JsonObject; }; type StudioSessionLogAppendInput = { sessionId: string; runId?: string; level: StudioSessionLogLevel; category: StudioSessionLogCategory; event: string; message: string; metadata?: JsonObject; }; type StudioSessionLogListOptions = { sessionId: string; limit: number; after?: number; }; type StudioSessionLogEvent = { type: "session_log"; log: StudioSessionLogEntry; }; ``` Purpose: arguments for session store methods. Return behavior: used by `StudioSessionStore` and streaming session log events. Notable errors: store implementations may reject invalid or conflicting inputs. ## StudioSessionStore [#studiosessionstore] ```ts type StudioSessionStore = MemoryStore & { readonly kind?: string; listSessions(options: StudioSessionListOptions): StudioSessionSummary[] | Promise; createSession(input: StudioSessionCreateInput): StudioSessionSummary | Promise; getSession(id: string): StudioSession | undefined | Promise; saveSessionRunTranscript( input: StudioSessionRunTranscriptInput, ): StudioSession | undefined | Promise; appendSessionLog?(input: StudioSessionLogAppendInput): StudioSessionLogEntry | Promise; listSessionLogs?(options: StudioSessionLogListOptions): StudioSessionLogEntry[] | Promise; deleteSession?(id: string): boolean | Promise; }; ``` Purpose: persistence adapter for Studio sessions. Return behavior: methods may be sync or async. Because the store extends `MemoryStore`, it also loads, appends, and clears model messages for Studio-backed sessions. Log methods are optional for custom stores; the default SQLite store implements them. Notable errors: persistence failures should throw or reject. # Studio Stores (/docs/reference/studio/stores) Import from `@anvia/studio`. ## Store Bundle Types [#store-bundle-types] ```ts type StudioStores = { sessions?: StudioSessionStore | false; traces?: StudioTraceStore; pipelineLogs?: StudioPipelineLogStore | false; pipelineRuns?: StudioPipelineRunStore | false; }; ``` Purpose: optional persistence stores for Studio sessions, traces, pipeline logs, and pipeline run history. Return behavior: `false` disables sessions, pipeline logs, or pipeline runs. Omitted traces disable trace routes unless the session store also implements `StudioTraceStore`. The default SQLite store implements session, trace, pipeline log, and pipeline run storage. Notable errors: none directly. ## SQLite Store [#sqlite-store] ```ts type SqliteSessionStoreOptions = { path?: string; }; function createSqliteSessionStore( options?: SqliteSessionStoreOptions, ): StudioSessionStore & StudioTraceStore & StudioPipelineLogStore & StudioPipelineRunStore; ``` Purpose: create a SQLite-backed session, trace, pipeline log, and pipeline run store. Return behavior: defaults to in-memory SQLite when `path` is omitted. Notable errors: throws when `node:sqlite` is unavailable, the database cannot open, or SQLite operations fail. # Studio Traces (/docs/reference/studio/traces) Import from `@anvia/studio`. ## Trace Types [#trace-types] ```ts type StudioTraceStatus = "running" | "success" | "error"; type StudioTraceObservationKind = "generation" | "tool"; type StudioTraceObservation = { id: string; kind: StudioTraceObservationKind; name: string; status: StudioTraceStatus; turn: number; startedAt: string; endedAt?: string; durationMs?: number; input?: JsonValue; output?: JsonValue; error?: JsonValue; metadata?: JsonObject; }; type StudioTraceSummary = { id: string; sessionId: string; name?: string; status: StudioTraceStatus; startedAt: string; endedAt?: string; durationMs?: number; output?: string; error?: JsonValue; usage?: Usage; metadata?: JsonObject; observationCount: number; }; type StudioTrace = StudioTraceSummary & { trace?: AgentTraceInfo; input?: JsonValue; observations: StudioTraceObservation[]; }; ``` Purpose: Studio-native trace persistence and UI contracts. Return behavior: trace stores return summaries for lists and full traces for detail. Notable errors: none directly. ## Trace Store Types [#trace-store-types] ```ts type StudioTraceListOptions = { limit: number; agentId?: string; sessionId?: string; status?: StudioTraceStatus; }; type StudioSessionTraceListOptions = { sessionId: string; limit: number; }; type StudioTraceStore = { readonly kind?: string; listTraces?(options: StudioTraceListOptions): StudioTraceSummary[] | Promise; listSessionTraces(options: StudioSessionTraceListOptions): StudioTraceSummary[] | Promise; getTrace(id: string): StudioTrace | undefined | Promise; saveTrace(trace: StudioTrace): StudioTrace | Promise; }; ``` Purpose: persistence adapter for Studio traces. Return behavior: methods may be sync or async. Notable errors: persistence failures should throw or reject. ## StudioTraceObserver [#studiotraceobserver] ```ts type StudioTraceObserverOptions = { store: StudioTraceStore | (() => StudioTraceStore | undefined) | undefined; }; class StudioTraceObserver implements AgentObserver { constructor(options: StudioTraceObserverOptions); startRun(args: AgentRunStartArgs): AgentRunObserver; } ``` Purpose: converts agent observer events into `StudioTrace` records. Return behavior: `startRun(...)` returns an `AgentRunObserver`; traces are saved when the run ends or errors. Notable errors: save failures reject observer lifecycle methods and can fail the run when observer errors are configured to fail. # Studio Types (/docs/reference/studio/types) Import from `@anvia/studio`. ## Config Types [#config-types] ```ts type StudioCapability = | "agents" | "approvals" | "knowledge" | "mcps" | "observability" | "pipelines" | "sessions" | "tools" | "traces"; type StudioAgent = { id: string; agent: Agent; name?: string; description?: string; quickPrompts?: string[]; metadata?: JsonObject; }; type StudioAgentConfig = { id: string; name?: string; description?: string; quickPrompts: string[]; metadata?: JsonObject; }; type StudioTarget = Agent | Pipeline; type StudioPipeline = { id: string; pipeline: Pipeline; name?: string; description?: string; metadata?: JsonObject; }; type StudioPipelineConfig = { id: string; name?: string; description?: string; metadata?: JsonObject; stageCount: number; edgeCount: number; hasParallelStages: boolean; agentCount: number; extractorCount: number; }; type StudioPipelineDetail = StudioPipelineConfig & { graph: PipelineGraph; }; type StudioCapabilityConfig = { enabled: boolean; reason?: string; }; type StudioConfig = { id: string; name?: string; description?: string; version?: string; agents: StudioAgentConfig[]; pipelines: StudioPipelineConfig[]; chat: { quickPrompts: Record }; capabilities: Partial>; unsupportedCapabilities: StudioCapability[]; }; ``` Purpose: configuration returned by Studio runtime and HTTP config endpoints. Return behavior: `Studio.config()` returns `StudioConfig`. Pipelines are top-level Studio targets beside agents. Notable errors: none directly. ## Tool Metadata Types [#tool-metadata-types] ```ts type StudioAgentToolSource = "static" | "dynamic"; type StudioAgentToolApprovalMetadata = { required: boolean; reason?: string; rejectMessage?: string; }; type StudioAgentToolMetadata = { agentId: string; name: string; description: string; parameters: JsonObject; source: StudioAgentToolSource; approval: StudioAgentToolApprovalMetadata; }; type StudioAgentToolsSummary = { agentId: string; tools: StudioAgentToolMetadata[]; }; type StudioAgentMcpToolMetadata = { name: string; description: string; parameters: JsonObject; source: StudioAgentToolSource; }; type StudioAgentMcpServerMetadata = { agentId: string; name: string; toolCount: number; tools: StudioAgentMcpToolMetadata[]; }; type StudioAgentMcpsSummary = { agentId: string; servers: StudioAgentMcpServerMetadata[]; }; ``` Purpose: metadata returned by `GET /agents/:agentId/tools` and used by the Studio Tools inspector. Return behavior: static tools come from `agent.toolSet`; dynamic tools are listed when the dynamic tool index exposes its underlying `ToolSet`. Notable errors: unknown agents return `not_found`. `GET /agents/:agentId/mcps` returns the MCP subset grouped by server name. MCP metadata is available for tools registered through `.mcp(...)`. ## Run Types [#run-types] ```ts type AgentRunRequest = { message: string | Message; history?: Message[]; sessionId?: string; stream?: boolean; maxTurns?: number; toolConcurrency?: number; metadata?: JsonObject; trace?: AgentTraceOptions; }; type AgentRunResponse = PromptResponse; type AgentRunStreamEvent = | AgentStreamEvent | StudioToolApprovalRequestEvent | StudioToolApprovalResultEvent | StudioToolQuestionRequestEvent | StudioToolQuestionResultEvent | StudioSessionLogEvent | StudioPipelineLogEvent | StudioPipelineFinalEvent; ``` Purpose: HTTP request and response contracts for Studio agent runs. Return behavior: non-streaming agent runs return `AgentRunResponse`; streaming agent runs emit newline-delimited `AgentRunStreamEvent` values. `session_log` and `pipeline_log` events are Studio-owned metadata-only runtime logs. Notable errors: invalid run request bodies return `bad_request`; unsupported stores or capabilities return `unsupported_capability`. ## Pipeline Run Types [#pipeline-run-types] ```ts type StudioPipelineLogLevel = "debug" | "info" | "warn" | "error"; type StudioPipelineLogCategory = | "pipeline" | "run" | "stage" | "parallel" | "agent" | "extractor" | "api"; type StudioPipelineLogEntry = { id: string; pipelineId: string; runId?: string; sequence: number; timestamp: string; level: StudioPipelineLogLevel; category: StudioPipelineLogCategory; event: string; message: string; metadata?: JsonObject; }; type StudioPipelineLogAppendInput = { pipelineId: string; runId?: string; level: StudioPipelineLogLevel; category: StudioPipelineLogCategory; event: string; message: string; metadata?: JsonObject; }; type StudioPipelineLogListOptions = { pipelineId: string; limit: number; after?: number; }; type StudioPipelineLogEvent = { type: "pipeline_log"; log: StudioPipelineLogEntry; }; type StudioPipelineFinalEvent = { type: "pipeline_final"; runId: string; pipelineId: string; output: JsonValue; }; type StudioPipelineRunRequest = { input: JsonValue; stream?: boolean; metadata?: JsonObject; }; type StudioPipelineRunResponse = { runId: string; pipelineId: string; output: JsonValue; }; ``` Purpose: HTTP request, response, stream, and persisted log contracts for Studio pipeline runs. Return behavior: non-streaming pipeline runs return `StudioPipelineRunResponse`; streaming runs emit `pipeline_log` events and one `pipeline_final` event. Notable errors: Studio HTTP pipeline inputs must be JSON-compatible. Use direct `pipeline.run(...)` for non-JSON inputs. ## Knowledge Types [#knowledge-types] ```ts type StudioKnowledgeSourceKind = "static_context" | "dynamic_context" | "dynamic_tools"; type StudioKnowledgeSourceSummary = { kind: StudioKnowledgeSourceKind; count: number; }; type StudioStaticKnowledgeDocument = { id: string; text: string; additionalProps?: JsonObject; }; type StudioKnowledgeEvidenceDocument = { id?: string; text?: string; additionalProps?: JsonObject; }; type StudioKnowledgeEvidence = { traceId: string; sessionId: string; observationId: string; observationName: string; turn: number; startedAt: string; query?: string; documentCount: number; toolCount: number; documents: StudioKnowledgeEvidenceDocument[]; tools: string[]; }; type StudioAgentKnowledgeConfig = { agentId: string; agentName?: string; sources: StudioKnowledgeSourceSummary[]; staticContext: StudioStaticKnowledgeDocument[]; }; type StudioKnowledgeItemKind = "static_context" | "dynamic_context" | "dynamic_tool"; type StudioKnowledgeItem = { id: string; kind: StudioKnowledgeItemKind; text?: string; document?: JsonValue; toolName?: string; description?: string; parameterKeys?: string[]; metadata?: JsonObject; }; type StudioKnowledgeItemsPage = { agentId: string; sourceId: string; kind: StudioKnowledgeSourceKind; inspectable: boolean; items: StudioKnowledgeItem[]; nextCursor?: string; totalCount?: number; message?: string; }; type StudioKnowledgeSummary = { agents: StudioAgentKnowledgeConfig[]; evidence: StudioKnowledgeEvidence[]; }; ``` Purpose: summarize each agent's inspectable static context, dynamic context, dynamic tools, and recent trace evidence. Return behavior: `GET /knowledge` returns `StudioKnowledgeSummary`; source item routes return `StudioKnowledgeItemsPage`. Notable errors: invalid `limit` query values return `bad_request`. ## Pipeline Run Persistence Types [#pipeline-run-persistence-types] ```ts type StudioPipelineRunStatus = "running" | "success" | "error"; type StudioPipelineRunRecord = { runId: string; pipelineId: string; status: StudioPipelineRunStatus; input: JsonValue; output?: JsonValue; error?: JsonValue; metadata?: JsonObject; startedAt: string; endedAt?: string; durationMs?: number; }; type StudioPipelineRunSaveInput = { runId: string; pipelineId: string; status: StudioPipelineRunStatus; input: JsonValue; output?: JsonValue; error?: JsonValue; metadata?: JsonObject; startedAt: string; endedAt?: string; durationMs?: number; }; type StudioPipelineRunListOptions = { pipelineId: string; limit: number; }; type StudioPipelineRunStore = { savePipelineRun( input: StudioPipelineRunSaveInput, ): StudioPipelineRunRecord | Promise; listPipelineRuns( options: StudioPipelineRunListOptions, ): StudioPipelineRunRecord[] | Promise; }; ``` Purpose: persistence contract for Studio pipeline run history. Return behavior: stores normalize saved run input into `StudioPipelineRunRecord` values and list recent records per pipeline. Notable errors: store failures surface through the Studio runtime or HTTP route handling the pipeline request. ## Error Types [#error-types] ```ts type StudioErrorCode = | "bad_request" | "conflict" | "not_found" | "unsupported_capability" | "internal_error"; type StudioErrorResponse = { error: { code: StudioErrorCode; message: string; details?: JsonValue; }; }; ``` Purpose: normalized Studio HTTP error shape. Return behavior: returned by API routes on request or runtime failures. Notable errors: `internal_error` responses serialize thrown errors into `details` when available. # Multiple Agents (/docs/studio/agents/multiple-agents) Register multiple agents when you want one playground for several roles. ## Example [#example] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { Studio } from "@anvia/studio"; const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); const supportAgent = new AgentBuilder("support-triage", model) .name("Support Triage") .description("Summarizes customer-facing support tickets.") .instructions("Return concise support summaries and next actions.") .build(); const engineeringAgent = new AgentBuilder("engineering-triage", model) .name("Engineering Triage") .description("Turns incidents into engineering diagnostics.") .instructions("Return concrete diagnostics, owners, and unknowns.") .build(); const commsAgent = new AgentBuilder("customer-comms", model) .name("Customer Comms") .description("Drafts customer updates for incidents.") .instructions("Avoid unverified root-cause claims.") .build(); new Studio([supportAgent, engineeringAgent, commsAgent]).start({ port: 4021, }); ``` Studio uses the agent id from `AgentBuilder`. If two agents share the same id, Studio makes the later ids unique by adding a suffix. ## Check Agents [#check-agents] ```bash curl http://localhost:4021/agents ``` ```json { "agents": [ { "id": "support-triage", "name": "Support Triage", "quickPrompts": ["Summarize TICKET-1001 for the support lead."] } ] } ``` ## Add Quick Prompts [#add-quick-prompts] Quick prompts make multi-agent playgrounds easier to test repeatedly. ```ts new Studio([supportAgent, engineeringAgent, commsAgent], { quickPrompts: { "support-triage": ["Summarize TICKET-1001 for the support lead."], "engineering-triage": ["Prepare diagnostics for a webhook retry incident."], "customer-comms": ["Draft a customer update for a degraded webhook incident."], }, }).start({ port: 4021 }); ``` See [`Quick Prompts`](/docs/studio/configure/quick-prompts) for the focused configuration guide. # Register Agents (/docs/studio/agents/register-agents) Studio runs agents that were already built with the Anvia SDK. It does not define model, tool, instruction, or output behavior by itself. ## Register One Agent [#register-one-agent] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { Studio } from "@anvia/studio"; const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); const agent = new AgentBuilder("support-operations", model) .name("Support Operations") .description("Answers operational support questions.") .instructions("Use tools for private order data. Keep answers concise.") .build(); new Studio([agent]).start({ port: 4021 }); ``` Open `http://localhost:4021/playground`. ## Agent Identity [#agent-identity] Studio uses the built agent id as the stable runtime id. | Field | Use it for | | ----------------- | -------------------------------------------------------- | | Agent id | Stable API path, session ownership, and quick prompt key | | Agent name | Human-readable label in Studio | | Agent description | Short explanation of the agent role | | Instructions | Actual runtime behavior | Keep ids stable. Saved sessions, trace metadata, and API calls use them. ## Check Registered Agents [#check-registered-agents] ```bash curl http://localhost:4021/agents ``` ```json { "agents": [ { "id": "support-operations", "name": "Support Operations", "description": "Answers operational support questions.", "quickPrompts": [] } ] } ``` Use [`Multiple Agents`](/docs/studio/agents/multiple-agents) when you want one Studio runtime for several roles. # Capabilities (/docs/studio/configure/capabilities) Studio reports runtime capabilities from `/config`. The UI uses this shape to decide which surfaces should be available. ```bash curl http://localhost:4021/config ``` ```json { "id": "anvia-studio", "agents": [], "pipelines": [], "chat": { "quickPrompts": {} }, "capabilities": { "agents": { "enabled": true }, "sessions": { "enabled": true }, "traces": { "enabled": true } }, "unsupportedCapabilities": [] } ``` ## Capability List [#capability-list] | Capability | Enabled when | | --------------- | ----------------------------------------------------------------------------------- | | `agents` | Always enabled | | `sessions` | Session storage is available | | `traces` | Trace storage is available | | `tools` | At least one registered agent has static tools or dynamic tool indexes | | `mcps` | At least one registered agent has tools registered from an MCP server | | `pipelines` | At least one pipeline is registered | | `observability` | At least one registered agent has observers | | `approvals` | At least one registered agent has a tool with approval metadata or a runtime hook | | `knowledge` | At least one registered agent has static context, dynamic context, or dynamic tools | ## Unsupported Capabilities [#unsupported-capabilities] If a capability is unavailable, Studio lists it in `unsupportedCapabilities`. Requests to unsupported routes return an `unsupported_capability` error. ```json { "unsupportedCapabilities": ["sessions", "traces"] } ``` Most applications should inspect `/config` before building custom tooling around optional Studio surfaces. # Quick Prompts (/docs/studio/configure/quick-prompts) Quick prompts are predefined messages shown by Studio for a registered agent. Use them for repeatable test cases, demos, and regression checks while tuning instructions or tools. ```ts new Studio([supportAgent, engineeringAgent], { quickPrompts: { "support-triage": ["Summarize TICKET-1001 for the support lead."], "engineering-triage": ["Prepare diagnostics for a webhook retry incident."], }, }).start({ port: 4021 }); ``` Quick prompts are keyed by Studio agent id. For agents registered from `AgentBuilder`, that id comes from the builder id. ```ts const supportAgent = new AgentBuilder("support-triage", model) .instructions("Summarize support tickets.") .build(); ``` ## Inspect Quick Prompts [#inspect-quick-prompts] ```bash curl http://localhost:4021/config ``` ```json { "chat": { "quickPrompts": { "support-triage": [ "Summarize TICKET-1001 for the support lead." ] } } } ``` ## Guidance [#guidance] | Use quick prompts for | Avoid using them for | | ----------------------------- | ----------------------------- | | Common smoke tests | Secrets or real customer data | | Demo scenarios | Production prompt routing | | Repro cases for bugs | Long fixture files | | Comparing instruction changes | Replacing automated tests | # Serve Options (/docs/studio/configure/serve-options) Call `.start(...)` to serve the Studio UI and HTTP API. ```ts new Studio([agent]).start({ port: 4021, hostname: "localhost", log: true, }); ``` ## Options [#options] | Option | Type | Default | | ---------- | --------- | --------------------------- | | `port` | `number` | `RUNNER_PORT`, then `4021` | | `hostname` | `string` | Hono's default host binding | | `log` | `boolean` | `true` | When logging is enabled, Studio prints the local URL. ```txt Studio UI: http://localhost:4021/playground ``` If you omit `port`, Studio reads `RUNNER_PORT` first and falls back to `4021`. ```bash RUNNER_PORT=4040 pnpm tsx studio.ts ``` Use `.close()` when tests or scripts need to stop the server explicitly. ```ts const studio = new Studio([agent]).start({ port: 4021 }); // ... studio.close(); ``` # Storage and Persistence (/docs/studio/configure/storage-and-persistence) Studio creates a local SQLite store by default. It stores sessions, runtime messages, transcript entries, traces, and pipeline logs for local inspection. ## Default Path [#default-path] By default, Studio writes to: ```txt .anvia-studio/anvia-studio.sqlite ``` Set `ANVIA_STUDIO_DB` when you want a specific file. ```bash ANVIA_STUDIO_DB=.data/studio.sqlite pnpm tsx studio.ts ``` `AION_STUDIO_DB` is also read as a compatibility fallback. ## What Gets Stored [#what-gets-stored] | Data | Purpose | | ------------------ | --------------------------------------------------------------------------------------- | | Sessions | Local conversations for one registered agent | | Messages | Runtime history used when continuing a session, stored as message and message-part rows | | Transcript entries | UI-friendly run output with messages, reasoning, tool calls, approvals, and questions | | Traces | Generation and tool observations captured during runs | | Pipeline logs | Metadata-only audit events for Studio pipeline runs | ## Sessions and Traces [#sessions-and-traces] When storage is available, Studio enables both `sessions` and `traces` capabilities. Runs with a `sessionId` load the stored messages as history and append the final messages after the run completes. ```bash curl http://localhost:4021/config ``` ```json { "capabilities": { "sessions": { "enabled": true }, "traces": { "enabled": true } }, "unsupportedCapabilities": [] } ``` Use [`Sessions`](/docs/studio/runs/sessions) and [`Traces`](/docs/studio/runs/traces) for the runtime workflow. # Endpoints (/docs/studio/http-api/endpoints) Studio exposes the same runtime used by the bundled UI as an HTTP API. ## Core Endpoints [#core-endpoints] | Endpoint | Method | Purpose | | --------------------------------- | --------------- | --------------------------------------------------------------------------- | | `/health` | `GET` | Check that the runtime is alive | | `/config` | `GET` | Read agents, quick prompts, and enabled capabilities | | `/agents` | `GET` | List registered agents | | `/agents/:agentId` | `GET` | Read one agent config | | `/agents/:agentId/tools` | `GET` | Inspect registered tool metadata for one agent | | `/agents/:agentId/mcps` | `GET` | Inspect registered MCP server and tool metadata for one agent | | `/agents/:agentId/runs` | `POST` | Run an agent | | `/pipelines` | `GET` | List registered pipelines | | `/pipelines/:pipelineId` | `GET` | Read pipeline metadata and graph | | `/pipelines/:pipelineId/runs` | `POST` | Run a pipeline with JSON input | | `/pipelines/:pipelineId/logs` | `GET` | Read metadata-only pipeline audit logs | | `/sessions` | `GET`, `POST` | List or create sessions | | `/sessions/:sessionId` | `GET`, `DELETE` | Read or delete a session | | `/sessions/:sessionId/logs` | `GET` | Read metadata-only session audit logs | | `/sessions/:sessionId/traces` | `GET` | List traces for a session | | `/traces` | `GET` | List traces | | `/traces/:traceId` | `GET` | Read one trace | | `/approvals` | `GET` | List pending or resolved approvals | | `/approvals/:approvalId/decision` | `POST` | Approve or reject a tool call | | `/questions` | `GET` | List pending or answered human question requests | | `/questions/:questionId/answer` | `POST` | Answer an `ask_question` tool request | | `/knowledge` | `GET` | Inspect agent context, dynamic retrieval sources, and recent trace evidence | ## Error Shape [#error-shape] ```json { "error": { "code": "bad_request", "message": "Request body must be JSON" } } ``` Error codes are `bad_request`, `conflict`, `not_found`, `unsupported_capability`, and `internal_error`. For request examples, see [`Run Requests`](/docs/studio/runs/run-requests), [`Streaming Runs`](/docs/studio/runs/streaming-runs), [`Tool Approvals`](/docs/studio/human-in-the-loop/tool-approvals), and [`Human Questions`](/docs/studio/human-in-the-loop/human-questions). # Test Without a Port (/docs/studio/http-api/test-without-a-port) Use `.fetch(...)` when testing Studio wiring. This calls the same Hono app as the served runtime without opening a local port. ```ts import { Studio } from "@anvia/studio"; const studio = new Studio([agent]); const response = await studio.fetch( new Request("http://studio.test/agents/support/runs", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ message: "Hello" }), }), ); const body = await response.json(); ``` Use this for route-level tests around registered agents, run request shape, sessions, approvals, questions, and knowledge inspection. ```ts const config = await studio.fetch(new Request("http://studio.test/config")); expect(config.status).toBe(200); ``` Call `.start(...)` only when you need the browser UI or a real HTTP listener. # Human Questions (/docs/studio/human-in-the-loop/human-questions) Studio can pause a run when an agent calls the `ask_question` tool. The pending question can be answered from the Studio UI or HTTP API, then the run continues with the supplied answers. Human questions are most useful with streaming runs, because the `tool_question_request` event reaches the caller while the run is waiting. ## Streaming Event [#streaming-event] ```json {"type":"tool_question_request","question":{"id":"question_123","status":"pending"}} ``` After an answer is submitted, Studio emits a result event. ```json {"type":"tool_question_result","question":{"id":"question_123","status":"answered"}} ``` ## List Pending Questions [#list-pending-questions] ```bash curl http://localhost:4021/questions?status=pending ``` `status` accepts `pending` or `resolved`. `runId`, `agentId`, and `sessionId` can filter the list. ## Answer a Question [#answer-a-question] Answer with one entry per prompt. ```bash curl -X POST http://localhost:4021/questions/question_123/answer \ -H 'content-type: application/json' \ -d '{ "answers": [ { "questionId": "priority", "answer": "High", "choice": "high" } ] }' ``` The runtime returns the answers to the model as the result of the intercepted `ask_question` tool call. # Tool Approvals (/docs/studio/human-in-the-loop/tool-approvals) Studio can handle tool approvals from passive tool metadata or `tool.requestApproval(...)` in runtime hooks. Core still runs tools normally; Studio installs a per-run request hook that creates approvals and waits for human decisions. ## 1. Add Approval Metadata [#1-add-approval-metadata] ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { Studio } from "@anvia/studio"; import { z } from "zod"; const refundOrder = createTool({ name: "refund_order", description: "Issue a customer refund.", input: z.object({ orderId: z.string(), amount: z.number().positive(), reason: z.string(), }), output: z.object({ refundId: z.string(), status: z.enum(["issued"]), }), approval: { when: ({ args }) => args.amount > 100, reason: ({ args }) => `Review refund of $${args.amount} for ${args.orderId}.`, rejectMessage: "Refund rejected in Anvia Studio.", }, async execute({ orderId }) { return { refundId: `rf_${orderId.toLowerCase()}`, status: "issued" as const, }; }, }); const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); const agent = new AgentBuilder("support-operations", model) .instructions("Look up order state before refunding. Keep the final answer short.") .tool(refundOrder) .defaultMaxTurns(3) .build(); new Studio([agent]).start({ port: 4021 }); ``` `approval.when(...)` receives parsed tool arguments. Return `true` when Studio should ask for approval, and `false` when the tool can run immediately. ## 2. Start a Streaming Run [#2-start-a-streaming-run] Approvals are most useful with streaming because the pending approval event can reach the UI while the run waits. ```bash curl -N -X POST http://localhost:4021/agents/support-operations/runs \ -H 'content-type: application/json' \ -d '{"message":"Refund order ORD-1001 for 25 dollars.","stream":true}' ``` When the model calls `refund_order`, Studio emits a `tool_approval_request` event and keeps the tool paused. ## 3. Approve or Reject [#3-approve-or-reject] ```bash curl http://localhost:4021/approvals?status=pending ``` ```bash curl -X POST http://localhost:4021/approvals/approval_123/decision \ -H 'content-type: application/json' \ -d '{"approved":true,"reason":"Support lead approved."}' ``` Reject with `approved: false`. ```bash curl -X POST http://localhost:4021/approvals/approval_123/decision \ -H 'content-type: application/json' \ -d '{"approved":false,"reason":"Amount needs finance review."}' ``` ## Hook-Based Approval [#hook-based-approval] Use a hook when the approval rule depends on request-local state or crosses multiple tools. ```ts import { createHook } from "@anvia/core"; const approvalHook = createHook({ onToolCall({ toolName, tool }) { if (toolName !== "refund_order") { return tool.run(); } return tool.requestApproval({ reason: "Review this refund before it is issued.", rejectMessage: "Refund rejected in Anvia Studio.", }); }, }); ``` Attach the hook to the agent with `.hook(approvalHook)` or to one run with `.requestHook(approvalHook)`. Studio emits the same approval request and result events for hook-based approvals. ## Approval Flow [#approval-flow] 1. The model requests a tool call. 2. Studio finds approval metadata on the tool or receives `tool.requestApproval(...)` from a hook. 3. The metadata policy or hook request says approval is required. 4. Studio creates a pending approval. 5. A human approves or rejects it in the UI or API. 6. Anvia either runs the tool or sends the rejection message back to the model. Use tool approval for side effects such as refunds, account changes, deletes, external messages, and workflow transitions. ## Advanced Control [#advanced-control] If your app owns a custom approval service outside Studio, use an `onToolCall` request hook and return `tool.run()` or `tool.skip(...)` after your service resolves. # Knowledge (/docs/studio/inspect-context/knowledge) The knowledge endpoint summarizes context sources registered on Studio agents and recent evidence captured from traces. ```bash curl http://localhost:4021/knowledge?limit=25 ``` ## What Studio Reports [#what-studio-reports] | Source | Meaning | | ----------------- | ------------------------------------------------------------- | | `static_context` | Static documents attached to the agent | | `dynamic_context` | Dynamic context providers registered on the agent | | `dynamic_tools` | Dynamic tools registered on the agent | | Evidence | Documents and tool names observed in recent generation traces | ## Response Shape [#response-shape] ```json { "agents": [ { "agentId": "support-operations", "agentName": "Support Operations", "sources": [ { "kind": "static_context", "count": 2 }, { "kind": "dynamic_context", "count": 1 }, { "kind": "dynamic_tools", "count": 0 } ], "staticContext": [] } ], "evidence": [] } ``` Evidence is available when traces are stored. Without trace storage, Studio can still report configured agent knowledge sources, but recent evidence is empty. Use this page when you need to confirm which context sources are available to an agent and whether retrieval evidence appeared in recent runs. # Studio Overview (/docs/studio/overview) Anvia Studio is a local UI and HTTP runtime for built agents. Use it when you want to try prompts, inspect tool calls, review traces, and approve protected tool actions without building your own internal console first. ## The Primitive [#the-primitive] The primitive is `Studio`. ```ts import { Studio } from "@anvia/studio"; new Studio([agent]).start(); ``` `Studio` does three things: | Primitive | What it does | | ------------------------------- | ----------------------------------------------------------------------------- | | `new Studio([agent, pipeline])` | Registers built Anvia agents and pipelines | | `.start(...)` | Starts the local HTTP runtime and bundled Studio UI | | `.fetch(request)` | Lets tests or another runtime call the same Studio API without opening a port | Studio also exposes development surfaces around the runtime: | Surface | What it helps inspect | | ----------------- | --------------------------------------------------------------------------------- | | Agents | Registered agents, names, descriptions, and quick prompts | | Pipelines | Registered pipeline graphs, stage status, JSON test runs, and pipeline logs | | Runs | One-off prompts, streaming output, max turns, tool concurrency, and trace options | | Sessions | Persisted local conversation history and transcripts | | Traces | Model generations, tool spans, usage, errors, and run metadata | | Human in the loop | Tool approvals and `ask_question` requests | | Knowledge | Static context, dynamic context, dynamic tools, and recent retrieval evidence | ## What To Do First [#what-to-do-first] 1. Build an agent with `AgentBuilder`. 2. Register the built agent with `new Studio([agent])`. 3. Start Studio on a local port. 4. Open the Studio UI and send a prompt. 5. Add tools, approval hooks, sessions, and traces as the workflow becomes more realistic. ## Minimal Shape [#minimal-shape] ```ts import { AgentBuilder } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { Studio } from "@anvia/studio"; const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); const agent = new AgentBuilder("support", model) .name("Support") .description("Answers short support questions.") .instructions("Keep answers concise and ask for missing order ids.") .build(); new Studio([agent]).start({ port: 4021 }); ``` Open `http://localhost:4021/playground`. ## When To Use Studio [#when-to-use-studio] | Use Studio when | Keep using the SDK directly when | | ------------------------------------------------ | ---------------------------------------- | | You want a local playground for agents | You are shipping your product runtime | | You need to inspect messages, tools, and traces | You only need one scripted call | | You want human approval flows during development | Your app already owns the full UI | | You want persisted local sessions | You have production conversation storage | Studio is a development and internal-operations surface. Your application still owns product auth, user permissions, storage policy, and production UI. # Inspect Pipelines (/docs/studio/pipelines/inspect-pipelines) Studio can inspect built pipelines beside agents. Register both as top-level targets: ```ts import { AgentBuilder, PipelineBuilder } from "@anvia/core"; import { Studio } from "@anvia/studio"; const supportAgent = new AgentBuilder("support", model).build(); const ticketPipeline = new PipelineBuilder({ id: "ticket-pipeline", name: "Ticket Pipeline", }) .step((ticket) => ticket.trim(), { name: "Normalize" }) .prompt(supportAgent, { name: "Draft Reply" }) .build(); new Studio([supportAgent, ticketPipeline]).start({ port: 4021 }); ``` Studio reads pipeline graph metadata automatically from build order. Optional pipeline and stage metadata improves labels, descriptions, and inspector details, but every stage gets a generated id and label when metadata is omitted. ## What Studio Shows [#what-studio-shows] | Surface | What it shows | | ------------ | --------------------------------------------------------------------------------- | | Graph | Input, steps, nested pipelines, parallel branches, agents, extractors, and output | | Node details | Kind, label, description, metadata, branch key, agent id, or nested pipeline id | | Run controls | JSON-compatible input sent to `POST /pipelines/:pipelineId/runs` | | Logs | Persisted metadata-only pipeline audit logs with live stream updates | Pipeline logs include ids, stage kinds, status, durations, counts, and byte lengths. They do not persist raw input, output, prompt text, model output, tool arguments, tool results, documents, images, or reasoning text. ## HTTP Routes [#http-routes] ```bash curl http://localhost:4021/pipelines curl http://localhost:4021/pipelines/ticket-pipeline curl 'http://localhost:4021/pipelines/ticket-pipeline/logs?limit=200' ``` Run a pipeline with JSON input: ```bash curl -X POST http://localhost:4021/pipelines/ticket-pipeline/runs \ -H 'content-type: application/json' \ -d '{ "input": "Customer cannot check out", "stream": true }' ``` Studio HTTP pipeline inputs must be JSON-compatible. Use direct `pipeline.run(...)` in application code when a pipeline takes non-JSON values. # Run Studio (/docs/studio/run-studio) Start with a normal Anvia agent. Studio does not define agent behavior. It only runs and inspects agents you already built. ## 1. Install the Studio Package [#1-install-the-studio-package] ```bash pnpm add @anvia/core @anvia/openai @anvia/studio ``` ## 2. Create a Studio Entry File [#2-create-a-studio-entry-file] ```ts import { AgentBuilder, createTool } from "@anvia/core"; import { OpenAIClient } from "@anvia/openai"; import { Studio } from "@anvia/studio"; import { z } from "zod"; const client = new OpenAIClient({ apiKey }); const model = client.completionModel("gpt-5.5"); const getOrder = createTool({ name: "get_order", description: "Read an order summary from application state.", input: z.object({ id: z.string().describe("The order id to read."), }), output: z.object({ id: z.string(), status: z.enum(["processing", "blocked", "shipped"]), notes: z.string(), }), async execute({ id }) { return { id, status: "blocked" as const, notes: "Payment review is complete, but allocation is still pending.", }; }, }); const agent = new AgentBuilder("support-operations", model) .name("Support Operations") .description("Answers operational support questions.") .instructions("Use tools for private order data. Keep answers concise.") .tool(getOrder) .defaultMaxTurns(3) .build(); new Studio([agent]).start({ port: 4021 }); ``` ## 3. Start It [#3-start-it] ```bash pnpm tsx studio.ts ``` Open `http://localhost:4021/playground`. If you omit the port, Studio uses `RUNNER_PORT` and then falls back to `4021`. ## 4. Check the Runtime [#4-check-the-runtime] ```bash curl http://localhost:4021/health ``` ```json { "status": "ok", "runner": { "id": "anvia-studio" } } ``` ## 5. Run Through the API [#5-run-through-the-api] ```bash curl -X POST http://localhost:4021/agents/support-operations/runs \ -H 'content-type: application/json' \ -d '{"message":"What is happening with order ORD-1001?"}' ``` Studio returns the same kind of prompt response as `agent.prompt(...).send()`. ## Development Loop [#development-loop] 1. Edit the agent, tools, instructions, or hooks. 2. Restart the Studio entry file. 3. Try prompts in the playground. 4. Inspect tool calls, output, sessions, and traces. # Run Requests (/docs/studio/runs/run-requests) Run requests use the same agent runtime as `agent.prompt(...).send()`, with Studio-specific wiring for sessions, traces, approvals, and questions. ## Request Shape [#request-shape] ```ts type AgentRunRequest = { message: string | Message; history?: Message[]; sessionId?: string; stream?: boolean; maxTurns?: number; toolConcurrency?: number; metadata?: JsonObject; trace?: AgentTraceOptions; }; ``` Use `history` for one-off calls. Use `sessionId` when Studio should load and persist conversation history. ## Non-Streaming Run [#non-streaming-run] ```bash curl -X POST http://localhost:4021/agents/support-operations/runs \ -H 'content-type: application/json' \ -d '{ "message": "Summarize order ORD-1001.", "maxTurns": 3, "trace": { "name": "manual-test", "userId": "dev_1", "tags": ["studio"] } }' ``` The response is a prompt response. ```json { "output": "Order ORD-1001 is blocked while allocation is pending.", "messages": [], "usage": { "inputTokens": 0, "outputTokens": 0, "totalTokens": 0 } } ``` ## Session-Backed Run [#session-backed-run] ```bash curl -X POST http://localhost:4021/agents/support-operations/runs \ -H 'content-type: application/json' \ -d '{ "sessionId": "session_123", "message": "Continue from the last order update." }' ``` Studio verifies that the session belongs to the requested agent before running. # Sessions (/docs/studio/runs/sessions) Sessions store conversation history for one registered agent. Use them to replay development cases and compare behavior after changing instructions or tools. ## Create a Session [#create-a-session] ```bash curl -X POST http://localhost:4021/sessions \ -H 'content-type: application/json' \ -d '{"agentId":"support-operations","title":"Order triage"}' ``` ```json { "id": "session_123", "agentId": "support-operations", "title": "Order triage", "messageCount": 0 } ``` ## Run With a Session [#run-with-a-session] ```bash curl -X POST http://localhost:4021/agents/support-operations/runs \ -H 'content-type: application/json' \ -d '{ "sessionId": "session_123", "message": "Check order ORD-1001." }' ``` When `sessionId` is present, Studio passes the stored messages as history and appends the new run after the final response. ## List Sessions [#list-sessions] ```bash curl 'http://localhost:4021/sessions?agentId=support-operations' ``` The `limit` query parameter defaults to `50` and is capped at `100`. ## Read or Delete a Session [#read-or-delete-a-session] ```bash curl http://localhost:4021/sessions/session_123 ``` ```bash curl -X DELETE http://localhost:4021/sessions/session_123 ``` Deleting a session also removes its stored traces from the default SQLite store. # Streaming Runs (/docs/studio/runs/streaming-runs) Set `stream: true` to receive newline-delimited JSON events from a run. ```bash curl -N -X POST http://localhost:4021/agents/support-operations/runs \ -H 'content-type: application/json' \ -d '{"message":"Summarize order ORD-1001.","stream":true}' ``` Studio returns `application/x-ndjson`. ```json {"type":"turn_start","turn":1} {"type":"text_delta","turn":1,"delta":"Order ORD-1001"} {"type":"turn_end","turn":1} {"type":"final","output":"Order ORD-1001 is blocked.","messages":[]} ``` ## Runtime Events [#runtime-events] Studio can add human-in-the-loop events to the same stream. ```json {"type":"tool_approval_request","approval":{"id":"approval_123","status":"pending"}} {"type":"tool_approval_result","approval":{"id":"approval_123","status":"approved"}} ``` ```json {"type":"tool_question_request","question":{"id":"question_123","status":"pending"}} {"type":"tool_question_result","question":{"id":"question_123","status":"answered"}} ``` Streaming is the best mode for approvals and questions because the run can pause while the UI or API resolves the pending human action. ## Persisting Streaming Runs [#persisting-streaming-runs] When `sessionId` is present, Studio persists transcript updates while the stream runs. ```bash curl -N -X POST http://localhost:4021/agents/support-operations/runs \ -H 'content-type: application/json' \ -d '{ "sessionId": "session_123", "message": "Check order ORD-1001.", "stream": true }' ``` # Traces (/docs/studio/runs/traces) Studio adds a local trace observer when trace storage is available. Runs with a session get trace metadata automatically. ## List Traces [#list-traces] ```bash curl http://localhost:4021/traces ``` Filter by agent, session, or status. ```bash curl 'http://localhost:4021/traces?agentId=support-operations&status=success' ``` ```bash curl http://localhost:4021/sessions/session_123/traces ``` The `status` query parameter accepts `running`, `success`, or `error`. ## Read a Trace [#read-a-trace] ```bash curl http://localhost:4021/traces/trace_123 ``` Traces include: | Field | Meaning | | -------------- | ----------------------------------------------- | | `status` | `running`, `success`, or `error` | | `output` | Final agent output when available | | `usage` | Token usage from the model response | | `observations` | Model generation and tool execution spans | | `metadata` | Agent id, trace tags, user id, and run metadata | For external observability, use [Langfuse](/docs/guides/observability/langfuse). Studio traces are local and optimized for the development UI.