Anvia Docs

A request runner is the function your route, job, queue worker, or test calls. It owns the product workflow around one agent run. Keep it small enough to test directly and explicit enough that auth, context, tools, persistence, and error mapping are visible.

Runner Responsibilities

Responsibility	Why it belongs in the runner
validate input	reject bad product requests before model calls
resolve app context	auth, tenant, feature flags, service handles, and transactions are app-owned
load state	conversation history, memory ids, idempotency records, and product records are storage-owned
create scoped runtime	request-scoped tools and context should not leak into globals
call the agent	the runner controls trace metadata, limits, and streaming vs final response
persist output	`response.messages`, events, audit records, and summaries are product state
map failures	routes and jobs need predictable product errors

Standard Runner Shape

import { Message, PromptCancelledError } from "@anvia/core";
import { model } from "./model";
import { createSupportAgent } from "./support-agent";

type SupportRunnerInput = {
  conversationId: string;
  message: string;
  auth: AuthService;
  conversations: ConversationStore;
  services: {
    orders: OrdersService;
    tickets: TicketsService;
  };
};

export async function runSupportTurn(input: SupportRunnerInput) {
  const message = input.message.trim();

  if (message.length === 0) {
    return { ok: false as const, error: "message_required" };
  }

  const user = await input.auth.requireUser();
  const history = await input.conversations.loadMessages(input.conversationId);

  const agent = createSupportAgent(model, {
    userId: user.id,
    tenantId: user.tenantId,
    plan: user.plan,
    services: input.services,
  });

  try {
    const response = await agent
      .prompt([...history, Message.user(message)])
      .withTrace({
        name: "support-chat",
        userId: user.id,
        metadata: {
          tenantId: user.tenantId,
          conversationId: input.conversationId,
        },
      })
      .send();

    await input.conversations.append(input.conversationId, response.messages);

    return {
      ok: true as const,
      output: response.output,
      usage: response.usage,
    };
  } catch (error) {
    if (error instanceof PromptCancelledError) {
      return { ok: false as const, error: "cancelled" };
    }

    throw error;
  }
}

The route can stay thin:

export async function POST(request: Request) {
  const body = await request.json();

  const result = await runSupportTurn({
    conversationId: body.conversationId,
    message: body.message,
    auth,
    conversations,
    services: { orders, tickets },
  });

  if (!result.ok) {
    return Response.json({ error: result.error }, { status: 400 });
  }

  return Response.json({ output: result.output });
}

Load Context Before the Prompt

Do not ask the model to discover product context that the application can resolve deterministically. Load the known values first and attach only the facts the agent needs.

const account = await accounts.findForUser(user.id);
const openTickets = await tickets.listOpen({ userId: user.id, limit: 5 });

const agent = createSupportAgent(model, {
  userId: user.id,
  tenantId: user.tenantId,
  plan: account.plan,
  services: input.services,
});

const prompt = [
  ...history,
  Message.user(`
Current account plan: ${account.plan}
Open ticket count: ${openTickets.length}

User message:
${message}
  `),
];

const response = await agent.prompt(prompt).send();

Use tools when the model needs to decide whether to fetch or change state. Use preloaded context when the application already knows the fact is required.

Timeouts and Retries

Anvia prompt requests do not replace product-level timeout or retry policy. Put bounded-latency behavior around the runner.

export async function withTimeout<T>(promise: Promise<T>, timeoutMs: number): Promise<T> {
  let timeout: ReturnType<typeof setTimeout> | undefined;

  const timeoutPromise = new Promise<never>((_, reject) => {
    timeout = setTimeout(() => {
      reject(new Error("support_agent_timeout"));
    }, timeoutMs);
  });

  try {
    return await Promise.race([promise, timeoutPromise]);
  } finally {
    if (timeout !== undefined) {
      clearTimeout(timeout);
    }
  }
}

const result = await withTimeout(runSupportTurn(input), 30_000);

Retry only when the whole operation is safe to retry. If tools can create side effects, the service layer should use idempotency keys or transactions before the runner retries the prompt.

Testing Boundary

A runner is easy to test because every product dependency is passed in.

it("rejects empty messages before calling the agent", async () => {
  const result = await runSupportTurn({
    conversationId: "conv_123",
    message: " ",
    auth: fakeAuth(),
    conversations: fakeConversations(),
    services: fakeServices(),
  });

  expect(result).toEqual({ ok: false, error: "message_required" });
});

Test validation, auth failures, storage calls, known cancellations, and persistence without a provider call. Use provider-backed tests only for the model behavior the workflow depends on.

Request Runners

Runner Responsibilities

Standard Runner Shape

Load Context Before the Prompt

Timeouts and Retries

Testing Boundary