Anvia Docs

Import from @anvia/transformers.

DEFAULT_TRANSFORMERS_EMBEDDING_MODEL

const DEFAULT_TRANSFORMERS_EMBEDDING_MODEL = "Xenova/all-MiniLM-L6-v2";

Purpose: default Hugging Face model id used by TransformersEmbeddingModel.create(...).

Return behavior: constant string.

Notable errors: none directly.

Transformers Types

type TransformersPooling = "mean" | "cls";

type TransformersFeatureExtractionPipeline = (
  texts: string[],
  options: { pooling: TransformersPooling; normalize: boolean },
) => Promise<{ tolist(): unknown }>;

type TransformersEmbeddingModelOptions = {
  model?: string;
  pooling?: TransformersPooling;
  normalize?: boolean;
  maxBatchSize?: number;
};

Purpose: local embedding model configuration and pipeline contract.

Return behavior: consumed by the embedding model.

Notable errors: invalid pipeline output causes embedding calls to throw.

TransformersEmbeddingModel

class TransformersEmbeddingModel implements EmbeddingModel {
  readonly model: string;
  readonly maxBatchSize: number;
  constructor(
    extractor: TransformersFeatureExtractionPipeline,
    options?: TransformersEmbeddingModelOptions,
  );
  static create(options?: TransformersEmbeddingModelOptions): Promise<TransformersEmbeddingModel>;
  embedTexts(texts: string[]): Promise<Embedding[]>;
}

Purpose: local Transformers.js feature-extraction adapter.

Return behavior: create(...) loads the feature-extraction pipeline; embedTexts(...) returns one embedding per text.

Notable errors: rejects if model loading fails, the extractor output shape is invalid, or the embedding count does not match input length.

createTransformersEmbeddingModel

function createTransformersEmbeddingModel(
  options?: TransformersEmbeddingModelOptions,
): Promise<TransformersEmbeddingModel>;

Purpose: convenience wrapper around TransformersEmbeddingModel.create(...).

Return behavior: resolves a ready embedding model.

Notable errors: same as TransformersEmbeddingModel.create(...).