3.8 KiB
Embedding and Rerank
These are retrieval operations, not conversations. Use a model enabled for the actual endpoint. Model names and scores do not establish dimensionality, compatibility or calibrated probabilities.
Embedding
POST /v1/embeddings:
{
"model": "text-embedding-3-small",
"input": ["blue", "red"],
"encoding_format": "float"
}
Read each data[*].embedding and associate it with the input using its index. Confirm item count, finite values, actual vector length and consistent dimensions. Preserve the embedding model/version and preprocessing used to build the index; do not mix vectors from incompatible spaces. Base64 encodings and token-array inputs need their own provider-compatible handling; the bundled probe checks float vectors from text input.
The live gateway returned 1024 dimensions for this alias. Adding dimensions:128 still returned 1024, so the dimension option was not honored in this tested route. Do not size a vector database solely from a public model name or assume every accepted parameter affects the result. Treat a required dimension mismatch as a failed contract. Other deployments/channels may behave differently.
dimensions is an optional upstream feature of certain models. This gateway's source forwards it, but code forwarding alone cannot prove implementation. Batch input order must be recovered by indices even if response rows arrive in a different order. Context/input and batch limits also depend on the provider.
Rerank
POST /v1/rerank:
{
"model": "bge-reranker-v2-m3-Q8_0.gguf",
"query": "How do I reset my password?",
"documents": [
"Fresh apples for sale.",
"Open account settings and choose reset password.",
"Rain is expected tomorrow."
],
"top_n": 2,
"return_documents": true
}
Read results[*].index and relevance_score; the index identifies the original candidate, not its new rank. Preserve candidate IDs separately and map them by index. This deployment returned [1,0] for the example and scores about 1.77 and -11.02; these are ranking scores, not probabilities bounded by 0–1. Do not invent a universal confidence threshold across models or queries.
Although return_documents:true was supplied, this route returned only indices/scores. Recover source documents from your original list. Object-shaped documents, chunk limits and additional options depend on the selected adapter; string documents are the tested baseline. The source preserves explicit top_n and return_documents through pointer fields, including zero/false values, but their accepted semantics remain provider-specific.
Embedding retrieval finds candidates; reranking reorders a candidate set against the current query. Choose how many candidates to retrieve/rerank for the application's quality and cost requirements rather than inheriting a fixed number from an example.
Probes
Put the matching request into UTF-8 request.json:
python '<skill-dir>/scripts/smoke_api.py' --mode embedding --payload-file request.json
python '<skill-dir>/scripts/smoke_api.py' --mode rerank --payload-file request.json
The helper does not dump vectors or documents. It validates indices, counts and finite numeric data; it reports actual dimensions and rejects a requested dimension mismatch. A valid vector or a sorted result is not proof of application retrieval quality.
Sources
Reviewed 2026-09-30: local New-API dto/embedding.go, dto/rerank.go, relay/embedding_handler.go, relay/rerank_handler.go, relay/common_handler/rerank.go and the dated live verification.
OpenAI embedding guide describes upstream embedding behavior; the alias-specific observations above come from this gateway's tests.