API documentation

Fluence routes each request to one of its providers. Your application needs one Fluence key and one endpoint. Access is invite-only during the private beta.

Quickstart

1. Ask your Fluence contact to invite your email, or open an invite link, then sign in with the one-time code or link and accept the terms.
2. Add prepaid USD credits in Billing. The minimum purchase is $5, and a first top-up of $10 or more can earn a bonus.
3. Create an API key with the inference permission on API keys and save it before closing the dialog. An empty access list lets it call every routing profile and catalog model.
4. Use https://api.fluence.cloud/v1 as the OpenAI-compatible base URL.

curl https://api.fluence.cloud/v1/models \
  -H "Authorization: Bearer $FLUENCE_API_KEY"

Discovery lists what the key can call: its routing profiles with Auto on, the default profile first, then its concrete models. Send one of these IDs as model, such as fluence/coder, and keep it for main, auxiliary and follow-up requests. For coding tools, follow the agent guide or a profile's export.

curl https://api.fluence.cloud/v1/chat/completions \
  -H "Authorization: Bearer $FLUENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"fluence/coder","messages":[{"role":"user","content":"Hello"}],"max_tokens":128}'

New accounts start with a guardrail that redacts sensitive data and inspects every response with execution rules and AI risk review. It refuses images and "zdr": true, holds each stream until inspection passes, and charges the classifier fee per check: about $0.00007 for a short request (Inspection and cost). Change it on Security before you rely on images, ZDR or token-by-token streaming.

Models and prices

The catalog lists each model's ID, price, context limit, privacy options and protocols. A price is the lowest eligible route under the key's default routing profile on Chat Completions, or on the protocols a model without Chat lists. Other protocols can route to other providers at other prices, and the faster provider preference can quote a dearer provider. Models stay available when their suppliers cost more than OpenRouter. A short supplier ID such as gpt-4o resolves to one catalog model; send the full catalog ID to remove any doubt.

Prices include the route's nominal markup. When OpenRouter's current featured rate is lower and still covers supplier cost, Fluence matches it. list_pricing holds the stable list benchmark for markup; openrouter_pricing and price_comparison hold the comparison, with basis featured or list. A negative price_comparison.discount_percent is a premium, zero an equal price, and null means no usable percentage.

A provider admits a request whose estimated size fits its context: one token per three ASCII bytes of request JSON and one per other byte, plus 1,024, plus the output limit the router sends. A provider with an unknown long-context price counts one token per byte. The estimate runs high (about 1.3 times GPT-6 Sol's own count on Codex requests), so set a coding tool's context window to about 70% of the model's context_length. A request no provider admits returns 400 no_eligible_route.

Catalogs refresh in the background. The Models page keeps the last good prices through a provider outage and marks them stale; GET /v1/models and inference use only fresh quotes, with standard and ZDR prices cached separately. When no fresh quote can be listed, discovery returns 503 catalog_unavailable with Retry-After.

Each model entry also lists max_output_tokens, zdr_available, tools, reasoning, structured_output and, where LiveBench has run the model, benchmarks with each run's revision, rank and scores. ?zdr=true lists ZDR routes and their prices; a value other than true, false, 1 or 0 returns 400 invalid_zdr. GET /v1/models/{id} is not served.

Each routing profile in discovery carries protocol_limits, the context and output limits per protocol. Its reported_context picks the smallest or largest model's limits, and the top-level limits are the smallest across protocols. They describe the profile's models, not a promise that every model fits every request. Profile entries also carry routing_profile (id and revision), pricing_kind: "upper_bound" and effort_policy; built-ins add builtin_key and template_version. A profile's prices are ceilings: the rates of the dearest model it can route to, within its price caps. Each request bills the rates of the model and route that served it, and the response's model names that model.

Long-context prices

Some models cost more for long prompts. Their catalog entry lists pricing_tiers, each with above_input_tokens, pricing, list_pricing, discount_percent and context_length. The model's context_length is the longest request any tier accepts; base_context_length is the longest billed at the base rate.

{"id": "openai/gpt-5.5", "context_length": 1000000, "base_context_length": 272000,
 "pricing": {"input": "3.85", "output": "23.1", ...},
 "pricing_tiers": [{"above_input_tokens": 272000, "pricing": {"input": "7.7", "output": "34.65", ...}, "context_length": 1000000, ...}]}

Input above a tier's above_input_tokens, cached tokens included, pays that tier's rates on every token: input, cache reads and writes, and output. Input equal to the threshold pays the tier below. Each tier is priced from the cheapest route that accepts prompts that long, so it can cost several times the base rate. Featured matching uses OpenRouter's thresholds and can add a tier where the supplier's price is flat. A route with an unknown long-context price accepts only requests below its threshold. The Models page shows each tier under the base price; Long-context charges covers the credit hold.

Sign-in

Each sign-in email carries a 6-digit code and a link. Both work once and expire after 10 minutes. A code allows 5 wrong attempts; after 20 wrong codes in a day, code sign-in for that address stops until 00:00 UTC and we email the address owner, while the link in a new email still works. Open the link in the browser that asked for it: any other browser names the account and waits for you to confirm. The workspace names the account you signed in to.

Report a security vulnerability to support@fluence.ai, also published at /.well-known/security.txt.

Request rules

Inference bodies hold up to 3,500,000 bytes, 100,000 JSON values and 128 nesting levels, and must arrive within 10 seconds. Past that, the request returns 413 request_too_large or request_too_complex, or 408 upload_timeout, and nothing is dispatched.

Fluence accepts the fields each protocol section names. Any other top-level field, such as Chat's verbosity, Responses' truncation and max_tool_calls, or OpenRouter's provider, models, transforms and plugins, returns 400 unsupported_provider_control before credit is reserved. Features Fluence does not serve return their own 400 code:

Chat Completions is the exception for output limits: send max_tokens with max_completion_tokens and Fluence checks both and sends the smaller. It also converts legacy functions and function_call to tools.

Send an Idempotency-Key of up to 200 characters so the router never sends the same request twice. Once a request reserves credit, its key stays used for the whole account, across all its API keys, without expiry: a reuse returns 409 duplicate_request and replays nothing, since response bodies are not stored. A request refused before credit is reserved, or refused for capacity before dispatch (X-Should-Retry: true), leaves its key free for the retry.

Chat Completions

Request one completion at a time (n=1) and set an output limit; reasoning tokens count toward it.

curl https://api.fluence.cloud/v1/chat/completions \
  -H "Authorization: Bearer $FLUENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"fluence/coder","messages":[{"role":"user","content":"Hello"}],"max_tokens":128,"stream":true}'

The router checks sampling controls on every protocol before it reserves credit:

• temperature from 0 to 2 (0 to 1 on Anthropic Messages), top_p from 0 to 1, top_k an integer of 0 or more
• frequency_penalty and presence_penalty from -2 to 2, seed a signed 64-bit integer
• stop a string or an array of strings, stop_sequences an array of strings
• logit_bias token IDs mapped to numbers from -100 to 100

null keeps the protocol default. Any other value returns 400 invalid_ plus the field name, such as invalid_temperature, with no generation or charge.

Read streams through [DONE]. A usage-only chunk can follow finish_reason with empty choices or deltas. Process tool arguments before a finish in the same chunk, and keep the final usage and billing. A length finish can carry reasoning tokens without visible text.

Responses

POST /v1/responses takes model, input and max_output_tokens, with streaming and client-executed tools on eligible models. It is stateless: send the history with each request. Stored responses, previous_response_id, background jobs, hosted tools and unbounded multi-pass Pro mode are not supported. A tool namespace is accepted when every tool in it is client-executed.

Fluence neither requests nor returns encrypted reasoning: it drops an include that lists only reasoning.encrypted_content, strips encrypted_content from output and rejects encrypted reasoning in input. On openai/ models other than open-weight gpt-oss it drops reasoning input items without encrypted content; other models keep items with summary or reasoning text.

Streams follow the OpenAI event order the SDK stream helpers check: response.created first, each output item and content part announced before its deltas, sequence_number counted from 0, and response.in_progress at most once, before any output. A provider's response.reasoning.delta and .done arrive as response.reasoning_text.delta and .done; other non-standard events, keepalives and lifecycle events without a response are dropped. Response objects echo your own instructions, prompt_cache_key, safety_identifier, user, tools, tool_choice and parallel_tool_calls, never a provider's.

Structured output

Chat Completions takes response_format and Responses takes text.format: json_object for a JSON object, or json_schema with a named schema. Fluence checks the schema before generation, with local references only; a malformed, oversized or unsupported schema returns 400 invalid_response_schema.

Completed answers must satisfy the whole schema, including required fields and extra-property rules. Strict output must be raw JSON: Fluence does not strip fences, coerce values or repair it. Refusals, truncation and tool calls keep their protocol outcomes. Routes known not to support the format, strict schemas, namespaced tools or reasoning controls are filtered out before quoting, and another supplier of the same model can serve within your caps.

An answer that fails validation returns 502 invalid_structured_output; a validator failure returns 503 structured_output_validation_unavailable. Both are charged, carry should_retry: false and X-Should-Retry: false, and withhold JSON and protected answers. An unprotected stream may have sent partial output and ends with an error instead of success.

Anthropic Messages

Set the Anthropic base URL to https://api.fluence.cloud, without /v1, and send x-api-key with your Fluence key and anthropic-version: 2023-06-01. POST /v1/messages takes a model whose protocols include messages, plus messages and max_tokens; streams and tool calls keep the native format. POST /v1/messages/count_tokens returns 404, and Fluence does not forward anthropic-beta.

Claude Code works with its default settings. An output_config with only effort reaches the models that take it: Opus 4.5 takes low to high, Opus 4.6 and Sonnet 4.6 add max, and Opus 4.7 and later, Sonnet 5 and later, Fable and Mythos add xhigh. Other models run at their default effort. Other output_config fields return 400 unsupported_output_config. A context_management that only clears thinking blocks (clear_thinking_20251015) is dropped, so the model sees the conversation as sent; other edits return 400 unsupported_server_control.

Jev decisions

POST /v1/systemone with model: "typesafe/jev" (or jev), a text or structured state and a map of questions returns categories, scores or yes/no probabilities in the TypeSafe schema. It uses your key, balance and model permissions.

{
  "model": "typesafe/jev",
  "state": "I was charged twice.",
  "questions": {
    "billing": {"type": "noul", "instructions": "Is this a billing issue?"}
  }
}

Send 1 to 64 questions in up to 30,000 UTF-8 bytes. Output tokens are free. Streaming, chat messages, generation controls and ZDR are unavailable. Prices are in GET /v1/models.

Routing profiles (Beta)

A routing profile holds every routing setting: Auto, an ordered model list, a price ceiling, a default model, optional task policies, the provider preference, failover and three timeouts. Manage profiles on Routing; an account has up to 20 custom profiles plus the built-ins.

Each API key has an access list and a default profile, both set on API keys. The default profile is the one you link, or the account default for an unlinked key, so changing the account default moves every unlinked key. The access list names the profiles and concrete models the key can call; an empty list allows all of them.

• A request that names a profile runs under that profile and needs its Auto on.
• A request that names a concrete model runs under the key's default profile: its price caps, provider preference, failover, fallback model and timeouts apply with Auto on or off. With a restricted access list, the fallback serves only when the list includes it.
• Allowing a profile allows its whole Auto pool.

A profile outside the access list answers 403 routing_profile_not_allowed and a concrete model outside it 403 model_not_allowed.

Built-in profiles

New accounts get four built-ins beside a Default profile with Auto off, which is the account default. Built-ins use the same editor as custom profiles, and your edits survive future default updates.

Architect starts with a 1,000,000-token context floor and Coder with 200,000. Architect starts with faster providers and the others with cheaper ones, and provider failover starts on for all but Cheapest. Fixed efforts on the first three override client effort where supported; key spending limits still apply. Routines has no Messages route within its ceiling today. Keep the ID on follow-ups. The response names and bills the model that served; discovery prices are upper bounds for client estimates.

Export in a built-in's details gives coding-tool names, limits and setup steps. The API is GET /v1/routing/exports?tool=opencode with the inference permission, 30 a minute per account. opencode, codex, claude-code, pi and omp return configuration; hermes, kilo-code, cline, openhands, openclaw and deepseek-harness return a setup prompt, as format=prompt does for any tool. A key with an access list exports every entry its discovery lists; an unrestricted key exports its profiles with Auto on. Add profile=coder (or architect, routines, cheapest or a profile UUID) to export one profile. OpenCode gets one provider holding them all, Codex one catalog entry and Claude Code one launcher per entry. Codex exports need the installed release's exact tool_version=X.Y.Z. Exports reference your key through private environment variables. OpenCode, pi and omp exports and every setup prompt cap each entry's output at its published limit or 16,384 tokens for Architect, 8,192 for Coder and concrete models and 4,096 for other profiles, so cheaper pool models stay eligible.

Auto

Send a profile's fluence/handle ID from discovery as model. Editing the handle on Routing changes the client model ID, so update clients that use the old one. Original built-in IDs and profile/UUID IDs keep working; fluence/auto is retired and answers 404 model_not_found.

Auto picks from the profile's list or the compatible catalog in one of three modes: My list order, Lowest estimated cost (the default for new custom profiles, over the compatible catalog) or By task. A configured fallback is tried last. Auto skips a model that left the catalog, cannot take the input or its limits, quotes above the ceiling, or whose providers are blocked by policy, unhealthy or at capacity. A model that fails before output starts hands over to the next; after output starts, a failure ends the stream.

By task asks Jev to label the task and choose the model in one bounded call over the full eligible pool. Jev receives redacted visible text, tool names and candidate model IDs, without attachments, hidden reasoning or tool definitions, over zero data retention routes. Its fee is supplier cost plus 50%, at most $0.002 of supplier cost ($0.003 with markup). Tool continuations keep the working choice. An uncertain label keeps the previous label or uses General for effort. A missing model choice keeps a valid earlier Jev choice or uses the configured fallback; with neither, the request cannot route By task. Saved task-list policies stay until you pick a shared-pool mode.

Use the same profile ID for main and auxiliary requests. A Chat request without model runs through the key's default profile when its Auto is on and answers 400 model_required otherwise; Responses and Messages always need model. The routing object is Chat-only.

{
  "model": "fluence/coder",
  "messages": [{"role": "user", "content": "Extract the city: Our office is in Lisbon."}],
  "max_tokens": 256,
  "routing": {"max_price": {"input": 0.20, "output": 1.00}}
}

The price ceiling caps input and output rates in USD per million tokens, markup included; an empty rate is uncapped. It binds every model Auto serves, the default model included, so a profile whose default model quotes above it cannot be saved. It applies at every long-context rate a request can reach, sizing the request as the credit hold does (one input token per byte plus 1,024), and skips a model over it as over_price_ceiling. With Auto off it still caps the first provider and every fallback of a concrete request. It compares quotes: a provider's valid higher bill can still charge more (see Charges).

Task policies are optional. With them on, the analyzer labels each new user message and the routine or reasoning list replaces the model list. General and an empty task list use the model list, and a task list that runs out goes to the default model.

A request can narrow its profile, never widen it. routing.models filters the order, the default model included; routing.max_price takes the lower ceiling per rate; routing.task takes auto or forces routine, general or reasoning on a profile with task policies, and answers 400 task_policies_off on one without.

When no model is eligible the request fails without generation or charge: 403 auto_model_not_allowed when routing.models removed every model, 503 auto_no_eligible_model with Retry-After when a skipped model was unhealthy or at capacity, and 400 auto_no_eligible_model otherwise. A chain that ends in failure returns the last attempt's error; an admitted request can still incur a Jev fee, shown in fluence.billing and Usage. Every code is listed under Routing errors.

Auto conversations

Keep the profile ID on follow-up turns and send the history. Auto keeps a session per API key, profile and protocol for 24 hours after its last completed request, storing keyed digests and routing metadata, not conversation text. Keep the opening messages and client identifiers stable; compaction starts a new session. Changing Auto selection settings restarts selection; provider preference and timeout edits keep the session.

after_model_change: "stay" (default) keeps a model that took over while it stays eligible; "return" goes back to list order on the next human message. Tool continuations keep the working model either way. A harder task label moves to its stronger list; a cheaper label moves down only when the new model's full input rate beats the current provider's cache-read rate, or its full rate with caching off or ZDR on. Provider preference can still move a conversation to a cheaper provider, so a session does not guarantee a cache hit.

Auto strips foreign reasoning state after a model or provider change and skips models that need state it does not have. Set min_context_tokens in Models to require a minimum provider context across the pool, Jev candidates and fallbacks: a positive integer up to 2,147,483,647, or null. It leaves concrete Auto-off requests unchanged. Set reported_context to "smallest" or "largest"; new catalog profiles start with largest, and profiles created with a model list with smallest. For Codex and Claude Code settings, follow the agent setup guide.

Reasoning effort

effort_mode: "request" (default) passes the app's effort. "task" needs task routing and maps routine, general and reasoning to low, medium and high, rounding to the nearest supported level and up on a tie. Read each model's effort_levels in GET /v1/routing before fixing one.

efforts fixes the effort per model in each list: models, routine and reasoning. Its default applies only to the appended fallback model: {"models":{"openai/gpt-6-sol":"low"},"default":"high"} runs Sol at low in the main list and the fallback at high. A PATCH replaces the whole object.

On Messages, Auto effort reaches only Claude models that support it, and models without reasoning run without reasoning controls. Fluence picks a supplier that supports the request's tools, structured output, reasoning state and effort before quoting. fluence.model_selection.effort reports the value and its source: fixed, task, request or null. Changing effort can force a full-price reread of the conversation.

Response metadata

The response model is the serving model. Successful responses carry x-fluence-routing-profile, x-fluence-routing (cheaper or faster) and x-fluence-failover (model, provider or none); errors carry the profile header once the profile is resolved. Auto responses add x-fluence-selected-model, x-fluence-model-selector and fluence.model_selection in the body or final stream chunk. A Responses stream carries selection and billing under response.fluence in its terminal event:

{
  "requested_model": "fluence/coder",
  "selected_model": "openai/gpt-6-sol",
  "profile": {"id": "6f1c…", "revision": 4},
  "task": "reasoning",
  "task_source": "analyzer",
  "reason": "model_failover",
  "position": 2,
  "skipped": [{"model": "z-ai/glm-5.3", "reason": "over_price_ceiling"}],
  "attempts": [
    {"model": "anthropic/claude-opus-5.5", "outcome": "failed"},
    {"model": "openai/gpt-6-sol", "outcome": "served"}
  ]
}

• reason: first_choice, model_failover, default_model, or session_model when Auto keeps a working model.
• position: the serving model's place in the list that applied; null for the appended default model.
• task_source: analyzer, request, session, analyzer_unavailable or off.
• skipped: models skipped as unsupported, over_price_ceiling or unavailable, leaving out those the key or request removed.
• session: the turn kind, whether the model started, stayed or changed, the label change and the turn count.
• selector: rules, with no fee, or jev, with version, task_confidence and selector_cost_usd.

Auto errors carry the same object. Provider names stay private. Billing and Usage record the serving model at its route's price. Try Auto in the playground.

Manage profiles

GET /v1/routing with account:read returns default_profile_id, profiles, zero_data_retention, platform_requires_zdr, platform_timeouts and the models a profile can list. Each model carries pricing_by_preference, pricing_tiers_by_preference, list_pricing, list_pricing_tiers, protocols, tools, structured_output and effort_levels. analyzer_available says whether Jev is available, and each profile names its linked keys as keys: {count, ids}. With account:write:

• POST /v1/routing/profiles creates a profile. Its name is 1 to 64 characters, unique ignoring case. Without model_scope, models or task_models, the profile uses the whole compatible catalog at lowest_cost, with reported_context: "largest" and provider_preference: "cheaper"; with models, it uses that list in order. Other omitted fields take the platform template, which has Auto off: send "auto": true, or the profile stays out of discovery and answers 403 auto_disabled. An optional handle sets fluence/handle: 1 to 48 lowercase letters, digits, hyphens or underscores, starting and ending with a letter or digit. auto and the built-in handles are reserved; without one, the server derives a handle.
• PATCH /v1/routing/profiles/{id} replaces each top-level field it names and carries the revision it edits; a stale one answers 409 routing_profile_changed. {"revision": 4, "default": true} makes a profile the default and increments the revision of both it and the previous default.
• DELETE /v1/routing/profiles/{id} answers 409 routing_profile_in_use for the default profile or one that unrevoked keys link or list, naming those keys.

Set key links and access lists in the console. max_price is null or an object with input, output or both, in USD per million tokens from 0 to 1,000; a missing or null rate is uncapped, but one rate needs a value. failover (new profiles: false) needs both rates, and a listed Auto profile also needs a candidate in models, a task list or default_model; otherwise the save answers 400 validation_error. A draft without failover previews with it off.

POST /v1/routing/preview with inference takes {"prompt":"hi","max_tokens":1024} and an optional complete draft profile. It reports the model Auto would pick, the task, skipped models, current rates and an estimated cost at your max_tokens, capped at 20,000 on JSON today (the Playground streams with a 9,800 cap). pricing_tiers lists the quote's tiers, price_tier the tier reached and estimate_pricing the rates used; failover prices are left out. Preview generates nothing, stores no prompt, reserves no credit and allows 30 checks a minute. A Jev profile's preview does not call the analyzer: it estimates General and returns analyzer_evaluated: false.

Provider routing

A profile's provider preference orders each model's providers, for Auto and named models alike. Cheaper sorts by this request's estimated cost at its long-context tier. Faster starts from that order and moves a dearer provider ahead only when its latency and failures over the last 30 minutes still rank it better after removing any one account's observations. Until several accounts have measured a provider, faster sorts by price, and up to 1% of faster requests try an unmeasured provider first at its quoted price. Profiles created with a model list start with faster; catalog profiles need cheaper. Neither setting guarantees capacity or latency.

The quote is the first-ranked provider's rate, so faster can quote more than cheaper. A supplier price below 10% of the reference for any rate, cache reads and writes included, does not route until an administrator approves it.

Failover

Before any output reaches you, the router can move a request. Provider failover, a profile setting that starts off on new custom profiles, tries another provider of the same model. Model failover belongs to Auto and tries the next model. A request that names a model keeps that model. Turning failover on needs both price caps and a selected model; the console fills an empty cap with the highest list price among the selected models, long-context rates included.

The router moves on after a provider error, a dropped connection, a provider at capacity, or a stream's missed platform first-response or idle deadline. It returns the error instead when output already reached you, the request is invalid (400 or 422, unless an earlier provider started and failed), a provider stops a Responses answer under its content policy, or a JSON request's provider errors more than 15 seconds after dispatch.

Every attempt keeps your protocol, privacy constraints, tool-call protection and price caps. A named model gets at most 3 attempts; Auto gets 2 per model, 3 on the chain's last model and 7 in total. With failover off, a provider failure ends that model's attempts; Auto can still move to the next model. The total deadline, at most 240 seconds, covers every attempt. Failed attempts cost nothing. The provider that serves charges its own price, its quote or its valid bill plus markup, which can land above or below the first quote (see Charges). x-fluence-failover reads provider when a provider other than the first-ranked one served and model when Auto moved to another model; Auto responses list each model tried in fluence.model_selection.

Retry yourself only after 429, 502, 503 or 504, with backoff and Retry-After, and respect X-Should-Retry: false and should_retry: false. Before retrying a 504 whose message says generation continues, check GET /v1/usage/requests/:id: the provider still finishes and bills that request.

Output limits

A provider can bill a cancelled request its whole output limit, so the router sends a provider only what it can deliver in time: 100 output tokens per second of the generation window. That window runs to the later of your profile's total deadline and the platform's 240 seconds, less 10 seconds for settlement and the time allowed for the first response. Today that is 20,000 tokens on a JSON request and 9,800 on a stream; a stream's 16 MiB size limit never allows more than 16,777. A retry, or the next Auto model, gets what fits in the time left; each Auto model is planned on its own.

Send max_tokens, max_completion_tokens or max_output_tokens up to the model's output maximum. A value above the cap but within the model's maximum is lowered to the cap, not refused, and the answer stops there with the protocol's length finish: finish_reason: "length", stop_reason: "max_tokens" or an incomplete Responses answer. Without a limit the router sends 4,096 on a JSON request, or less when a provider it may use has a lower maximum; on a stream it sends each provider as much as that provider can take, up to the stream cap. Providers are chosen by the limit the router sends, and a value above every provider's maximum for the model returns 400 no_eligible_route.

A reasoning or thinking budget at or above a lowered limit drops to one token below it, so thinking.budget_tokens stays below max_tokens; when no room is left for a 1,024-token budget, the request returns 400 thinking_budget_limit before any provider is called. The credit hold uses the lowered limit. Reasoning tokens count, so an answer can stop before visible text. A measured slow supplier can get a lower limit at dispatch. Tool-call protection uses lower caps.

Timeouts and retries

Each routing profile sets first-response, stream-idle and total deadlines on Routing, and a request follows the profile it runs under. An unset deadline (null) follows the platform: 120 seconds to the first response, 45 seconds idle and 240 seconds total. Deadlines must leave room for 4,096 output tokens after the first response, counted to the later of your total and the platform's, and a save that leaves less is refused. The platform keeps 10 seconds of its total for settlement, and an answer finished just before your cutoff can arrive up to about 4 seconds later.

On a stream, the first response is the first output or, for a model that hides its reasoning, its first finished reasoning step. The idle deadline counts from there, and on Claude models it also waits while a tool call is open, since Claude sends each tool argument whole. A JSON request has only the total deadline: the router waits for the provider and does not move a slow JSON request elsewhere.

A late retry gets a shorter first-response wait and a smaller output limit: after a full 120-second wait, a streaming retry has about 55 seconds to start and sends up to 5,500 tokens. No retry starts once it can neither keep your output limit nor send 1,000 tokens.

A JSON request whose provider answers 408, or a 5xx other than 503 and 529, more than 15 seconds after dispatch ends with that error and X-Should-Retry: false: the provider may bill it, and another would generate it again. Stream long answers or lower the output limit. After output starts, failures arrive as stream errors, so read the final event even after HTTP 200.

Once a request reaches a provider, a disconnect, your total deadline when shorter than the platform's, or a stream deadline shorter than the platform's ends your response, not the generation: the router keeps reading to final usage, bills it and does not fail over (see Cancelled requests). A JSON request returns 504 request_timeout at your cutoff; a stream that has not started returns request_timeout, first_token_timeout or idle_timeout, and a started stream closes with that error event. The message says when generation continues. An answer not yet released at the platform's own cutoff ends uncharged.

Deadlines you set shorter than the platform's never pause a provider for other accounts. A Responses answer the provider stops under its content policy returns 400 content_filtered without fallback; output already streamed is billed.

Retry 429, 502, 503 and 504 with bounded exponential backoff and jitter, respecting Retry-After, and send an Idempotency-Key so a retry never dispatches the same request twice.

Keys and limits

A key can also limit requests per minute and day, concurrency, a monthly USD budget that resets each UTC month, expiry, its access list and IPv4 or IPv6 addresses or CIDRs. Empty means no key restriction; account limits still apply. A key over its per-minute or daily limit gets 429 requests_per_minute or requests_per_day with Retry-After: 60; over its monthly budget it gets 429 monthly_budget with Retry-After to the next UTC month and X-Should-Retry: false. A missing permission returns 403 scope_required and an address outside the list 403 ip_not_allowed.

Creating or rotating a key, and adding scopes, IP ranges or time to one, need a sign-in from the last 10 minutes, including changes confirmed in support chat. Revoking, editing the access list and other changes work from any session. We email the owner whenever a key is created or rotated; an account can create or rotate 30 keys an hour. Security > Activity lists key changes from the past 31 days.

Uncharged requests

Requests that end without a charge because of the request itself draw on an hourly budget: client cancels and disconnects, invalid requests (a supplier's 400 or 422 included), first-response, idle and total deadlines, the response size limit, response protection blocks, and output too large to inspect. Supplier errors, capacity limits and platform failures add nothing.

Each such request uses one unit per started $0.01 of its estimated supplier cost, and at least one: the cost its attempts recorded (an attempt the router cut records its input and whole output limit), or, when a supplier accepted it and recorded nothing, its input and output limit at quoted rates. A stream cut after it sent output is charged for that output, and the rest of the provider's bill counts in units. A JSON attempt a provider failed more than 15 seconds after dispatch still counts its whole generation. A request another provider then answered counts nothing.

Each API key has 600 units an hour, shared with keys rotated from it, and the console has its own 600. The account has 1,800, to which one key or the console adds at most its own 600 an hour. When a budget is spent, new requests under it return 429 uncharged_limit until its hour ends, with the seconds left in Retry-After.

Running requests count at what they could cost uncharged: their input and output limit at quoted rates. When a budget's used units plus its running requests' units reach its limit, a new request gets 429 uncharged_limit with a 5-second Retry-After; the new request's own size does not count, and a request is admitted when nothing else is running. At $10 per million output tokens, a 9,800-token stream counts 10 units, so 60 fit in a key's budget. A model that hides its reasoning can trip a deadline on a short prompt; raise its profile's timeouts. Security > Activity lists each budget that filled.

Concurrency

Each account has one concurrency limit shared by its keys, models, endpoints and playground sessions, 50 by default; administrators can change it per account. Read yours from concurrency_limit in GET /v1/account. Past it, a request gets 429 account_concurrency without being queued or sent; a stricter key limit returns concurrent_requests. Requests count until settlement finishes, cleanup after a failure included, and a lowered limit lets running requests finish. Heavy requests can meet 429 uncharged_limit first.

Prompt caching

Set a key's default to passthrough, on or off (new keys start on), and override it per request with x-fluence-prompt-cache. Passthrough keeps supported cache controls. On marks shared instructions, tools and the growing Claude conversation as cacheable when you set no breakpoints, keeping existing ones. Off removes explicit controls but cannot stop a provider's implicit caching. x-fluence-prompt-cache-session labels a session, and x-fluence-prompt-cache-affinity reports the sticky-routing outcome: hit, miss, new, reused, sticky, off or unavailable. That is no cache-hit guarantee: read the cached-token counts in usage, where Chat Completions also reports billed cache writes in prompt_tokens_details.cache_write_tokens.

Support depends on the serving provider. Some accept native controls on all three APIs, some accept Claude cache markers, and some accept Messages markers only when the quote covers the requested lifetime; the router strips cache writes no quote prices. Routes with keyed automatic reuse get a cache key scoped to your API key: your prompt_cache_key, or a default one. Providers see cache keys and sessions only as key-scoped digests. With ZDR on, cache controls and session headers are removed. Paid retention windows are off, and cache rates are model-specific.

Security guardrails

A guardrail holds zero data retention, redaction and tool-call protection. Create guardrails on Security. Each API key links a guardrail, or follows the account default, independently of its routing profile, and an edit applies to its keys on their next request. The Playground can pick a guardrail for a test run.

New accounts get a Default guardrail from the platform template, and new guardrails start from it too. The template requires redaction and both tool-call protection controls, execution rules and AI risk review, with zero data retention off. So by default Fluence refuses images and "zdr": true, holds each response until inspection passes, caps output at the protected limits and charges every AI review check. Routing profiles change none of this. Read the saved guardrail before you rely on any setting.

Redaction works with tool-call protection or ZDR. ZDR works with execution rules but not with third-party AI risk review, whose retention is unverified; under ZDR, uninspectable tool calls must go to Block, so bind file and edit tools as non-executing first. Regional processing is not available yet.

Read guardrails with GET /v1/security/guardrails or GET /v1/security/guardrails/:id with the account:read scope. Only the signed-in console edits guardrails, defaults and key links, and previews drafts; API keys cannot, even with account:write. Per request, X-Fluence-Redaction: on and X-Fluence-Protection: on opt into the effective guardrail's saved settings, and off cannot weaken a requirement. Responses name the guardrail in X-Fluence-Security-Guardrail, and request metadata adds its revision.

A save that weakens protection, including one confirmed in support chat, needs a sign-in from the last 10 minutes. We email the account owner, and an account can make 10 such saves an hour. That covers turning ZDR, redaction, AI risk review or execution rules off; weakening an execution rule or tool binding; widening allowed tools; removing a built-in pattern; removing or editing a custom pattern; and switching patterns from block to redact. Turning protection on, adding patterns and switching to block save at once.

Sensitive data redaction

1. Open a guardrail on Security, expand Sensitive data redaction and select the categories you need.
2. Choose Redact to replace matching spans or Block request to reject the prompt.
3. Add custom patterns for your own identifiers and test representative input.
4. Turn on Require redaction for this guardrail, save, and link the guardrail on API keys or make it the default.

Selected patterns scan every message role, the history, tool arguments and results, metadata and text in tool definitions. Detection runs before dispatch and billing, and every failover attempt and tool-call protection receive the same cleaned request.

Placeholders and limits

Repeated values share a placeholder such as [EMAIL_1_FLUENCE_REDACTED], numbered by first appearance in prompt order: the cache key, tool definitions and output schemas, system and instructions, messages and input, then other fields. Appended turns keep earlier placeholders stable for caching, and numbers mark position, not the hidden value. Responses keep the placeholders: Fluence stores no mapping and restores nothing, so validate returned tool arguments before running them. Custom patterns also apply to markers.

A participant name on a Chat message or a Responses or Messages turn is checked like text. A detected name becomes a placeholder without brackets, such as PERSON_1_FLUENCE_REDACTED, so it stays a valid Chat name. Separators and capitals split words, so Ada_Lovelace and AdaLovelace read as "Ada Lovelace". Tool and function names are not replaced. The email detector leaves exactly noreply@anthropic.com and openhands@all-hands.dev unchanged; custom patterns still apply to them.

Detection is best effort, not anonymization: unknown formats, encoded or split values and unfamiliar names or places can be missed, and false positives happen. Name and address detection is an optional, English-focused beta that adds processing time.

Media, files, provider-stored content and encrypted inputs cannot be inspected and are rejected while redaction is on, as is a sensitive value in a protocol identifier or field name that cannot be replaced without breaking the request. The optional id of a Responses input item is dropped instead, since call_id still links tool calls; a block rule that matches it blocks the request. Inspection stops the request before dispatch past 50,000 characters of contextual analysis, 50,000 nodes, 48 levels or 10,000 detections, or when a detector fails.

A workspace inspects one request at a time per server, with one more waiting, and holds at most half the server's inspection memory. Past either, or after 0.5 seconds without a thread, a request gets 503 redaction_busy. Inspection then has 1.5 seconds and 128 MiB to finish. One that runs over pauses your workspace's redaction on that server for 5 seconds with redaction_cooldown, and a preflight past 2 seconds returns 503 preflight_timeout.

Custom patterns

Add up to 10 case-insensitive RE2 patterns of at most 256 characters, without slash delimiters, such as ACCT-[0-9]{6}. Lookarounds, backreferences and patterns that match empty input are rejected. Each pattern redacts or blocks, and a block match wins, also over a built-in redaction.

Patterns read every text value, field name and number, each counting its characters plus one. The count may reach 1,000,000, and the patterns' compiled size in RE2 instructions times the count must stay under 25,000,000. ACCT-[0-9]{6} compiles to 13 instructions, so ten like it cover about 190,000 characters, while [a-z]{1,1000} alone compiles to about 2,000. A request over budget returns redaction_pattern_limit before dispatch.

Test patterns previews the current selections, unsaved edits included: the cleaned text or block decision, match count and time. It calls no model, stores no sample and saves nothing, takes up to 50,000 characters and allows 30 previews a minute. Patterns are private and left out of audit details.

Per-request redaction

X-Fluence-Redaction: on applies the effective guardrail's saved patterns to one request, even when its redaction switch is off. Without the header or a guardrail requirement, redaction is off.

curl https://api.fluence.cloud/v1/chat/completions \
  -H "Authorization: Bearer $FLUENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Fluence-Redaction: on" \
  -d '{"model":"openai/gpt-5.6-luna","messages":[{"role":"user","content":"Email alice@example.com"}],"max_tokens":128}'

It works on Chat Completions, Responses and Messages, JSON or streaming. Responses report X-Fluence-Redaction: on, the count in X-Fluence-Redactions and fluence.redaction in the final result. Redaction alone does not buffer streams.

Tool-call protection

Tool-call protection inspects the complete response, streams included, before releasing it, looking for exfiltration, destructive or unauthorized actions and instruction injection. Turn it on for a guardrail under Tool-call protection on Security, in the playground, or per request with X-Fluence-Protection: on, which adds AI risk review. New guardrails start with it on. The guardrail also sets the protected output cap. Two controls work alone or together:

• Execution rules run inside Fluence, before AI review. They work with ZDR and add no fee.
• AI risk review sends the context, tool definitions and response to a third-party classifier, and costs its bill plus 50%. Turn on redaction to clean the request first.

The recommended setup combines them: known rule matches block at once, and unmapped tools or unsupported commands go to AI review, which must approve them. With ZDR, or when AI review is unavailable, it keeps strict blocking. A block, or a required inspection that is unavailable, ends the request without trying protection on another provider.

Execution rules

New guardrails bind the POSIX shell tools shell, exec_command, Bash, bash and terminal, bare and in functions. OpenCode's shell maps /command and /workdir; Codex's exec_command maps /cmd and /workdir. Use recommended settings adds these when your binding list is empty. Register other tools by exact name and optional namespace, with a JSON Pointer to the executable input (empty for raw custom-tool text) and an optional working-directory pointer. Declare a tool non-executing only when it has no executable input; its name cannot prove that. A tool request under rules with no bound tools returns 400 execution_policy_invalid.

The shell adapter detects curl and wget download-and-run chains, saved scripts and binaries, pipes, literal variables and escaping, supported wrappers, and base64 or xxd decoding into execution. Unknown syntax, unresolved expansion, embedded languages and ambiguous artifacts go to your Block or AI review fallback, and unbound tools follow your uninspectable-output setting. Past 64 proposed calls, 64,000 bytes per command, 4,096 syntax nodes or 16 nesting levels, or when getting a parser takes over one second or parsing over 1.5 seconds, output is withheld. History past 16,000 bytes keeps only the newest calls, and metadata marks it partial.

Rules do not read files, resolve symlinks, track omitted history or sandbox anything. Keep your client's permissions, approvals and containment, send the history rules need, and treat returned arguments as untrusted.

Test draft, beside Save, inspects inert samples without saving, logging or running them. Rules-only previews are free. An AI preview needs an explicit choice and a current maximum-charge estimate, and a completed one costs the classifier bill plus 50%, a block included. Unavailable previews are free.

Inspection and cost

The classifier sees up to 16,000 bytes of request context whole. A larger request, such as a coding agent's, is seen through a bounded view: your latest turn, the instructions, your earlier turns, the newest items and the called tools' definitions, each clipped, plus counts of what was left out. The response is never clipped: a long one is checked in up to 40 parts, each beside the same view with up to 4,000 bytes of overlap, and passes only when every part passes. Each check holds 64,000 bytes, the request view included. A tool call's arguments are never split, and a call too large for one check withholds the response. Protection takes text and client-executed tools only and refuses files and images; AI review also refuses ZDR requests. Buffered JSON is capped at 4,000,000 bytes and streams at 16 MiB.

A protected response is released only once it is generated and inspected, so its output limit keeps a second for each check: 9,100 tokens on a stream and 18,500 on JSON today, and 4,096 without a limit. A larger limit is lowered, never refused, and a protected output cap on Security lowers it further.

Suspicious output is withheld and ends the request; it is not generated again elsewhere. A classifier outage, malformed verdict or oversized response withholds it too. Before a verdict, a provider error fails over under the profile's usual rules and attempt limits.

AI review costs the classifier's bill plus 50%, $0.063 per million tokens each check reads at today's price: the request view, its part of the response and about 1,000 tokens of questions. The hold reserves every possible check; successful requests pay for the checks made, failover checks included, and failed requests pay nothing.

Read the verdict in fluence.protection or Usage. X-Fluence-Protection reads passed on a released response and rejected on any later error, timeouts included; errors raised while the body is read or validated carry it only when you sent X-Fluence-Protection: on. fluence.billing.charged_usd includes inference_usd and protection_usd. A verdict is probabilistic, not proof that a response is safe; keep client-side tool permissions, sandboxing and approvals.

Zero data retention

Set ZDR in a guardrail's Data retention section on Security. Linked keys use it and unlinked keys follow the default guardrail; platform requirements still apply. Only providers that advertise ZDR through their API are eligible, and Fluence turns off explicit prompt caching, response storage and temporary uploads. File inputs and hosted cache references are rejected; supported inline images work with ZDR alone.

Send "zdr": true to require ZDR for one request, and use GET /v1/models?zdr=true for eligible models and ZDR prices. A request cannot relax a guardrail or platform requirement. Every failover attempt keeps the policy, so no eligible capacity means a 503, never an unprotected route. Successful responses report x-fluence-zdr: true or false.

ZDR controls retention, not redaction or processing location. Prompts and completions stay out of request telemetry; operational and billing metadata remain.

Turning ZDR on saves at once. Turning it off needs a sign-in from the last 10 minutes (a support-chat request waits for one) and emails the owner. Every ZDR change is recorded in the audit history. API keys cannot change it: PATCH /v1/account accepts only display_name and timezone, and a body with settings returns 400 routing_settings_moved.

To register interest in processing in Canada, the United States, Europe or another country, ask in support chat. That enables no regional routing and makes no certification claim.

Activity and errors

Activity shows the latest 20 of each, from the past 31 days: key and protection changes (keys by prefix, whether a change gave a key more access, each way a save reduced protection, ZDR on or off, support-chat confirmations included), pre-dispatch blocks (request ID, time, categories and counts), and filled uncharged-request budgets (the key prefix or console, the request ID and when the window ends). Usage shows redaction counts and protection verdicts, scores, categories and latency. Admin views record the same events without prompt or response content.

Charges

Before dispatch the router holds a conservative credit estimate, and after settlement it releases the rest. A completed request reports its charge in fluence.billing.charged_usd, in a stream's final usage event, and splits out inference_usd, protection_usd and, for Jev, selector_usd. Supplier refusals and protection blocks leave inference uncharged; structured-output failures are charged.

When a provider reports what it billed, you pay that plus the route's nominal markup, lowered to the request's captured OpenRouter featured price for actual usage when that still covers the bill. A valid bill is not capped by the quote, profile caps, reference price or credit hold, so a charge can land above or below the quote, on a failover provider too. Some providers report cost in the response; for providers that keep a per-request billing record, the router reads it when the request completes, waiting up to 1.5 seconds. Without a reported cost, the serving provider's rates apply. A Gemini reference cache-write rate includes five minutes of cache storage, as OpenRouter bills it.

A charge that followed the provider's cost carries fluence.billing.charge_source: "supplier_cost", and GET /v1/usage/requests/:id adds billing.supplier_cost_charge: source (response or billing_api), quote_usd, and capped_by (market when matching lowered the margin, otherwise null; older records can show reference or hold).

Late supplier bills

If the bill is not readable by settlement, the request settles at quoted rates and fluence.billing.supplier_bill reads pending. After three unsuccessful checks, Fluence charges the model's captured full list price for actual usage, keeping protection fees separate: status fallback with provisional: true, source: "list_price" and inference_usd. The router keeps checking for 30 days, and the original featured amount stays the benchmark. When the bill arrives, the charge moves to the bill plus markup and your balance history shows a "Supplier bill" entry for the difference; billing.supplier_bill.status becomes applied with adjustment_usd, positive for a debit. With no bill in 30 days, the provisional charge stays.

Spend limits apply when a request starts, so a late bill can take spend past a monthly budget or the balance below zero.

Long-context charges

The provider-reported input picks the tier. The hold covers the dearest tier the request's size can reach, at one input token per byte plus 1,024, so a request near a threshold needs credit for the higher rate even when it settles at the base rate. A provider threshold up to 1,024 tokens below the reference's starts the tier there. A provider priced flat over its window is held at the reference's long-context rate once a request can reach it. When a provider's cost matches a long-context rate within 1% and exceeds the reported tier's price by at least 25%, the router records the matching tier.

fluence.billing.price_tier reports {"index": 1, "above_input_tokens": 272000} for the first tier and index: 0 with above_input_tokens: null for the base price. GET /v1/usage/requests/:id reports it as billing.price_tier and request rows as price_tier, null before settlement. The billing ledger marks such a charge "(long-context rate)".

Cancelled requests

A provider can bill a cancelled request its whole generation: in our 26 September measurement, one provider billed every cancel its full prompt and output up to max_tokens, hidden reasoning included. So once your request reaches a provider, a disconnect, your total deadline when shorter than the platform's, or a stream deadline shorter than the platform's ends only your response. The router keeps reading the provider to final usage within platform limits (240 seconds less 10 for settlement; on a stream, 120 seconds to the first response and 45 idle, or your longer timeouts) and charges that verified usage like a completed request: in full past your output limit, up to the credit hold within it. If the provider fails first, output it produced is charged at the estimate; with no output, nothing. A protected request that ends this way is never released and pays its verified inference plus any check already sent. Such requests hold concurrency until they settle.

Output streamed to you is billed even when the stream fails. A delivery cutoff, or your idle timeout when shorter than the platform's, after output starts closes the stream with a request_timeout or idle_timeout event whose fluence.billing status is pending, while the router drains the provider as above. When the provider has already reported final usage and only its closing event is late, your idle timeout ends the stream at that usage.

A stream with tool-call protection is released only after inspection, with its verified usage already read, at the pace your client reads it. If a delivery cutoff, a disconnect or a reader that stalls for ten seconds ends delivery after release, the request is still charged that usage. A protected stream whose provider reported final usage and a stop reason, then ended without its terminal event, is inspected and released as complete.

Estimated charges

A stream that sent output and ends without verified final usage, from a deadline, supplier error, broken or invalid stream, invalid supplier cost or size limit, is stopped and charged an estimate. It keeps its failure status, and its error event carries charge_basis: "estimated". Final usage the provider reported before the break still counts; running totals on content chunks do not. A reported zero output after streamed text is replaced by the estimate.

• Output: the larger of the number of events that carried text and one token per four UTF-8 bytes of answers, reasoning, refusals and tool arguments, at most your output limit.
• Input: the count a Messages stream reports in message_start when nonzero and within the hold, otherwise the request JSON outside images counted with the o200k tokenizer plus 1,000 per image, at most the reserved input.
• Hidden reasoning, for models that reason without streaming it (the OpenAI o-series, GPT-5 and later, gpt-oss and Codex; Claude 3.7 and later; Gemini 2.5 and later; Grok 4 and later, Grok 3 Mini and Grok Code; DeepSeek R1 and V3.1 and later; Qwen3 and later and QwQ; Kimi K2.5 and later; GLM 4.5 and later; MiniMax M1 and later; Magistral; IDs that name thinking or reasoning): the visible output rate times the wait before the first visible output, less a prefill allowance of 3 seconds plus 1 second per 5,000 input tokens, less reasoning already shown. It appears as reasoning tokens.

Without a valid verified bill, the estimate is priced at quoted rates within the inference hold. A completed response with a verified bill pays the bill plus markup, with the counts as estimates.

For other models, an output count larger than the returned text can hold is replaced by the estimate. A reported input above the hold's bound, such as a system prompt the provider added, is charged in full as reported, bill plus markup or quoted rates, even past the credit hold, with charge_basis: "input_limit". fluence.billing.input_over_bound names the bound and the report, and GET /v1/usage/requests/:id reports it as billing.input_over_bound. Fluence counts the report against the provider's health.

Some providers send more output than the limit Fluence forwarded. You receive the whole answer and pay for all of it, bill plus markup, even past the hold; fluence.billing.output_over_limit and billing.output_over_limit name the limit. A charge past the hold can take your balance below zero: requests then get 402 insufficient_balance until a top-up covers it, the console shows what you owe, and you can delete the account once the balance is $0 or above. A report past the model's maximum output is a miscount: the request fails with provider_output_limit, a stream that sent output is charged up to your limit with charge_basis: "capped", and anything else is free.

If unverified usage at the quote exceeds the hold, the request fails with reservation_exceeded and is charged at most the inference hold, with charge_basis: "capped", unless the excess is input the provider reported past the bound, which completes and is charged in full. A verified bill can exceed the hold and still complete. Charged requests read settled in GET /v1/usage/requests/:id. billing_pending means settlement was interrupted and recorded for recovery, which may charge streamed output; settlement_unavailable means the outcome is unconfirmed. Check the request before retrying either.

Payments

Credits are prepaid in USD, nonrefundable under the Fluence terms and never expire. Stripe runs checkout, taxes, card details and fraud checks; a payment under Radar review becomes credit once approved, and inference pauses with 403 payment_review meanwhile. We email a receipt after each top-up. Optional auto top-up needs a saved card and a monthly purchase cap.

While the offer lasts, a first top-up of $10 or more before tax earns a $5 bonus; an invite link can add sign-up credit and a share of the first top-up, up to the limit it shows. Usage spends bonus credit before paid credit, and a refund or dispute of the top-up removes its unused bonus.

GET /v1/account (account:read) returns balance_nano, reserved_nano, bonus_balance_nano, paid_balance_nano, first_topup_offer and concurrency_limit. POST /v1/account/topup (account:write) with amount_cents from 500 to 100,000 returns a Stripe Checkout url valid for 30 minutes, five an hour; another amount returns 400 invalid_amount.

Turning auto top-up on, raising its threshold, amount or cap, and saving or replacing its card need a sign-in from the last 10 minutes, support-chat confirmations included; turning it off and lowering limits work from any session. We email the owner after every change, and an account can save these settings 10 times an hour. Billing details and invoices come from Stripe, 30 reads per 10 minutes per account; invoice lists stay in server memory for up to 30 seconds.

Usage

GET /v1/usage (usage:read, with aliases /v1/usage/requests and /v1/usage/daily) returns stats and request rows priced in charged_nano, 50 per page; /v1/usage/summary returns totals. Filter with from and to (the last 30 days by default, at most 366, else 400 invalid_range), key and model. Page numbers stop at 20 (400 invalid_page): send pagination=cursor with up to 100 rows per limit and pass next_cursor as cursor while has_more is true. GET /v1/usage/requests/:id reports billing.status as pending, settled or not_charged, and 404 request_not_found for an unknown ID.

Exports

POST /v1/usage/exports with account:write and a body of from, to and optional key and model starts a CSV snapshot. Poll GET /v1/usage/exports/:id with usage:read until it is ready, then get a one-minute URL from GET /v1/usage/exports/:id/download; files expire after 24 hours. Cancel with POST /v1/usage/exports/:id/cancel. One export can run at a time, and five can start per hour; an export over its size or time budget fails rather than truncating, so pick a smaller range. GET /v1/usage/exports lists your latest 20. A download answers 404 export_not_found, 409 export_not_ready or 410 export_expired when it cannot be served.

Rows list request_id, created_at, model, status, input_tokens, output_tokens, charged_usd, price_tier (0 for the base price, 1 or higher for a long-context tier), requested_model and routing_profile_id (empty before routing profiles), security_guardrail_id and security_guardrail_revision. New columns are added at the end.

Errors

Each inference response carries x-request-id; find it in Usage. JSON errors contain error.type, error.code and error.message, and inference errors add a top-level request_id; Messages uses the Anthropic error envelope. X-Should-Retry: true marks a refusal that reached no provider, so a retry cannot be billed twice. X-Should-Retry: false means do not resend the request as is: it would fail the same way, or it may already be billed.

A provider's own error arrives as upstream_ plus its status: a provider 400 or 422 returns 400, a 429, 503 or 529 returns 503 (with Retry-After when known), a 408 or 504 returns 504, and anything else 502; a provider that cannot serve the model returns 502. These reach you only after failover runs out. Other codes clients meet:

A named model with no route that can take the request returns no_eligible_route: a 503 with Retry-After while its providers recover or refresh prices, and a 400 with X-Should-Retry: false when no provider serves the model on this endpoint with this request's inputs, limits or privacy policy, or its quotes await price review. A higher supplier price alone never removes a model.

Routing errors