Rate and spend limits

Keiro rate limits are Keiro account limits. They are calculated for the public eb1 model you request. Public limits describe customer-visible capacity, not private serving architecture or vendor economics.

Copy Page https://docs.keirolabs.ai/rate-and-spend-limits.md

Paid beta accounts can have request, token, concurrency, model-access, and spend limits. Limits are account-specific and may change during the beta. The console and the response headers on API calls are the source of truth for your current effective limits.

When a rate limit is exceeded, the API returns HTTP 429. Retriable denials carry a Retry-After header; wait at least that many seconds before retrying. Denials that cannot succeed on retry say so instead — see Deterministic denials.

How limits work#

A request is admitted only when every applicable limit allows it. The main limit dimensions are:

Keiro may enforce input-token and output-token buckets separately for eb1 models. When those buckets are present, they are Keiro account buckets for the public eb1 request, not upstream provider limits.

• requests per minute or day
• tokens per minute or day
• input tokens per minute or day
• output tokens per minute or day
• concurrent requests
• maximum context and output size
• monthly or daily spend
• maximum estimated exposure for a single request
• model access and streaming access

Account plans#

Your org plan defines the outer limits for:

Invite-only paid beta plans remain separate from self-serve usage tiers. Those invite-only plans are named by account type rather than numbered usage tiers:

• allowed models
• requests per minute or day
• input and output tokens per minute or day
• concurrent requests
• maximum context and output size
• monthly spend
• maximum estimated exposure for a single request
• whether streaming is enabled
• how many API keys can be active

Self-serve usage tiers use the Free plus Tier 1-5 model below, and numbered usage tiers are advanced from settled paid billing history. Some higher tiers may require capacity review before activation. These defaults are static account defaults for eb1-preview; the console and headers remain authoritative for your account's current effective limits.

Self-serve usage tiers are not open during the invite-only beta; the table below shows the defaults that will apply when signup opens. To use the API today, request access.

Self-serve usage tiers#

The current default self-serve limits for eb1-preview are:

Key overrides#

Individual API keys may have narrower limits than the org plan. A key cannot raise the limits of the org it belongs to.

If a key has no explicit override, it inherits the org plan. If a key is limited to specific models, model listing and request admission only show and allow the models that key can use.

Response headers#

Responses produced by the gateway include rate-limit headers describing the effective limits that applied to the call. Header names are case-insensitive; HTTP/2 and HTTP/3 deliver them lowercase on the wire, so match names case-insensitively.

x-ratelimit-reset-* values are durations such as 250ms or 56s. A dimension that is unlimited for your account emits no headers, and a fully unlimited request emits no x-ratelimit-* headers at all. When one of your account limits denies a request with 429, the response carries the headers named for the denied dimension and adds x-ratelimit-scope: a request-limit denial uses the -requests headers, an input- or output-token-limit denial uses the -input-tokens or -output-tokens headers (token limits are enforced per channel, so a denial never uses the combined -tokens names), a concurrency denial uses the -concurrency headers, and a spend-cap denial uses x-ratelimit-limit-spend. A per-request token-budget denial carries no token headers; its configured budget is in error.details.limit (dimension token_budget). A value the gateway does not know at denial time is omitted, never guessed. Two other 429 classes are the exception: a shared-capacity denial (capacity_exceeded) and a 429 passed through from an upstream provider carry no x-ratelimit-* headers, not even x-ratelimit-scope -- only the retry signals (Retry-After and retry-after-ms when retriable, x-should-retry: false on the insufficient_quota echo). Tell those apart by code, not by the presence of rate-limit headers.

These standing headers are promised on gateway-origin responses only: responses produced by the gateway itself, whether success or error. Responses generated at the network edge (for example a 502 returned while the gateway is restarting) may carry none of them.

Treat these as operational hints for the current request and account. Do not build client logic that assumes a public eb1 model is backed by a specific serving implementation.

Retry-After#

Retriable rate-limited (429) responses on authenticated API requests, and 503 and 504 responses from the gateway, carry a Retry-After header with a whole number of seconds (minimum 1). Wait at least that long before retrying; a client that always honors Retry-After never compounds an overload. Retriable denials also carry retry-after-ms, the same horizon in milliseconds with fractional precision; prefer it when both are present.

How the value is computed depends on the failure class:

• Request and token limits: computed from the limiter's refill state, so it reflects when capacity actually returns.
• Concurrency limits: a short randomized interval, so simultaneous retries spread out instead of colliding again.
• Daily and monthly spend caps: the time until the cap's UTC boundary — the start of the next UTC day for a daily cap, the first day of the next month (UTC) for a monthly cap — so the value can span many hours or days. Raising the cap in your spend settings takes effect sooner than waiting it out.
• Shared capacity and platform load: when shared capacity is expected to recover.
• 503 and 504 responses: a fixed short interval, or the upstream-provided value when one exists. 502 responses carry Retry-After only when the upstream supplied one.

Deterministic denials#

Some denials cannot succeed on retry because the request itself exceeds a cap that never self-clears: its token estimate is larger than a full minute of your token budget, it exceeds a per-request token budget, or its estimated cost exceeds a per-request spend cap. A spend cap set to $0.00 behaves the same way — no amount of waiting helps.

A request whose token estimate exceeds a full minute of your token budget is rejected as a 400 invalid_request_error with code request_exceeds_capacity and no retry header of any kind: it is a request problem, not a rate problem. Reduce the input or max_output_tokens, or raise the limit, and resend.

The remaining deterministic denials (per-request token budget, per-request spend cap, $0.00 cap) stay 429 and carry x-should-retry: false and "retryable": false in the error body; their message says what was measured and which knob to change. OpenAI-compatible SDKs honor x-should-retry and stop auto-retrying these responses without any client change. For one release the per-request token-budget and spend-cap denials still carry a Retry-After header for compatibility with clients that key on it; a $0.00 cap denial carries no retry header at all. Treat x-should-retry and retryable as authoritative, and expect Retry-After to drop from all deterministic denials in the next contract release.

Where limits are disclosed#

Limits are visible on three surfaces:

When surfaces disagree, the response headers win for the current moment; the configured values describe your steady-state entitlement.

• Response headers: the effective per-request values after any temporary compression (for example platform load). Authoritative for what your next request can do right now.
• GET /v1/models: the configured static limits for your account and model.
• The console and the tier table above: configured static defaults.

Token accounting#

Keiro bills and reports usage from canonical Keiro token accounting. For eb1 variants, the public model you request is the billing unit; the usage summary does not expose private serving architecture, non-public vendor data, or internal serving choices.

For each completed request, finalized usage is assembled from normalized Keiro request and response usage:

Provider-reported token counts can be used as evidence for reconciliation, but customer-visible usage and spend are based on the finalized Keiro usage record. If a paid non-BYOK request cannot be priced, it fails before provider dispatch instead of silently billing at zero.

• input tokens
• output tokens
• reasoning tokens when a model reports them
• the public model or eb1 variant
• the active price-book version
• the billing mode for your account

Spend exposure#

Spend checks happen before dispatch using conservative estimates. They are not an exact reservation of future output tokens: streaming and reasoning output are known only after completion. After the request finishes, finalized canonical usage settles the spend record.

If an org or key spend limit is exceeded, the API returns HTTP 429 with a generic rate-limit error. The error omits private billing details and sensitive operational context.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

from getpass import getpass

from openai import OpenAI

# The SDK already retries 429 and 5xx responses with jittered exponential
# backoff and honors Retry-After; max_retries is the only knob most
# applications need.
client = OpenAI(
    base_url="https://api.keirolabs.ai/v1",
    api_key=getpass("Keiro API key: "),
    max_retries=5,
)

response = client.chat.completions.create(
    model="eb1-efficient-preview",
    messages=[{"role": "user", "content": "Say hello in one sentence."}],
)

print(response.choices[0].message.content)

Use client.with_options(max_retries=0) on a call when you would rather fail fast than wait out a backoff. To observe the horizon yourself, catch openai.RateLimitError and read exc.response.headers["retry-after"].

Spend-limit changes are handled through support during paid beta. Do not share API keys in support tickets.

Related pages#

• Usage and billing
• Errors
• API keys
• Models