Rate Limits

Rate-limit behavior, response headers and retry handling.

---

Current Limits

Resource	Limit
Requests per minute (RPM)	60
Tokens per minute (TPM)	2,000,000

These limits apply per API key. Each key you create has its own independent rate limit.

---

How Token Limits Work

When a request includes max_tokens, TensorX reserves that value against the key's TPM allowance while the request is in flight. This is a concurrency reservation, not billed usage, and it is released when the request completes.

When max_tokens is omitted, TensorX applies an internal reservation estimate. Omitting the field is the normal choice when the client does not need to enforce a smaller response budget.

Reservation impact

A high explicit max_tokens value can reserve most of the TPM allowance even when the generated response is short. Concurrent requests can then receive token rate-limit errors.

Explicit response budgets

Set max_tokens only when the application needs a specific upper bound. There is no universal value by use case. Query GET /v1/model/info and inspect model_info.max_output_tokens for the selected model, then choose an application-specific budget at or below that value.

Some clients set max_tokens to the full context-window size. This can reserve most of a key's TPM allowance before the model generates a response. If requests receive TPM 429 errors, remove that client default where possible or set an intentional application limit.

---

Need Higher Limits?

Shared rate limits scale with your total consumed spend - the cumulative dollar value of tokens you have actually used on the platform. This is not the same as your credit balance. Depositing credits does not trigger a tier upgrade; only actual usage counts.

Tiers are applied manually after consumed spend crosses a threshold. Contact support to request a tier change. Tiers affect rate limits but not direct throughput (TPS), and do not guarantee throughput:

Total spend (USD)	Requests per minute	Tokens per minute
Default	60	2,000,000
$250	up to 600	20,000,000
$1,000	2,500	30,000,000
$2,500	5,000	180,000,000

Beyond the top tier, or when you need guaranteed throughput, we move you to dedicated inference.

For higher limits, contact [e-mail].

Include in your request:

• Your use case and expected scale

• Current bottlenecks you're experiencing

• Your account email

Workloads beyond the top shared tier, or workloads that require guaranteed throughput, use dedicated inference.

---

What Happens When You Hit a Limit

When you exceed a limit, the API returns HTTP 429 Too Many Requests with these headers:

Header	Description
retry-after	Seconds to wait before you retry
x-ratelimit-reset	Time when the limit resets (UTC)

The response body identifies the limit that you hit:

{
  "error": {
    "message": "Rate limit exceeded for api_key: xxx...xxx. Limit type: requests. Current limit: 60, Remaining: 0. Limit resets at: 2026-03-21 20:38:07 UTC",
    "type": "rate_limit_error",
    "param": null,
    "code": "429",
    "reason": "rate_limit_requests"
  }
}

Reason Values

Branch on error.reason to select the correct remediation:

Reason	Limit	Recommended action
rate_limit_requests	Requests per minute	Wait for retry-after seconds. Batch calls where possible.
rate_limit_tokens	Tokens per minute	Reduce max_tokens or shorten your prompts.
rate_limit_concurrent	Concurrent requests	Reduce the number of parallel requests.

The reason field is not present when the cause is unknown. Fall back to the retry-after header in that case.

---

Checking Your Rate Limit Status

Every API response includes headers showing your current usage:

Header	Example Value	Description
x-ratelimit-limit-requests	60	Max requests per minute
x-ratelimit-remaining-requests	45	Requests remaining this minute
x-ratelimit-limit-tokens	2000000	Max tokens per minute
x-ratelimit-remaining-tokens	1850000	Tokens remaining this minute

Check these headers to monitor your usage before hitting limits.

---

Handling Rate Limits

When you receive a 429:

1. Read the retry-after header.

2. Wait that many seconds before you retry.

3. Apply exponential backoff if the error repeats.

4. Monitor the x-ratelimit-remaining-* headers on successful responses to stay under your limits.

Retry with Exponential Backoff

The best practice is to retry with increasing delays:

Python

import time
from openai import OpenAI, RateLimitError

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.tensorx.ai/v1"
)

def call_with_retry(messages, max_retries=5):
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(
                model="z-ai/glm-5.2",
                messages=messages
            )
        except RateLimitError:
            if attempt == max_retries - 1:
                raise
            wait_time = 2 ** attempt  # 1, 2, 4, 8, 16 seconds
            print(f"Rate limited. Waiting {wait_time}s...")
            time.sleep(wait_time)

JavaScript

async function callWithRetry(messages, maxRetries = 5) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await client.chat.completions.create({
        model: 'z-ai/glm-5.2',
        messages
      });
    } catch (error) {
      if (error.status !== 429 || attempt === maxRetries - 1) {
        throw error;
      }
      const waitTime = Math.pow(2, attempt) * 1000;
      console.log(`Rate limited. Waiting ${waitTime}ms...`);
      await new Promise(r => setTimeout(r, waitTime));
    }
  }
}

Spread Out Requests

If you're making many requests, add small delays between them:

import time

for item in items:
    response = make_api_call(item)
    time.sleep(1)  # Wait 1 second between requests

---

Tips to Stay Under Limits

Tip	How It Helps
Batch similar requests	Fewer API calls
Cache responses	Don't repeat identical queries
Use streaming	One request for long outputs
Set explicit response budgets only when needed	Omit max_tokens or set an application-specific limit. See How Token Limits Work.
Queue requests	Smooth out traffic spikes

---

Monitoring Your Usage

Check your request patterns in your Usage Dashboard:

• See request counts over time

• Identify peak usage periods

• Spot patterns that might cause rate limiting