> For the complete documentation index, see [llms.txt](https://docs.tensorx.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.tensorx.ai/api-reference/rate-limits.md).

# Rate Limits

Rate-limit behavior, response headers and retry handling.

***

## Current Limits

| Resource                      | Limit     |
| ----------------------------- | --------- |
| **Requests per minute (RPM)** | 60        |
| **Tokens per minute (TPM)**   | 2,000,000 |

These limits apply **per API key**. Each key you create has its own independent rate limit.

***

## How Token Limits Work

When a request includes `max_tokens`, TensorX reserves that value against the key's TPM allowance while the request is in flight. This is a concurrency reservation, not billed usage, and it is released when the request completes.

When `max_tokens` is omitted, TensorX applies an internal reservation estimate. Omitting the field is the normal choice when the client does not need to enforce a smaller response budget.

### Reservation impact

A high explicit `max_tokens` value can reserve most of the TPM allowance even when the generated response is short. Concurrent requests can then receive token rate-limit errors.

### Explicit response budgets

Set `max_tokens` only when the application needs a specific upper bound. There is no universal value by use case. Query `GET /v1/model/info` and inspect `model_info.max_output_tokens` for the selected model, then choose an application-specific budget at or below that value.

{% hint style="warning" %}
Some clients set `max_tokens` to the full context-window size. This can reserve most of a key's TPM allowance before the model generates a response. If requests receive TPM 429 errors, remove that client default where possible or set an intentional application limit.
{% endhint %}

***

## Need Higher Limits?

Shared rate limits scale with your **total consumed spend** - the cumulative dollar value of tokens you have actually used on the platform. This is not the same as your credit balance. Depositing credits does not trigger a tier upgrade; only actual usage counts.

Tiers are applied manually after consumed spend crosses a threshold. Contact support to request a tier change. Tiers affect rate limits but not direct throughput (TPS), and do not guarantee throughput:

| Total spend (USD) | Requests per minute | Tokens per minute |
| ----------------- | ------------------- | ----------------- |
| Default           | 60                  | 2,000,000         |
| $250              | up to 600           | 20,000,000        |
| $1,000            | 2,500               | 30,000,000        |
| $2,500            | 5,000               | 180,000,000       |

Beyond the top tier, or when you need guaranteed throughput, we move you to dedicated inference.

For higher limits, contact <support@tensorx.ai>.

Include in your request:

* Your use case and expected scale
* Current bottlenecks you're experiencing
* Your account email

Workloads beyond the top shared tier, or workloads that require guaranteed throughput, use dedicated inference.

***

## What Happens When You Hit a Limit

When you exceed a limit, the API returns HTTP `429 Too Many Requests` with these headers:

| Header              | Description                      |
| ------------------- | -------------------------------- |
| `retry-after`       | Seconds to wait before you retry |
| `x-ratelimit-reset` | Time when the limit resets (UTC) |

The response body identifies the limit that you hit:

```json
{
  "error": {
    "message": "Rate limit exceeded for api_key: xxx...xxx. Limit type: requests. Current limit: 60, Remaining: 0. Limit resets at: 2026-03-21 20:38:07 UTC",
    "type": "rate_limit_error",
    "param": null,
    "code": "429",
    "reason": "rate_limit_requests"
  }
}
```

### Reason Values

Branch on `error.reason` to select the correct remediation:

| Reason                  | Limit               | Recommended action                                          |
| ----------------------- | ------------------- | ----------------------------------------------------------- |
| `rate_limit_requests`   | Requests per minute | Wait for `retry-after` seconds. Batch calls where possible. |
| `rate_limit_tokens`     | Tokens per minute   | Reduce `max_tokens` or shorten your prompts.                |
| `rate_limit_concurrent` | Concurrent requests | Reduce the number of parallel requests.                     |

{% hint style="info" %}
The `reason` field is not present when the cause is unknown. Fall back to the `retry-after` header in that case.
{% endhint %}

***

## Checking Your Rate Limit Status

Every API response includes headers showing your current usage:

| Header                           | Example Value | Description                    |
| -------------------------------- | ------------- | ------------------------------ |
| `x-ratelimit-limit-requests`     | 60            | Max requests per minute        |
| `x-ratelimit-remaining-requests` | 45            | Requests remaining this minute |
| `x-ratelimit-limit-tokens`       | 2000000       | Max tokens per minute          |
| `x-ratelimit-remaining-tokens`   | 1850000       | Tokens remaining this minute   |

Check these headers to monitor your usage before hitting limits.

***

## Handling Rate Limits

When you receive a 429:

1. Read the `retry-after` header.
2. Wait that many seconds before you retry.
3. Apply exponential backoff if the error repeats.
4. Monitor the `x-ratelimit-remaining-*` headers on successful responses to stay under your limits.

### Retry with Exponential Backoff

The best practice is to retry with increasing delays:

{% tabs %}
{% tab title="Python" %}

```python
import time
from openai import OpenAI, RateLimitError

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.tensorx.ai/v1"
)

def call_with_retry(messages, max_retries=5):
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(
                model="z-ai/glm-5.2",
                messages=messages
            )
        except RateLimitError:
            if attempt == max_retries - 1:
                raise
            wait_time = 2 ** attempt  # 1, 2, 4, 8, 16 seconds
            print(f"Rate limited. Waiting {wait_time}s...")
            time.sleep(wait_time)
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
async function callWithRetry(messages, maxRetries = 5) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await client.chat.completions.create({
        model: 'z-ai/glm-5.2',
        messages
      });
    } catch (error) {
      if (error.status !== 429 || attempt === maxRetries - 1) {
        throw error;
      }
      const waitTime = Math.pow(2, attempt) * 1000;
      console.log(`Rate limited. Waiting ${waitTime}ms...`);
      await new Promise(r => setTimeout(r, waitTime));
    }
  }
}
```

{% endtab %}
{% endtabs %}

### Spread Out Requests

If you're making many requests, add small delays between them:

```python
import time

for item in items:
    response = make_api_call(item)
    time.sleep(1)  # Wait 1 second between requests
```

***

## Tips to Stay Under Limits

| Tip                                                | How It Helps                                                                                                 |
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| **Batch similar requests**                         | Fewer API calls                                                                                              |
| **Cache responses**                                | Don't repeat identical queries                                                                               |
| **Use streaming**                                  | One request for long outputs                                                                                 |
| **Set explicit response budgets only when needed** | Omit `max_tokens` or set an application-specific limit. See [How Token Limits Work](#how-token-limits-work). |
| **Queue requests**                                 | Smooth out traffic spikes                                                                                    |

***

## Monitoring Your Usage

Check your request patterns in your [Usage Dashboard](https://app.tensorx.ai/dashboard/usage):

* See request counts over time
* Identify peak usage periods
* Spot patterns that might cause rate limiting


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.tensorx.ai/api-reference/rate-limits.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.