> For the complete documentation index, see [llms.txt](https://docs.tensorx.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.tensorx.ai/api-reference/rate-limits.md).

# Rate Limits

Understanding API rate limits and how to manage them effectively.

***

## Current Limits

| Resource                      | Limit     |
| ----------------------------- | --------- |
| **Requests per minute (RPM)** | 60        |
| **Tokens per minute (TPM)**   | 2,000,000 |

These limits apply **per API key**. Each key you create has its own independent rate limit.

***

## How Token Limits Work

When a request arrives, TensorX **reserves** your `max_tokens` value against your TPM allowance while the request is in flight. This is a reservation, not actual consumption - the reserved amount is released back once the request completes.

### Why This Matters

If `max_tokens` is set very high (e.g. `1048576`), each request reserves \~1M tokens against your TPM allowance — even if the actual output is only a few thousand tokens.

**Example:** With a 2M TPM allowance and `max_tokens: 1048576`, only \~2 concurrent requests fit before hitting the TPM limit.

### Recommended Values

Set `max_tokens` to a realistic value so each request only reserves what it plausibly needs:

| Use case          | Suggested `max_tokens` |
| ----------------- | ---------------------- |
| Coding tasks      | 8192–16384             |
| Short responses   | 4096                   |
| Long-form content | 16384–32768            |

{% hint style="warning" %}
Some editors and IDEs (e.g. Zed) default `max_tokens` to the full context window size (1,048,576). If you're getting TPM 429 errors after just a few requests, check your client's `max_tokens` setting.
{% endhint %}

***

## Need Higher Limits?

Our standard limits support most development and production workloads. We're committed to supporting growing projects while ensuring platform stability for all users.

Shared rate limits scale with your **total consumed spend** - the cumulative dollar value of tokens you have actually used on the platform. This is not the same as your credit balance. Depositing credits does not trigger a tier upgrade; only actual usage counts.

Tiers are applied manually once your consumed spend crosses the threshold. Should you wish to avail of a higher tier, ensure you have used your spend as noted below and reach out to our support team. These tiers will affect your rate limits, but will not affect direct throughput (TPS). The tiers below are best-effort and are not guaranteed throughput:

| Total spend (USD) | Requests per minute | Tokens per minute |
| ----------------- | ------------------- | ----------------- |
| Default           | 60                  | 2,000,000         |
| $250              | up to 600           | 20,000,000        |
| $1,000            | 2,500               | 30,000,000        |
| $2,500            | 5,000               | 180,000,000       |

Beyond the top tier, or when you need guaranteed throughput, we move you to dedicated inference.

**For enterprise workloads requiring higher limits:**

📧 **Contact us**: <support@tensorx.ai>

Include in your request:

* Your use case and expected scale
* Current bottlenecks you're experiencing
* Your account email

{% hint style="info" %}
**Enterprise clients** needing volume beyond the top tier, or guaranteed throughput rather than best-effort shared limits, move to dedicated inference. We'll work with you to find the right balance for your needs.
{% endhint %}

***

## What Happens When You Hit a Limit

When you exceed rate limits, the API returns a `429 Too Many Requests` error:

```json
{
  "error": {
    "message": "Rate limit exceeded for api_key: xxx...xxx. Limit type: requests. Current limit: 60, Remaining: 0. Limit resets at: 2026-03-21 20:38:07 UTC",
    "type": "None",
    "param": "None",
    "code": "429"
  }
}
```

The error message includes:

* Which limit you hit (requests or tokens)
* Your current limit and remaining count
* When the limit resets

***

## Checking Your Rate Limit Status

Every API response includes headers showing your current usage:

| Header                                   | Example Value | Description                    |
| ---------------------------------------- | ------------- | ------------------------------ |
| `x-ratelimit-api_key-limit-requests`     | 60            | Max requests per minute        |
| `x-ratelimit-api_key-remaining-requests` | 45            | Requests remaining this minute |
| `x-ratelimit-api_key-limit-tokens`       | 2000000       | Max tokens per minute          |
| `x-ratelimit-api_key-remaining-tokens`   | 1850000       | Tokens remaining this minute   |

Check these headers to monitor your usage before hitting limits.

***

## Handling Rate Limits

### Retry with Exponential Backoff

The best practice is to retry with increasing delays:

{% tabs %}
{% tab title="Python" %}

```python
import time
from openai import OpenAI, RateLimitError

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.tensorx.ai/v1"
)

def call_with_retry(messages, max_retries=5):
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(
                model="deepseek/deepseek-chat-v3.1",
                messages=messages
            )
        except RateLimitError:
            if attempt == max_retries - 1:
                raise
            wait_time = 2 ** attempt  # 1, 2, 4, 8, 16 seconds
            print(f"Rate limited. Waiting {wait_time}s...")
            time.sleep(wait_time)
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
async function callWithRetry(messages, maxRetries = 5) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await client.chat.completions.create({
        model: 'deepseek/deepseek-chat-v3.1',
        messages
      });
    } catch (error) {
      if (error.status !== 429 || attempt === maxRetries - 1) {
        throw error;
      }
      const waitTime = Math.pow(2, attempt) * 1000;
      console.log(`Rate limited. Waiting ${waitTime}ms...`);
      await new Promise(r => setTimeout(r, waitTime));
    }
  }
}
```

{% endtab %}
{% endtabs %}

### Spread Out Requests

If you're making many requests, add small delays between them:

```python
import time

for item in items:
    response = make_api_call(item)
    time.sleep(1)  # Wait 1 second between requests
```

***

## Tips to Stay Under Limits

| Tip                             | How It Helps                                                                                                                                                   |
| ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Batch similar requests**      | Fewer API calls                                                                                                                                                |
| **Cache responses**             | Don't repeat identical queries                                                                                                                                 |
| **Use streaming**               | One request for long outputs                                                                                                                                   |
| **Set appropriate max\_tokens** | Set `max_tokens` to a realistic value (e.g. 8192) to avoid exhausting your token limit prematurely. See [How Token Limits Work](#how-token-limits-work) above. |
| **Queue requests**              | Smooth out traffic spikes                                                                                                                                      |

***

## Monitoring Your Usage

Check your request patterns in your [Usage Dashboard](https://app.tensorx.ai/dashboard/usage):

* See request counts over time
* Identify peak usage periods
* Spot patterns that might cause rate limiting


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.tensorx.ai/api-reference/rate-limits.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.