Rate limits
Limits protect the service and every other customer. They apply per API key and are set per account.
What is limited
- Requests per minute, per key.
- Tokens per minute, per key.
- Concurrent streaming requests, per account.
Your account's current limits are shown in the console. If you need more, contact support with your expected volume.
When you hit a limit
The request is rejected with 429 rate_limited and no tokens are generated or billed. Read the headers to decide when to retry:
HTTP/2 429retry-after: 12x-ratelimit-limit-requests: 60x-ratelimit-remaining-requests: 0x-ratelimit-reset-requests: 12sHandling 429
Wait at least the number of seconds in retry-after before the next attempt. If you send requests in parallel, keep a shared queue per key so one burst does not trigger the limit for the whole service.
Limits can change as capacity grows. The headers are authoritative for the request they accompany.