Rate & concurrency limits

Concurrency is the number of API requests you can have in progress simultaneously. Your plan sets the number of slots: with 10, an 11th request sent while 10 are still processing gets a rate limit error rather than queueing.

Rate limits vs. concurrency limits

Free tier accounts are limited to 1 concurrent request. Upgrade for multi-threaded workloads. See the pricing table for concurrency limits per plan. cloro applies two types of limit depending on the endpoint:

Limit type	Endpoints affected	How it works
Rate limits	All endpoints (`/v1/*`)	1,000 requests per second per endpoint
Concurrency limits	Monitor endpoints (`/v1/monitor/*`)	Based on your subscription plan (simultaneous requests)

Monitor endpoints are subject to both.

Monitoring concurrency with headers

Each response includes HTTP headers to help you manage your API usage:

Header	Description
`X-Concurrent-Limit`	Total concurrent requests allowed by your plan
`X-Concurrent-Current`	Number of requests currently processing
`X-Concurrent-Remaining`	Available concurrent slots when request was received

On a 20-slot plan with 3 requests in flight:

X-Concurrent-Limit: 20
X-Concurrent-Current: 3
X-Concurrent-Remaining: 17

Monitoring rate limits with headers

All endpoints include rate limit headers in each response:

Header	Description
`X-RateLimit-Limit`	Maximum requests per second allowed (1,000)
`X-RateLimit-Remaining`	Remaining requests available in this second

The limit is per endpoint per API key — each /v1/* path has its own independent 1,000 RPS bucket, and the counter resets every second. After one request to /v1/monitor/chatgpt:

X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 999

Credit headers

Monitor endpoints (/v1/monitor/*) also return the credit balance and the amount charged for each request:

Header	Description
`X-Credits-Remaining`	Number of credits remaining in your account
`X-Credits-Charged`	Number of credits charged for this request

Track spend per request and alert before you run low:

function trackCredits(response) {
  const creditsRemaining = response.headers["x-credits-remaining"];
  const creditsCharged = response.headers["x-credits-charged"];

  console.log(`Credits charged: ${creditsCharged}`);
  console.log(`Credits remaining: ${creditsRemaining}`);

  // Alert if running low
  if (creditsRemaining < 100) {
    console.warn("Low credit balance. Consider topping up");
  }
}

These headers are only returned on sync /v1/monitor/* responses. For async-only workloads, read your balance from GET /v1/credits instead. For how credits are allocated, expire, and are priced, see Billing & credits and Providers.

Latency header

Every response carries X-Latency-Ms — the milliseconds the API spent on the request, from arrival to the start of the response. It excludes network transit, so wall-clock timing on your side is always higher.

X-Latency-Ms: 3420

Async tasks report the equivalent in the task body as latencyMs — see asynchronous requests.

Using headers for optimization

Size each batch against the slots you have left:

function checkConcurrencyUsage(response) {
  const limit = parseInt(response.headers["x-concurrent-limit"]);
  const current = parseInt(response.headers["x-concurrent-current"]);
  const remaining = parseInt(response.headers["x-concurrent-remaining"]);

  console.log(`Concurrency: ${current}/${limit} (${remaining} available)`);

  // Adjust your batch size based on remaining slots
  return Math.min(remaining, 5); // Don't exceed 5 requests per batch
}

Or throttle before you exhaust the per-second budget:

async function checkRateLimit(response) {
  const limit = parseInt(response.headers["x-ratelimit-limit"]);
  const remaining = parseInt(response.headers["x-ratelimit-remaining"]);

  console.log(`Rate limit: ${remaining}/${limit} requests remaining`);

  // If you're running low on requests, wait before continuing
  if (remaining < 50) {
    const waitTime = 1000; // Wait 1 second for the counter to reset
    console.log(`Rate limit nearly exceeded. Waiting ${waitTime}ms...`);
    await new Promise((resolve) => setTimeout(resolve, waitTime));
  }

  return remaining;
}

When you do hit a 429, retry with exponential backoff rather than immediately re-sending:

async function makeRequestWithRetry(requestFn, maxRetries = 3) {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await requestFn();
    } catch (error) {
      if (error.status === 429 && i < maxRetries - 1) {
        const delay = Math.pow(2, i) * 1000; // Exponential backoff
        await new Promise((resolve) => setTimeout(resolve, delay));
        continue;
      }
      throw error;
    }
  }
}

See Error handling for the full 429 error shapes.

Implementation patterns

Pattern 1: Async with webhooks

For large-scale processing, submit every task concurrently (one request per task) and handle results via webhooks — cloro queues them for you, so you don’t have to batch:

import axios from "axios";

const API_KEY = process.env.API_KEY;
const TASK_API = "https://api.cloro.dev/v1/async/task";

async function submitTasks(tasks, webhookUrl) {
  // Send API requests concurrently (one request per task)
  await Promise.all(
    tasks.map((task) =>
      axios.post(
        TASK_API,
        {
          taskType: "CHATGPT",
          webhook: { url: webhookUrl },
          payload: task,
        },
        {
          headers: { Authorization: `Bearer ${API_KEY}` },
        }
      )
    )
  );
}

// Webhook handler (Express.js)
app.post("/webhook-handler", (req, res) => {
  const { task, response } = req.body;
  console.log(`Task ${task.id} completed: ${response.text.slice(0, 100)}...`);

  // Process your result here
  saveResult(task.id, response);

  // Always respond quickly
  res.status(200).send();
});

// Usage
const tasks = [
  { prompt: "Analyze market trends", country: "US" },
  { prompt: "Research competitors", country: "US" },
  // ... hundreds more
];

submitTasks(tasks, "https://your-app.com/webhook-handler");

import requests
import asyncio
import json

API_KEY = "YOUR_API_KEY"
TASK_API = "https://api.cloro.dev/v1/async/task"

async def submit_tasks(tasks, webhook_url):
    # Send API requests concurrently (one request per task)
    submit_tasks = []
    for task in tasks:
        submit_tasks.append(
            requests.post(
                TASK_API,
                json={
                    "taskType": "CHATGPT",
                    "webhook": {"url": webhook_url},
                    "payload": task
                },
                headers={"Authorization": f"Bearer {API_KEY}"}
            )
        )

    # Wait for all submissions to complete
    responses = await asyncio.gather(*[asyncio.to_thread(req) for req in submit_tasks])
    print("All tasks submitted")

# Usage
tasks = [
    {"prompt": "Analyze market trends", "country": "US"},
    {"prompt": "Research competitors", "country": "US"},
    # ... hundreds more
]

asyncio.run(submit_tasks(tasks, "https://your-app.com/webhook-handler"))

# Webhook handler example (Flask)
"""
from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route('/webhook-handler', methods=['POST'])
def handle_webhook():
    data = request.json
    task = data.get('task', {})
    response = data.get('response', {})

    print(f"Task {task.get('id')} completed: {response.get('text', '')[:100]}...")

    # Process your result here
    save_result(task.get('id'), response)

    return '', 200
"""

#!/bin/bash

API_KEY="YOUR_API_KEY"
TASK_API="https://api.cloro.dev/v1/async/task"
WEBHOOK_URL="https://your-app.com/webhook-handler"

# Submit tasks individually (cloro handles concurrency automatically)
submit_task() {
  local prompt="$1"
  local country="$2"

  curl -s -X POST "$TASK_API" \
    -H "Authorization: Bearer $API_KEY" \
    -H "Content-Type: application/json" \
    -d "{
      \"taskType\": \"CHATGPT\",
      \"webhook\": {\"url\": \"$WEBHOOK_URL\"},
      \"payload\": {
        \"prompt\": \"$prompt\",
        \"country\": \"$country\"
      }
    }" &
}

# Example usage - send API requests concurrently
tasks=(
  "Analyze market trends|US"
  "Research competitors|US"
  "Extract pricing data|US"
  "Summarize industry reports|US"
  "Identify key trends|US"
  # ... add more tasks as needed
)

echo "Submitting ${#tasks[@]} tasks..."

# Submit all tasks in background (no need to limit concurrency)
for task_data in "${tasks[@]}"; do
  IFS='|' read -r prompt country <<< "$task_data"
  submit_task "$prompt" "$country"
done

# Wait for all submissions to complete
wait

echo "All tasks submitted successfully"

Pattern 2: Concurrent workers

For real-time processing where you want immediate results, run multiple workers that make direct API calls:

import axios from "axios";

const API_KEY = process.env.API_KEY;
const API_URL = "https://api.cloro.dev/v1/monitor/chatgpt";

async function makeRequest(id, prompt) {
  const start = Date.now();

  try {
    const response = await axios.post(API_URL, {
      prompt: prompt,
      country: "US",
    }, {
      headers: {
        Authorization: `Bearer ${API_KEY}`
      }
    });

    const latency = Date.now() - start;
    console.log(`Request #${id}: Success (${latency}ms)`);

    // Monitor concurrency usage
    const limit = parseInt(response.headers["x-concurrent-limit"]);
    const current = parseInt(response.headers["x-concurrent-current"]);
    const remaining = parseInt(response.headers["x-concurrent-remaining"]);

    return {
      success: true,
      latency,
      data: response.data,
      usage: { limit, current, remaining }
    };

  } catch (error) {
    const latency = Date.now() - start;
    console.log(`Request #${id}: Failed (${latency}ms)`);

    if (error.response?.status === 429) {
      console.log(`Rate limited - ${error.response.headers["retry-after"] || "unknown"} seconds`);
    }

    return { success: false, latency, error: error.message };
  }
}

async function runConcurrentRequests(prompts, concurrency = 10) {
  console.log(`Starting ${prompts.length} requests with ${concurrency} concurrent workers\n`);

  const startTime = Date.now();
  const results = [];
  let requestId = 0;

  // Worker function
  async function worker() {
    while (requestId < prompts.length) {
      const id = ++requestId;
      const result = await makeRequest(id, prompts[requestId - 1]);
      results.push(result);
    }
  }

  // Run concurrent workers
  await Promise.all(
    Array(concurrency).fill(0).map(() => worker())
  );

  const duration = Date.now() - startTime;
  const successful = results.filter(r => r.success).length;
  const rateLimited = results.filter(r => r.error?.includes('Rate limited')).length;

  console.log("\n" + "=".repeat(40));
  console.log(`Total: ${prompts.length}`);
  console.log(`Success: ${successful} (${((successful/prompts.length)*100).toFixed(1)}%)`);
  console.log(`Rate limited: ${rateLimited}`);
  console.log(`Duration: ${(duration/1000).toFixed(1)}s`);
  console.log(`RPS: ${(prompts.length/duration*1000).toFixed(1)}`);
  console.log("=".repeat(40));

  return results;
}

// Usage
const prompts = [
  "What is AI and how does it work?",
  "Explain machine learning basics",
  "What are neural networks?",
  "How does deep learning work?",
  "What is natural language processing?",
  // ... add more prompts as needed
];

runConcurrentRequests(prompts, 5) // Start with conservative concurrency
  .then(results => console.log(`Completed processing`))
  .catch(console.error);

import asyncio
import aiohttp
import time
from typing import List, Dict, Any

class ConcurrentWorker:
    def __init__(self, api_key: str, api_url: str):
        self.api_key = api_key
        self.api_url = api_url

    async def make_request(self, session, id: int, prompt: str):
        start_time = time.time()

        try:
            async with session.post(
                self.api_url,
                json={
                    "prompt": prompt,
                    "country": "US"
                },
                headers={
                    "Authorization": f"Bearer {self.api_key}"
                }
            ) as response:
                data = await response.json()
                latency = (time.time() - start_time) * 1000

                # Monitor concurrency usage
                limit = int(response.headers.get("x-concurrent-limit", 0))
                current = int(response.headers.get("x-concurrent-current", 0))
                remaining = int(response.headers.get("x-concurrent-remaining", 0))

                print(f"Request #{id}: Success ({latency:.0f}ms)")

                return {
                    "success": True,
                    "latency": latency,
                    "data": data,
                    "usage": {"limit": limit, "current": current, "remaining": remaining}
                }

        except Exception as e:
            latency = (time.time() - start_time) * 1000
            print(f"Request #{id}: Failed ({latency:.0f}ms)")

            return {
                "success": False,
                "latency": latency,
                "error": str(e)
            }

    async def run_concurrent_requests(self, prompts: List[str], concurrency: int = 10):
        print(f"Starting {len(prompts)} requests with {concurrency} concurrent workers\n")

        start_time = time.time()
        results = []
        request_id = 0

        async with aiohttp.ClientSession() as session:
            async def worker():
                nonlocal request_id
                while request_id < len(prompts):
                    id = request_id + 1
                    result = await self.make_request(session, id, prompts[request_id])
                    results.append(result)
                    request_id += 1

            # Run concurrent workers
            await asyncio.gather(*[worker() for _ in range(concurrency)])

        duration = time.time() - start_time
        successful = sum(1 for r in results if r["success"])
        rate_limited = sum(1 for r in results if "rate limited" in r.get("error", "").lower())

        print("\n" + "=" * 40)
        print(f"Total: {len(prompts)}")
        print(f"Success: {successful} ({(successful/len(prompts)*100):.1f}%)")
        print(f"Rate limited: {rate_limited}")
        print(f"Duration: {duration:.1f}s")
        print(f"RPS: {len(prompts)/duration:.1f}")
        print("=" * 40)

        return results

# Usage
async def main():
    worker = ConcurrentWorker("YOUR_API_KEY", "https://api.cloro.dev/v1/monitor/chatgpt")

    prompts = [
        "What is AI and how does it work?",
        "Explain machine learning basics",
        "What are neural networks?",
        # ... add more prompts as needed
    ]

    results = await worker.run_concurrent_requests(prompts, concurrency=5)
    print(f"Completed processing")

if __name__ == "__main__":
    asyncio.run(main())

#!/bin/bash

API_KEY="YOUR_API_KEY"
API_URL="https://api.cloro.dev/v1/monitor/chatgpt"
TOTAL_REQUESTS=10
CONCURRENCY=5

# Function to make a single request
make_request() {
  local id=$1
  local prompt=$2
  local start_time=$(date +%s%3N)

  response=$(curl -s -w "\n%{http_code}\n%{time_total}\n%{x-concurrent-limit}\n%{x-concurrent-current}\n%{x-concurrent-remaining}" \
    -X POST "$API_URL" \
    -H "Authorization: Bearer $API_KEY" \
    -H "Content-Type: application/json" \
    -d "{\"prompt\":\"$prompt\",\"country\":\"US\"}" \
    2>/dev/null)

  # Parse response
  http_code=$(echo "$response" | sed -n '2p')
  time_total=$(echo "$response" | sed -n '3p')
  limit=$(echo "$response" | sed -n '4p')
  current=$(echo "$response" | sed -n '5p')
  remaining=$(echo "$response" | sed -n '6p')
  body=$(echo "$response" | sed -n '1p')

  latency=$(echo "$time_total * 1000" | bc)

  if [[ "$http_code" == "200" ]]; then
    echo "Request #$id: Success (${latency%.*}ms) - Usage: $current/$limit ($remaining available)"
  else
    echo "Request #$id: Failed (${latency%.*}ms) - HTTP $http_code"
  fi
}

# Worker function that processes multiple requests
worker() {
  local worker_id=$1
  local max_requests=$2

  while true; do
    # Get next request ID from shared counter
    id=$(wget -qO- "http://localhost:8080/next-id" 2>/dev/null || echo "0")

    if [[ $id -gt $max_requests ]]; then
      break
    fi

    prompt="Query $id: What are the benefits of concurrent processing?"
    make_request "$id" "$prompt"
  done
}

# Simple ID counter using files (you could use Redis or a database)
echo "1" > /tmp/next_id

# Start workers
for ((i=1; i<=CONCURRENCY; i++)); do
  worker $i $TOTAL_REQUESTS &
done

# Wait for all workers to complete
wait

echo -e "\n$(printf '=%.0s' {1..40})"
echo "Load test completed"

# Cleanup
rm -f /tmp/next_id

Common questions

Why am I getting 429 rate limit errors?

Two causes: Concurrency limit exceeded (monitor endpoints only) — more simultaneous requests than your plan’s slots. Watch X-Concurrent-Remaining, queue requests in your application, retry with exponential backoff, or upgrade your plan. Rate limit exceeded (all endpoints) — over 1,000 requests per second to a single endpoint. Watch X-RateLimit-Remaining, spread requests over time, or move non-time-sensitive work to the async queue. See the error handling guide for the error responses.

How do I check my concurrency limit?

Read X-Concurrent-Limit on any monitor response, or call the async status endpoint for your account’s concurrency stats.

Can I increase my concurrency limit?

Yes. Self-serve plans can be upgraded directly in the dashboard — the new limit applies immediately, no support ticket required. If you need concurrency above the highest self-serve tier, email info@cloro.dev for an enterprise quote.

Does higher concurrency delay my logs or dashboards?

No. Dashboard log ingestion runs independently from request processing. If logs look delayed during heavy load, the cause is usually batching on the dashboard side, not concurrency — entries normally surface within a minute.

Can I burst above my concurrency limit?

No. The limit is hard — the (N+1)th simultaneous request gets a 429 immediately rather than queueing. Use the async API if you want cloro to handle queueing for you instead of managing burst capacity yourself.

What’s the best way to handle large batches of requests?

For non-time-sensitive batches, use Pattern 1: Async with webhooks: send everything concurrently, let cloro queue it, and take results on the webhook. When you need results in real time, use Pattern 2: Concurrent workers: stay within your plan’s concurrency limit, watch X-Concurrent-Remaining, and back off exponentially on 429s. Either way, submit through the batch endpoint rather than one call per task. See how to submit many requests in one call for its limits.

Introduction

Guides

Integrations

Rate & concurrency limits

Rate limits vs. concurrency limits

Monitoring concurrency with headers

Monitoring rate limits with headers

Credit headers

Latency header

Using headers for optimization

Implementation patterns

Pattern 1: Async with webhooks

Pattern 2: Concurrent workers

Common questions

Why am I getting 429 rate limit errors?

How do I check my concurrency limit?

Can I increase my concurrency limit?

Does higher concurrency delay my logs or dashboards?

Can I burst above my concurrency limit?

What’s the best way to handle large batches of requests?

​Rate limits vs. concurrency limits

​Monitoring concurrency with headers

​Monitoring rate limits with headers

​Credit headers

​Latency header

​Using headers for optimization

​Implementation patterns

​Pattern 1: Async with webhooks

​Pattern 2: Concurrent workers

​Common questions

​Why am I getting 429 rate limit errors?

​How do I check my concurrency limit?

​Can I increase my concurrency limit?

​Does higher concurrency delay my logs or dashboards?

​Can I burst above my concurrency limit?

​What’s the best way to handle large batches of requests?

Rate limits vs. concurrency limits

Monitoring concurrency with headers

Monitoring rate limits with headers

Credit headers

Latency header

Using headers for optimization

Implementation patterns

Pattern 1: Async with webhooks

Pattern 2: Concurrent workers

Common questions

Why am I getting 429 rate limit errors?

How do I check my concurrency limit?

Can I increase my concurrency limit?

Does higher concurrency delay my logs or dashboards?

Can I burst above my concurrency limit?

What’s the best way to handle large batches of requests?