How can I properly interact with the OpenAI API in a loop to prevent 429 Too Many Requests errors?

I’m working on a test project where I need to send data to the Groq API and receive responses in a loop. Below is my current code:

<?php

require 'vendor/autoload.php';

use GuzzleHttp\Client;

// API key for the GroqAPI
$apiKey = 'APIKEY';
$apiUrl = 'https://api.groq.com/openai/v1/chat/completions';

// Load the content of the files
$jsonContent = file_get_contents($jsonFilePath);
$promptContent = file_get_contents($promptFilePath);
$configArray = json_decode($jsonContent, true);

// Function to send API request
function sendChatRequest($userMessage, $systemPrompt, $apiKey, $apiUrl)
{
    $client = new Client([
        'base_uri' => $apiUrl,
        'headers' => [
            'Content-Type' => 'application/json',
            'Authorization' => 'Bearer ' . $apiKey,
        ],
    ]);

    $requestData = [
        'messages' => [
            ['role' => 'user', 'content' => $userMessage],
            ['role' => 'system', 'content' => $systemPrompt],
        ],
        'model' => 'gemma2-9b-it',
        'temperature' => 1,
        'max_tokens' => 1024,
        'top_p' => 1,
        'stream' => false,
        'stop' => null,
    ];

    // Perform the POST request
    $response = $client->post('', ['json' => $requestData]);

    return json_decode($response->getBody(), true);
}

// Process each item
foreach ($configArray as $item) {
    if (isset($item['id'])) {
        $userMessage = json_encode($item);

        $aiResponse = sendChatRequest($userMessage, $promptContent, $apiKey, $apiUrl);

        $generatedContent = $aiResponse['choices'][0]['message']['content'];
        echo $generatedContent;

        sleep(5);  // I added this line to avoid "429 Too Many Requests"
    } else {
        echo "Error: The object does not contain the 'id' key.\n";
    }
}

This is a test project to experiment with looping requests and interacting with the OpenAI API. I added a sleep(5); in an attempt to prevent getting a 429 Too Many Requests error from the API, but it doesn’t seem to resolve the issue completely.

The error I am encountering is:

Client error: `POST https://api.groq.com/openai/v1/chat/completions` resulted in a `429 Too Many Requests`.

While adding the sleep function reduces the frequency of errors, this isn’t the most efficient way to interact with an AI API.

What would be a better approach to handle requests in a loop to prevent hitting rate limits, and ensure the interaction with the AI is smooth and optimized? Should I use retry mechanisms, manage concurrency, or adopt another method?

>Solution :

You can check the official documentation on https://console.groq.com/docs/rate-limits on how to deal with the rate limit. Check the response headers to see the current rate limit you have and act accordingly:

Status code & rate limit headers

We set the following x-ratelimit headers to inform you on current rate limits applicable to the API key and associated organization.

(…)

There is even a retry-after header to indicate, when you can send a new request once the limit has been reached.