is my curl_multi_exec "ALTERNATOR" script realllllly working?

[ChenXiu]

A company allows me to curl their API for up to 3 sku numbers at a time (e.g. appending "sku[]=12345&sku[]=36545&sku[]=88775" to their URL).

To exceed their "3 sku limit," they provided me a 2nd API key.

The following script is supposed to divide my total number of skus in half, and curl both halves in parallel (half using one API key, half using the other).

🤢 I have a feeling that my script isn't really properly dividing my skus amongst my API url/key pairs! 🤢

<?php
$sku_numbers = array(
'12345',
'33333',
'98745',
'44444',
'11111',
'05054'
)
	
$key1 = 'first_SECRET_key';
$key2 = 'second_SECRET_key';
$keycount = 0;
foreach(array_chunk($sku_numbers,3) as $chunk) {
	$api_KEY = $key1;
	$keycount++;
	if($keycount % 2 != 0) {
		$api_KEY = $key2;
	}
	$sku_LISTINGS = '&sku[]=' . implode('&sku[]=',$chunk);

	$url = 'https://api.data_companneee.com/script.php?apiKey='.$api_KEY.$sku_LISTINGS;
	$url2 = 'https://api.data_companneee.com/script.php?apiKey='.$api_KEY.$sku_LISTINGS;
	$ch1 = curl_init($url);
	$ch2 = curl_init($url2);
	curl_setopt($ch1, CURLOPT_HEADER, 0);
	curl_setopt($ch1, CURLOPT_RETURNTRANSFER, true);
	curl_setopt($ch2, CURLOPT_HEADER, 0);
	curl_setopt($ch2, CURLOPT_RETURNTRANSFER, true);
	$mh = curl_multi_init();
	curl_multi_add_handle($mh, $ch1);
	curl_multi_add_handle($mh, $ch2);
	
	$active = null;
	do {
		$status = curl_multi_exec($mh, $active);
		curl_multi_select($mh);
	} while ($status === CURLM_CALL_MULTI_PERFORM || $active);
	
	$response_A = curl_multi_getcontent($ch1);
	$response_B = curl_multi_getcontent($ch2);
	curl_multi_remove_handle($mh, $ch1);
	curl_multi_remove_handle($mh, $ch2);
	curl_multi_close($mh);

	
	foreach($chunk as $each_sku) {
	
		// do stuff (process $response_A) etc.
		// do stuff (process $response_B) etc.
		
	}
}

May I please have your thoughts on this?
Thank you!!

 

 

[requinix]

4 hours ago, ChenXiu said:

To exceed their "3 sku limit," they provided me a 2nd API key.

Ha ha, what? That's their solution? To make you get another API key so you can query for 2x the SKUs? What happens when you need 9? 10? 20? Is the API so expensive for them to run that they can, really, only handle 3 at a time?
And then, why not simply run multiple requests? You already have that there - just use the same key. Is there also time-based throttling on what you can do?

This is so weird.

That aside, work it like this:

Using one API key, get yourself a loop that can do all the SKUs. So basically what you have there (if it didn't have the key stuff). That's the basic functionality you need here, and you can think of the "swap between API keys" as a small layer to be added on top of the functionality.

Then, set up an array of keys - because distinct variables makes this awkward to work with, and even more awkward to maintain if/when you discover that you need to add a third key.

To pick the key to use, think of it in the general sense of "I have multiple keys and I want to cycle through them". Because a mindset of a fixed number of keys (especially 2) will get you stuck into a narrow line of thinking (like needing to alternate between them).
"Cycling" works simply and doesn't need to be adjusted based on the number of keys: cycling is picking key 1, then key 2, then key 3... then when you're on the last key, you go back to the beginning.

Cycling requires a counter, of course, but tou can get one from the foreach/array_chunk and that will count up automatically without you having to increment it yourself.
Then take your counter, add modulus based on the number of keys, grab that key, and stick it into your API.

const MAX_SKUS_PER_REQUEST = 3;

$keys = ["one", "two", "three", ...];

foreach (array_chunk($sku_numbers, MAX_SKUS_PER_REQUEST) as $i => $chunk) {
	$key = $keys[$i % count($keys)];

	...
}

 

[Danishhafeez]

Your script is indeed attempting to divide the SKUs between the two API keys, but there are a few issues that need to be addressed:

Duplicate API URLs: You are using the same URL for both cURL handles ($url and $url2), which means you're sending the same requests with both API keys. You should generate separate URLs for each key.

Incorrect API Key Assignment: The logic to assign API keys based on the chunk index isn't correct. You're assigning the same API key to both chunks if the chunk index is odd. Instead, you should alternate between the two keys for each chunk.

Here's the modified script to address these issues: 

<?php
$sku_numbers = array(
    '12345',
    '33333',
    '98745',
    '44444',
    '11111',
    '05054'
);

$key1 = 'first_SECRET_key';
$key2 = 'second_SECRET_key';
$keycount = 0;

foreach (array_chunk($sku_numbers, 3) as $chunk) {
    $api_KEY = ($keycount % 2 == 0) ? $key1 : $key2; // Alternating between keys for each chunk
    $sku_LISTINGS = '&sku[]=' . implode('&sku[]=', $chunk);

    $url = 'https://api.data_companneee.com/script.php?apiKey=' . $api_KEY . $sku_LISTINGS;
    $ch = curl_init($url);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

    $mh = curl_multi_init();
    curl_multi_add_handle($mh, $ch);

    $active = null;
    do {
        $status = curl_multi_exec($mh, $active);
        curl_multi_select($mh);
    } while ($status === CURLM_CALL_MULTI_PERFORM || $active);

    $response = curl_multi_getcontent($ch);
    curl_multi_remove_handle($mh, $ch);
    curl_multi_close($mh);

    // Process the response for this chunk immediately
    // Assuming response processing code here

    $keycount++; // Increment key count for the next chunk
}
?>

This script will now properly alternate between the two API keys for each chunk and process responses immediately after receiving them. Additionally, it sends separate requests for each API key. Make sure to add your response processing code where indicated.i hope it will solve your 50 percent issue.

 

Best Regard

Danish Hafeez | QA Assistant

ICTInnovations