Jump to content
NotionCommotion

cURL both a file and POST data

Recommended Posts

The following script receives an array of files and optional POST data.  The POST data requires http_build_query(), however, this seems to prevent sending the files.  How can this be resolved?  Thank you

PS.  You think it is worthless to delete no longer needed files from the tmp directory?
 

    public function proxyFile(array $files, array $data=[])
    {
        $this->debugDump($data, 'proxyFile data');

        $url=$this->getUrl();

        foreach($files as $name=>$file) {
            $data[$name] = new \CURLFile($file['tmp_name'],$file['type'],$file['name']);
        }

        $data=http_build_query($data);//Doesn't work with files??? http_build_query($data),
        $this->debugDump($data, 'proxyFile data');

        $options=[
            CURLOPT_URL             =>  $url,
            CURLOPT_HTTPHEADER      =>  ['X-Secret-Key: '.$this->datalogger['key']],
            CURLOPT_POST            =>  1,
            CURLOPT_POSTFIELDS      =>  $data,
            CURLOPT_RETURNTRANSFER  =>  true,     // return web page
            CURLOPT_HEADER          =>  false,    // don't return headers
            CURLOPT_FOLLOWLOCATION  =>  true,     // follow redirects
            CURLOPT_ENCODING        =>  "",       // handle all encodings
            CURLOPT_USERAGENT       =>  "unknown",// who am i
            CURLOPT_AUTOREFERER     =>  true,     // set referrer on redirect
            CURLOPT_CONNECTTIMEOUT  =>  120,      // timeout on connect
            CURLOPT_TIMEOUT         =>  120,      // timeout on response
            CURLOPT_MAXREDIRS       =>  10,       // stop after 10 redirects
        ];
        $ch      = curl_init();
        curl_setopt_array( $ch, $options );
        $rsp=['rsp'=>curl_exec( $ch ),'errno'=>curl_errno($ch),'code'=>curl_getinfo($ch, CURLINFO_HTTP_CODE),'error'=>false];
        if($rsp['errno']) {
            $rsp['error']=curl_error($ch);
        }
        curl_close( $ch );
        //syslog(LOG_INFO, json_encode($rsp));
        if($rsp['errno']) {
            $rsp['code']=400;
            $obj=($rsp['errno']==6)
            ?['message'=>'Invalid Datalogger IP','code'=>1]
            :['message'=>"cURL Error: $rsp[error] ($rsp[errno])"];
        }
        elseif(!isset($rsp['rsp']) || !$rsp['rsp']) {
            $obj=null;
        }
        else {
            $obj=json_decode($rsp['rsp']);
            if(json_last_error() != JSON_ERROR_NONE) {
                syslog(LOG_ERR, 'proxyFile(). '.$rsp['rsp']);
                $rsp['code']=400;
                $obj=['message'=>'Invalid JSON response','code'=>1];
            }
        }
        foreach($_FILES as $file) { // Is this necessary?
            unlink($file['tmp_name']);
        }
        return [$obj, $rsp['code']];
    }

 

Share this post


Link to post
Share on other sites

You can't http_build_query an array of CURLFile objects. Think about it.

POSTFIELDS can be an array too.

Share this post


Link to post
Share on other sites
54 minutes ago, requinix said:

You can't http_build_query an array of CURLFile objects. Think about it.

POSTFIELDS can be an array too.

Yes, I realize I can not encode objects.  But when I don't, the post variables come over as a text "array" and not an array of values.  The array in question is a sequential array (i.e. id[], id[], etc).  I was thinking I might need to loop through the array and only encode certain parts, but was not successful.  

Share this post


Link to post
Share on other sites

Is the method inheriting $data from somewhere? So the problem is you have multidimensional data and you need to add in CURLFile objects. That's a bother. I vaguely remember cURL not supporting that.

Good news: check out this user comment.

Share this post


Link to post
Share on other sites

Building the request body manually is worse.

1. You cannot use CURLFile and http_build_query. You have to choose.
2. You know what http_build_query does, right? You know what passing an array for POSTFIELDS does, right?

If you've decided that the hammer for this screw will be http_build_query then you're in for a hard time.

Share this post


Link to post
Share on other sites

Thanks requinix,

Even before seeing your reply, I came to the same conclusion.

I thought http_build_query converts an array such that it could be sent in the URL.  Which never made sense to me that I would use to pass to POSTFIELDS.  I mean, is it in the body or the url?  Granted, I need to better understand the http protocol.  And if in the url, why am I having these problems?  I know that a post request could also have $_GET values which are presumably passed not in the body but the url.

Dang, wife is telling me I need to take the dog for a pop.
 

function convert($x) {
    $rs=[];
    if(!is_array($x) && !is_object($x)) {
        throw new Exception('not an array or object');
    }
    foreach($x as $key=>$value) {
        if(is_array($value) || is_object($value)) {
            $rs[$key]=convert($value);
        }
        else {
            $rs[$key]=$value;
        }
    }
    return $rs;
}

<?php
echo('$_POST:<br>');
var_dump($_POST);
echo('<br>$_GET:<br>');
var_dump($_GET);
echo('<br>');

$a=['x'=>123, 'y'=>[5,7,9,0], 'z'=>['a'=>123,'b'=>321,'c'=>111]];
$http_build_query=http_build_query($a);
echo('$http_build_query: '.$http_build_query.'<br>');
$converted=convert($a);
echo('$converted '.var_dump($converted).'<br>');
$x=123;

?>
<form method="post" action="<?php echo $_SERVER['PHP_SELF']; ?>?a=1&b=2&c=3">
   <input type="text" name="name"><br>
   <input type="submit" name="submit" value="Submit Form"><br>
</form>

 

Share this post


Link to post
Share on other sites
13 minutes ago, NotionCommotion said:

I thought http_build_query converts an array such that it could be sent in the URL.

PHP parses query strings automatically (for $_GET) by using parse_str(). http_build_query() is the opposite of that.

It's great for building query strings that you want to embed in links or forms or wherever that you need to get the query string as a, well, string. But the situation there is that you need it as a string. Like if you wanted to pass $_GET to another script you would

<a href="/path/to/script.php?<?=htmlspecialchars(http_build_query($_GET))?>">Link</a>

Not only is creating that query string manually yourself difficult with something dynamic like $_GET (you don't know the parameters), you'd also have to deal with URL-encoding stuff. http_build_query does all that. To make this even more complicated, consider

<a href="/path/to/script.php?<?=htmlspecialchars(http_build_query(["get" => $_GET]))?>">Link</a>

then in the other script

parse_str($_GET["get"], $get);

Point is, it takes a $_GET-type array and gives you a string, and you use it because you want the string. If you don't want or need a string, like you wouldn't with CURLOPT_POSTFIELDS because cURL will do that for you, then you wouldn't use it.

13 minutes ago, NotionCommotion said:

Which never made sense to me that I would use to pass to POSTFIELDS.  I mean, is it in the body or the url?

Yes.

There are two types of POST bodies, so to speak. The normal one is "application/x-www-form-urlencoded" and sends data that looks and works just like a query string, except it goes in the request body instead of in the URL. The other one, needed by file uploads, is "multipart/form-data" and is much less compact and incidentally much more human readable.

If you're using cURL for an upload then you'll have to get that second format. http_build_query won't help you there.

13 minutes ago, NotionCommotion said:

And if in the url, why am I having these problems?  I know that a post request could also have $_GET values which are presumably passed not in the body but the url.

The problems are indeed because it's not going in the URL. And yes, $_GET values are in the URL - $_GET is always from the URL, $_POST is always the POSTed data (if it's one of the two body types I mentioned above). The REQUEST_METHOD doesn't switch where data goes but rather instructs PHP that it should look for the presence of POSTed data. Kinda.

Share this post


Link to post
Share on other sites

Looking at that comment on PHP.net linked earlier, it seems like you just need to make sure your $data array is a single dimension.   You can use http_build_query to do some of the work of converting your potentionaly multi-dimensional array into a single dimension, then add your files to the end.

For example:

<?php

$data = [
    'name' => 'Kicken'
    , 'date' => [
        'month' => 10
        , 'day' => 15
        , 'year' => 2018
    ]
];

$files = [
    'photo' => [
        'tmp_name' => '/tmp/blah.jpg'
        , 'type' => 'image/jpeg'
        , 'name' => 'blah.jpg'
    ]
    , 'cv' => [
        'tmp_name' => '/tmp/cv.txt'
        , 'type' => 'text/plain'
        , 'name' => 'cv.txt'
    ]
];


$postData = [];
foreach (explode('&', http_build_query($data)) as $pair){
    list($name, $value) = explode('=', $pair, 2);
    $postData[$name] = $value;
}

foreach ($files as $name=>$file){
    $postData[$name] = new \CURLFile($file['tmp_name'],$file['type'],$file['name']);
}

var_dump($postData);

Output:

array(6) {
  'name' =>
  string(6) "Kicken"
  'date%5Bmonth%5D' =>
  string(2) "10"
  'date%5Bday%5D' =>
  string(2) "15"
  'date%5Byear%5D' =>
  string(4) "2018"
  'photo' =>
  class CURLFile#1 (3) {
    public $name =>
    string(13) "/tmp/blah.jpg"
    public $mime =>
    string(10) "image/jpeg"
    public $postname =>
    string(8) "blah.jpg"
  }
  'cv' =>
  class CURLFile#2 (3) {
    public $name =>
    string(11) "/tmp/cv.txt"
    public $mime =>
    string(10) "text/plain"
    public $postname =>
    string(6) "cv.txt"
  }
}

You could also make your own version of http_build_query that's aware of CURLFile if that'd make things easier.  Then you could just build up your $data array with the file objects where they need to be and then collapse it with your custom function.

edit: Might need to urldecode() the name/value pairs after the http_build_query, not sure on that.

Edited by kicken

Share this post


Link to post
Share on other sites
2 minutes ago, kicken said:

edit: Might need to urldecode() the name/value pairs after the http_build_query, not sure on that.

Probably.

Share this post


Link to post
Share on other sites

Thanks kicken,

Why do you think I will need urldecode()?  See below outputs.

Also, back to my original line of thought, any reason (other than being non-typical and thus confusing maybe) that the data array couldn't be transferred in the url and arrive as GET parameters, and CURLOPT_POSTFIELDS used solely for files?

With uldecode():

Array
(
    [name] => Kicken
    [date[month]] => 10
    [date[day]] => 15
    [date[year]] => 2018
    [date[extraSeq][0]] => a 1
    [date[extraSeq][1]] => b 1
    [date[extraSeq][2]] => c 1
    [date[extraAssoc][a 1]] => A 1
    [date[extraAssoc][b 1]] => B 1
    [date[extraAssoc][c 1]] => C 1
)
 

Without uldecode():

Array
(
    [name] => Kicken
    [date%5Bmonth%5D] => 10
    [date%5Bday%5D] => 15
    [date%5Byear%5D] => 2018
    [date%5BextraSeq%5D%5B0%5D] => a+1
    [date%5BextraSeq%5D%5B1%5D] => b+1
    [date%5BextraSeq%5D%5B2%5D] => c+1
    [date%5BextraAssoc%5D%5Ba+1%5D] => A+1
    [date%5BextraAssoc%5D%5Bb+1%5D] => B+1
    [date%5BextraAssoc%5D%5Bc+1%5D] => C+1
)

Share this post


Link to post
Share on other sites
2 hours ago, NotionCommotion said:

Why do you think I will need urldecode()?  See below outputs.

Because it won't work if I don't.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.