Jump to content

Search the Community

Showing results for tags 'curl'.

  • Search By Tags

    Type tags separated by commas.
  • Search By Author

Content Type


Forums

  • Welcome to PHP Freaks
    • Announcements
    • Introductions
  • PHP Coding
    • PHP Coding Help
    • Regex Help
    • Third Party Scripts
    • FAQ/Code Snippet Repository
  • SQL / Database
    • MySQL Help
    • PostgreSQL
    • Microsoft SQL - MSSQL
    • Other RDBMS and SQL dialects
  • Client Side
    • HTML Help
    • CSS Help
    • Javascript Help
    • Other
  • Applications and Frameworks
    • Applications
    • Frameworks
    • Other Libraries
  • Web Server Administration
    • PHP Installation and Configuration
    • Linux
    • Apache HTTP Server
    • Microsoft IIS
    • Other Web Server Software
  • Other
    • Application Design
    • Other Programming Languages
    • Editor Help (PhpStorm, VS Code, etc)
    • Website Critique
    • Beta Test Your Stuff!
  • Freelance, Contracts, Employment, etc.
    • Services Offered
    • Job Offerings
  • General Discussion
    • PHPFreaks.com Website Feedback
    • Miscellaneous

Find results in...

Find results that contain...


Date Created

  • Start

    End


Last Updated

  • Start

    End


Filter by number of...

Joined

  • Start

    End


Group


AIM


MSN


Website URL


ICQ


Yahoo


Jabber


Skype


Location


Interests


Age


Donation Link

  1. <?php $url = "http://www.something.com"; $curl = curl_init(); curl_setopt($curl, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']); curl_setopt ($curl, CURLOPT_URL, $url); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $curlresult = curl_exec ($curl); curl_close($curl); preg_match_all("/ something=something/", $curlresult, $theurls); $string = serialize($theurls); $curl = curl_init(); curl_setopt($curl, CURLOPT_USERAGENT, $_SERVER2['HTTP_USER_AGENT']); curl_setopt ($curl, CURLOPT_URL, $string); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $curlresult2 = curl_exec ($curl); curl_close ($curl); print $curlresult2; ?> so im realy new to php and its probably a silly mistake .-. this is the answer i got from wamp syntax error, unexpected '$curl' (T_VARIABLE) anyway , can some one help me?
  2. 0 down vote favorite Using PHP/cURL I log into a web site and attempt to upload a file. It returns the error message, "Error: File upload not processed. Please ensure that an applicable excel file in the specified template format is selected for upload." This error is different from the message I get if I upload from a browser and give it a file of the wrong format (like a .jpg). If I upload the same excel file through Firefox it works and is accepted by the server. So how can I make this work through PHP/cURL the way it does with Firefox? I tried the same upload to my own PHP script and it also works fine. I also reviewed some relevant questions here and tried various things, none of which made any difference. I tried with and without the MIME type also with and without options CURLOPT_SSL_VERIFYPEER and CURLOPT_HTTPHEADER. I posted this same question to StackOverflow, but no one there was able to help. The owner of the web site was not particularly helpful, only telling me that others are doing it. Below is my code: $url = 'https://www.example.com/uploadController_u.jsp'; curl_setopt( $ch, CURLOPT_COOKIEJAR, $cookie_file ); curl_setopt( $ch, CURLOPT_COOKIEFILE, $cookie_file ); curl_setopt( $ch, CURLOPT_RETURNTRANSFER, 1 ); curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, 1 ); curl_setopt( $ch, CURLOPT_MAXREDIRS, 10 ); curl_setopt( $ch, CURLOPT_FAILONERROR, true ); curl_setopt( $ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0"); curl_setopt( $ch, CURLOPT_REFERER, $curl_referer ); curl_setopt( $ch, CURLOPT_URL, $url ); curl_setopt( $ch, CURLOPT_VERBOSE, true ); curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, 0 ); // Added based on another StackOverflow question curl_setopt( $ch, CURLOPT_HTTPHEADER, array('Expect:') ); // Added based on another StackOverflow question // Tried with and without the MIME type $post_vars = array( 'file' => "@/home/someuser/private/eapis-test/mx/Template-test.xls;type=application/vnd.ms-excel", 'submit' => 'Upload file' ); curl_setopt( $ch, CURLOPT_POST, true ); curl_setopt( $ch, CURLOPT_POSTFIELDS, $post_vars ); $response = curl_exec( $ch ); Below is the form I am posting to. I did not find any references to form1 in the javascript. <table border="0" cellspacing="0" cellpadding="0"> <form action="uploadController_u.jsp" method="post" enctype="multipart/form-data" name="form1" id="form1"> <tr> <td align="left"> <p> <h2>Upload Flight:</h2> </p> </td> </tr> <tr> <td> <b style="font-size: 90%;">To add a new flight from spreadsheet, </b> </td> <td align="left"> <input name="file" type="file" id="file" align="left" size="36"> <td> </tr> <tr> <td> </td> <td align="left"> <p> <input type="submit" name="Submit" value="Upload file"/> </p> </td> </tr> </form> </table> I tried these headers to better simulate a browser, but to no avail: $header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,"; $header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5"; $header[] = "Cache-Control: max-age=0"; $header[] = "Connection: keep-alive"; $header[] = "Keep-Alive: 300"; $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7"; $header[] = "Accept-Language: en-us,en;q=0.5"; $header[] = "Pragma: "; curl_setopt( $ch, CURLOPT_HTTPHEADER, $header );
  3. Hey Guys & Girls, Okay, so here's my little issue. I'm running a WordPress site with a custom PHP script using JSON/CURL/API pulling info from an API. All is good in the world in this respect. However, the API listings that are returned are only returned in a 50 item block. The API doesn't allow for more than 50 items per call. So, what I'm looking for is a way to paginate my results so that if there are more than 50 items in the API call, it will allow the user to click to page 2, 3, 4 etc. Here's my code: <?php //functions relating to wordpress go here: //---------------------------------------- $bg_colors = array('green', 'orange', 'blue', 'yellow', 'red', 'black'); //---------------------------------------- //End functions relating to wordpress // Start PetRescue PHP/API Code //---------------------------------------- // Open CuRL/JSON Stuff $ch = curl_init(); $category=$_GET['category']; $url="http://www.myapilink.com.au/api/listings?token=f716909f5d644fe3702be5c7895aa34e&group_id=10046&species=".$category; curl_setopt($ch, CURLOPT_URL,$url); curl_setopt($ch, CURLOPT_HEADER, 0); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_HTTPHEADER, array( 'Accept: application/json', 'X-some-API-Key: f716909f5d644fe3702be5c7895aa34e', )); $json = json_decode(curl_exec($ch), true); // Functions relating to the Echo Code foreach($json['listings'] as $listing) { $short_personality=substr($listing['personality'],0,500); $peturl="http://mysiteurl.com.au/beta/pet-info/?petid=".$listing['id']; $medium_photo=$listing['photos'][0]['large_340']; $gallery_listing=$listing['photos'][5]['large_900']; $gender_class=strtolower($listing['gender']); $breed_class=strtolower($listing['species']); $name=($listing['name']); $unique_gallery_name="lightbox['.$inc.']"; $inc++; foreach($listing["photos"] as $photosthumb) { $photo_thumb_large=$photosthumb["large_340"]; $photo_thumb_hidden=$photosthumb["small_80"]; } $rand_background = $bg_colors[array_rand($bg_colors)]; // General IF/AND/ELSE Statements to refine the Echo Output if($listing['photos'] == null) { $medium_photo="http://mysiteurl.com.au/beta/wp-content/themes/Archive/images/photo_coming_soon.png"; } if($listing['desexed'] == "Yes") { $desexed="yes"; } else { $desexed="no"; } if($listing['vaccinated'] == "Yes") { $vaccinated="yes"; } else { $vaccinated="no"; } if($listing['wormed'] == "Yes") { $wormed="yes"; } elseif($listing['wormed'] == "No") { $wormed="no"; } else { $wormed="no"; } if($listing['heart_worm_treated'] == "Yes") { $heart_worm_tested="yes"; } elseif($listing['heart_worm_treated'] == "No") { $heart_worm_tested="no"; } else { $heart_worm_tested="no"; } if($listing['species'] == "Dog") { $adoption_enquiry_link="http://mysiteurl.com.au/beta/pre-adoption-form-dogs/?dog_name=$name"; $hwt="list-$heart_worm_tested"; } elseif($listing['species'] == "Cat") { $adoption_enquiry_link="http://mysiteurl.com.au/beta/pre-adoption-form-cats/?cat_name=$name"; $hwt="list-hwt-hidden"; } // Echo the output echo'<div class="animal"> <div class="animal-image"> <a class="size-thumbnail thickbox" rel="'.$unique_gallery_name.'" href="'.$medium_photo.'"> <img src="'.$medium_photo.'" class="image-with-border" alt=""> <div class="border" style="width: 340px; height: 340px;"> <div class="open"></div> </div> </a> <div class="item-title-bg '.$rand_background.'"> <h2 class="entry-title">'.$listing['name'].'</h2> <div class="animal-adopt-button"> <a href="'.$adoption_enquiry_link.'" style="background-color: #575757; border-color: #494949; background-position:5px 0;" class="button medium">Enquire about '.$name.'</a> </div> </div> </div> <div class="animal-thumbnail hidden"> <a class="lightbox" rel="'.$unique_gallery_name.'" href="'.$photo_thumb_large.'"> <img class="animal-thumbnail" src="'.$photo_thumb_hidden.'" > </a> </div> <div class="animal-content"> <div class="animal-left"> <ul class="animal-list"> <li class="list-sex-'.$gender_class.'">'.$listing['gender'].'</li> <li class="list-breed-'.$breed_class.'">'.$listing['breeds_display'].'</li> <li class="list-age">'.$listing['age'].'</li> <li class="list-fee">'.$listing['adoption_fee'].'</li> </ul> </div> <div class="animal-right"> <ul class="animal-list"> <li class="list-'.$desexed.'">Desexed?</li> <li class="list-'.$vaccinated.'">Vaccinated?</li> <li class="list-'.$wormed.'">Wormed?</li> <li class="'.$hwt.'">Heart Worm Tested?</li> </ul> </div> <div class="animal-full"> <ul class="animal-list"> <li class="list-description">'.$listing['personality'].'</li> </ul> </div></div> <div class="clearer"></div> </div> <div class="delimiter"></div>'; // Close the CURL } curl_close($ch); ?> Attached is some dummy data that represents the JSON data returned from the API. As you can see in the file, the last line is where the paging details comes in. Any help you guys could give would be appreciated. Cheers, Dave listings.txt
  4. Hey guys I am new here, I am trying to Get from my Api,Im using curl in php to log into on one page and then get another page passing all cookies from the first page along with you but I keep getting Access denied!! $ch = curl_init(); curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/cookieFileName"); curl_setopt($ch, CURLOPT_URL,"http://www.supersaas.com/api/users"); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, "?account=myname&password=mypassword"); ob_start(); // prevent any output curl_exec ($ch); // execute the curl command ob_end_clean(); // stop preventing output curl_close ($ch); unset($ch); $ch = curl_init(); curl_setopt($ch, CURLOPT_RETURNTRANSFER,1); curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/cookieFileName"); curl_setopt($ch, CURLOPT_URL,"http://www.supersaas.com/api/users?id=12"); $buf2 = curl_exec ($ch); curl_close ($ch); echo "<PRE>".htmlentities($buf2); What's wrong with the code????
  5. I am trying to implement a validation process within the php wrapper for the new form. The automatically gets submitted and completely ignores the validation process. Is there any reason why? <?php //Initialize the $query_string variable for later use $query_string = ""; //If there are POST variables if ($_POST) { //Initialize the $kv array for later use $kv = array(); //For each POST variable as $name_of_input_field => $value_of_input_field foreach ($_POST as $key => $value) { //Set array element for each POST variable (ie. first_name=Arsham) $kv[] = stripslashes($key)."=".stripslashes($value); } //Create a query string with join function separted by & $query_string = join("&", $kv); } //Check to see if cURL is installed ... if (!function_exists('curl_init')){ die('Sorry cURL is not installed!'); } //The original form action URL from Step 2 $url = 'https://www.salesforce.com/servlet/servlet.WebToLead?encoding=UTF-8'; //Open cURL connection $ch = curl_init(); if(isset($_POST['submit'])){ if($_POST['first_name'] == ""){ $error="Please enter a topic<br>"; } if($_POST['last_name'] == ""){ $error="Please enter a topic<br>"; } if($_POST['phone'] == ""){ $error="Please enter a topic<br>"; } if($_POST['email'] == ""){ $error="Please enter a topic<br>"; } if($_POST['company'] == ""){ $error="Please enter a topic<br>"; } if(isset($error)){ echo $error; }else{ // Both fields have content in } } //Set the url, number of POST vars, POST data curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_POST, count($kv)); curl_setopt($ch, CURLOPT_POSTFIELDS, $query_string); //Set some settings that make it all work curl_setopt($ch, CURLOPT_HEADER, FALSE); curl_setopt($ch, CURLOPT_RETURNTRANSFER, FALSE); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); //Execute SalesForce web to lead PHP cURL $result = curl_exec($ch); //close cURL connection curl_close($ch); ?>
  6. Hello, I wanted to download flight from Ryanair page. I've checked all headers which are sended when browser "create" request. It looks like this: I'm entering at https://www.bookryanair.com/SkySales/Booking.aspx In headers I can see, that theres two GET requests to the following pages: 1) Booking.aspx (here is created session ID) 2) Search.aspx (here is generated and assigned _VIEWSTATE as hidden field in form) Until now, everything what I'm trying to do by curl is the same as in browser (responses are the same). Now I'm complete form and data are sended by POST to: 3) Search.aspx It uses the above mentioned SessionID and _VIEWSTATE, the rest of the fields I fill as in request from the browser. In browser I can see flights, while when I'm trying to generate it by curl, I have the same result as in first GET reuqest to Search.aspx (2) (no data about flight, like empty requets). Code from my last request to Search.aspx (3): $url = "https://www.bookryanair.com/SkySales/Search.aspx"; $fields = array( '_EVENTTARGET' => 'SearchInput$ButtonSubmit', '_EVENTARGUMENT' => '', '_VIEWSTATE' => $viewstate, 'formaction' => 'Search.aspx', 'errorlist' => '', 'SearchInput$IsFlexible' => 'on', 'SearchInput$TripType' => 'RoundTrip', 'SearchInput$Orig' => 'WMI', 'SearchInput$Dest' => 'NYO', 'SearchInput$DeptDate' => '06/05/2014', 'SearchInput$RetDate' => '10/05/2014', 'SearchInput$PaxTypeADT' => '1', 'SearchInput$PaxTypeCHD' => '0', 'SearchInput$PaxTypeINFANT' => '0'); foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; } rtrim($fields_string,'&'); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL,$url); curl_setopt($ch, CURLOPT_HTTPHEADER, array( 'Accept: text/html, */*; q=0.01', 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8', 'Connection: keep-alive', 'Cache-Control: no-cache', 'Pragma: no-cache', 'X-Requested-With: XMLHttpRequest' )); curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0'); curl_setopt($ch, CURLOPT_HEADER,1); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0); curl_setopt($ch, CURLOPT_COOKIEFILE, COOKIE_FILE); curl_setopt($ch, CURLOPT_REFERER, 'https://www.bookryanair.com/SkySales/Booking.aspx'); curl_setopt($ch, CURLOPT_COOKIEJAR, COOKIE_FILE); curl_setopt($ch, CURLOPT_POST,1); curl_setopt($ch, CURLOPT_POSTFIELDS,$fields_string); $result = curl_exec($ch); curl_close($ch) $viewstate is from request to Search.aspx (2) and retrieved from the hidden field generated _VIEWSTATE. Cookie_file - place with stored cookie.txt and sessionId. What am I doing wrong? Is the problem in the communication that uses https protocol? Thanks in advance.
  7. I am having trouble actually translating this curl API that is not given in PHP form into one that can be used with PHP. Below is what the documentation says. curl -s "https://api.example.com/v1/users/?per_page=3" \ -X GET \ -u app-id:api-key I have tried several examples such as: $ch = curl_init('https://api.example.com/v1/users/?per_page=3'); curl_setopt($ch,CURLOPT_HTTPHEADER,array('app-id:5435435435', 'api-key:fdskajf234jfdsakfhdjkfaas')); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($ch); curl_close($ch); And: $ch = curl_init('https://api.example.com/v1/users/?per_page=3'); curl_setopt($ch,CURLOPT_HTTPHEADER,array('5435435435:fdskajf234jfdsakfhdjkfaas')); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec($ch); curl_close($ch); Now what I believe is throwing me off is the "-u". When I navigate to the api URL in my browser, an authentication box comes up asking for my username and password. If I put in the app-id as the username and the api-key as the password it works. But again that is in the browser. How do I get past that with PHP? Any help is greatly appreciated.
  8. HI I've this simple code for downloading files from a direct link and move it to my webhost . <?php ini_set('display_errors', '1'); error_reporting(E_ALL); $v = "http://highestvideo.com/up_image/01-24-58-09!1!.jpeg"; $ch = curl_init($v); $fp = fopen('aaa/flower.jpg', 'w'); curl_setopt($ch, CURLOPT_FILE, $fp); curl_setopt($ch, CURLOPT_HEADER, 0); curl_exec($ch); curl_close($ch); fclose($fp); ?> I've set the permissions to 777 , It does create a file but it's 0 byte and empty . my webhost is not cpanel , it's kloxo server 6.1.2 stable . What is the problem ? How can I fix it ? thanks
  9. I am having a heck of a time trying to process a large cURL request. I keep running into issues with the mysql server timing out and also using the callback function within the cURL script (see below). What I am attempting to do is to utilize cURL to log a user into a system (*due to legality issues I cannot specify which) and pull all of their work for the day. I have been successful at pulling all of the work, but each order contains multiple sub-items, each with a specific url. For instance, 300 work orders would translate to approximately 2000 sub-items. Pulling the 300 work order takes approximately 1.6 minutes. For some reason just pulling 10 sub-items is taking in upwards of 3 minutes. After hundreds (and I am not exaggerating) of attempts I have finally decided to reach out to see if someone can take a look at my script and offer some knowledge. Here is the process from a logic standpoint: Pull all user login data from database and log them into the system through cURL (*Works fine) Request all activity and customer information and Insert into database (*Works fine) Get all sub-items and insert them into the database (*ISSUES) Here is the process from a script standpoint: User clicks "Import" button which sends AJAX request to run importWork PHP function. This function only handles requesting the activity and customer information through cURL. (Due to the amount of time it takes for the sub-items to process I have broken up the process). importWork function returns via jSON the number of work orders processed. ***In testing I have also had the importWork function store the urls for all of the sub-items to my database. The only issue is that the logins will start to timeout (Not on my server but the server I am pulling the data from) before all the sub-items can process. javascript automatically sends another AJAX request to pull all of the sub-items. I am using a cURL Multi function to process the url requests. The function will return an array containing the html for each of the urls. I then parse the html to search for the underlying hrefs I need to access the workorders, customer information, and sub-items. So overall, my question is, what is the best way to handle a large cURL request of 2000 urls? Below you will see the rolling_curl function which I am attempting to use to handle the line items. For some reason it doesnt work at all. What I would like to do is simply send an array of urls to the rolling_curl function and have it request all the html for each url. Once a url is finished processing it should run the callback script to insert the data into the database. I figured it would be the best way to handle such a large request in a timely manner. ROLLING CURL FUNCTION: explanation: A function will put all sub-item urls and the corresponding activity ids into an associative array and pass it to the rolling_curl function. The callback function will parse the html and insert the needed data into the database. The only thing this function is doing at this time is dumping "Failed". I have ran the script using the same urls through the standard cURL multi function (See Below) and verified it is pulling the html (So it isn't an issue with the urls). public function rolling_curl($urldata, $callback = null, $custom_options = null) { set_time_limit(0); //extract data from $urldata $urls = $urldata['urls']; $activities = $urldata['activities']; // make sure the rolling window isn't greater than the # of urls $rolling_window = 95; $rolling_window = (sizeof($urls) < $rolling_window) ? sizeof($urls) : $rolling_window; $master = curl_multi_init(); $curl_arr = array(); // add additional curl options here $std_options = array(CURLOPT_RETURNTRANSFER => true, CURLOPT_FOLLOWLOCATION => true, CURLOPT_MAXREDIRS => 5); $options = ($custom_options) ? ($std_options + $custom_options) : $std_options; // start the first batch of requests for ($i = 0; $i < $rolling_window; $i++) { $ch = curl_init(); $options[CURLOPT_URL] = $urls[$i]; curl_setopt_array($ch,$options); curl_multi_add_handle($master, $ch); } do { while(($execrun = curl_multi_exec($master, $running)) == CURLM_CALL_MULTI_PERFORM); if($execrun != CURLM_OK) break; // a request was just completed -- find out which one while($done = curl_multi_info_read($master)) { $info = curl_getinfo($done['handle']); if ($info['http_code'] == 200) { $output = curl_multi_getcontent($done['handle']); // request successful. process output using the callback function. $ref = array_search($info['url'],$urls); $callback($output, $activities[$ref],1); // start a new request (it's important to do this before removing the old one) $ch = curl_init(); $options[CURLOPT_URL] = $urls[$i++]; // increment i curl_setopt_array($ch,$options); curl_multi_add_handle($master, $ch); // remove the curl handle that just completed curl_multi_remove_handle($master, $done['handle']); } else { // request failed. add error handling. $dmp = 'Failed!'; var_dump($dmp); } } } while ($running); curl_multi_close($master); return true; } } STANDARD cURL MULTI FUNCTION: public function requestData($urls) { set_time_limit(0); // Create get requests for each URL $mh = curl_multi_init(); foreach($urls as $i => $url) { $ch[$i] = curl_init($url); curl_setopt($ch[$i], CURLOPT_RETURNTRANSFER, 1); curl_multi_add_handle($mh, $ch[$i]); } // Start performing the request do { $execReturnValue = curl_multi_exec($mh, $runningHandles); } while ($execReturnValue == CURLM_CALL_MULTI_PERFORM); // Loop and continue processing the request while ($runningHandles && $execReturnValue == CURLM_OK) { // Wait forever for network $numberReady = curl_multi_select($mh); if ($numberReady != -1) { // Pull in any new data, or at least handle timeouts do { $execReturnValue = curl_multi_exec($mh, $runningHandles); } while ($execReturnValue == CURLM_CALL_MULTI_PERFORM); } } // Check for any errors if ($execReturnValue != CURLM_OK) { trigger_error("Curl multi read error $execReturnValue\n", E_USER_WARNING); } // Extract the content foreach($urls as $i => $url) { // Check for errors $curlError = curl_error($ch[$i]); if($curlError == "") { $res[$i] = curl_multi_getcontent($ch[$i]); } else { return "Curl error on handle $i: $curlError\n"; } // Remove and close the handle curl_multi_remove_handle($mh, $ch[$i]); curl_close($ch[$i]); } // Clean up the curl_multi handle curl_multi_close($mh); // Print the response data return $res; } An assistance would be greatly appreciated!!! I am racking my head against my desk at this point =0). I am open to any suggestions. I will completely scrap the code and take an alternate approach if you would be so kind as to direct me accordingly. FYI - I am running on a hosted, shared server which I have little control over. PHP plugins might not be a route I can take at this point. But if there is something you know of that will assist me, shoot it at me and I will talk with my hosting provider. THANK YOU!!!!
  10. Hi there, I'm looking for a way how to obtain data for World of tanks account. I'm using Wargaming API and cURL. Address example with data looks like this: http://api.worldoftanks.eu/2.0/account/info/?application_id=d0a293dc77667c9328783d489c8cef73&account_id=509662813 With little php manual help I came up with (ehm copied) this. <?php $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, "http://api.worldoftanks.eu/2.0/account/info/?application_id=d0a293dc77667c9328783d489c8cef73&account_id=509662813"); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $output = curl_exec($ch); curl_close($ch); $json = json_decode($output); echo $json->{'data'}->{509662813}->{'statistics'}->{'all'}->{'battles'}; ?> It doesn't work but when I copy the contents of address to txt file and save it on my webhosting and then use the cURL - it works. I tried to use CURLOPT_CONNECTTIMEOUT because sometimes it takes a long time to load a page from API. It didn't work either. What am I missing here?
  11. I have the following code which works just fine sending the $xml. However when i try to add a file to be sent with the xml as well i get an error. I tried using a postData array but then got an array to string error. The way i have it now works but it only sends the xml not the file. Any ideas on how I can send the @file with the postfields? Any help is appreciated $header = array('Content-Type: multipart/form-data'); $xml['xml'] = '<UploadPhoto>'; $xml['xml'] .= '<ID>12345</ID>'; $xml['xml'] .= '<PhotoID>myphoto</PhotoID>'; $xml['xml'] .= '<Filename>myphoto.jpg</Filename>'; $xml['xml'] .= '<Instructions>Need cheeks less rosy</Instructions>'; $xml['xml'] .= '</UploadPhoto>'; $postData = array( 'file' => '@/myphoto.jpg', 'xml' => $xml ); } $connection = curl_init(); curl_setopt($connection, CURLOPT_URL, "http://www.myurl.com/api"); curl_setopt($connection, CURLOPT_HTTPHEADER, $header); curl_setopt($connection, CURLOPT_SSL_VERIFYPEER, 0); curl_setopt($connection, CURLOPT_SSL_VERIFYHOST, 0); curl_setopt($connection, CURLOPT_POST, 1); curl_setopt($connection, CURLOPT_POSTFIELDS, $xml); curl_setopt($connection, CURLOPT_RETURNTRANSFER, 1); set_time_limit(108000); $output = curl_exec($connection); curl_close($connection); print_r($output);
  12. Hey everyone, I'm trying to send a JSON array to an API via HTTP POST, get a response and print it. I tried using cURL to do so, but it doesn't seem to work. I simply get zero response, a blank page. My request: <?php $data = array( "login" => "myLogin", "password" => "myPassword", "id" => "12345", "tag" => "test" ); $json_data = json_encode($data); $ch = curl_init('URL/api/mylogin'); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST"); curl_setopt($ch, CURLOPT_HEADER, false); curl_setopt($ch, CURLOPT_POSTFIELDS, $json_data); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($ch, CURLOPT_HTTPHEADER, array( 'Content-Type: application/json', 'Content-Length: ' . strlen($json_data)) ); $output = curl_exec($ch); $result = json_decode($output); echo $result; ?> The response I should be getting: {"status": 200, "message": "OK", "login_key": "abcdefh532h235yh"} any idea why I'm not getting any response? (this works ok when I manually test it using a test REST client) Thanks, Rani
  13. Hi, I write this code to test the download time and size of a website, but is showing incorrct data. example: Took 0.586468 seconds to send a request to http://something.com download size 0 <?php include("../includes/layouts/header.php");?> <div id="getload"> <?php error_reporting(E_ALL | E_STRICT); // Initialize cURL with given url $url = $_POST['website_name']; $ch = curl_init($url); // Create a curl handle //$ch = curl_init('http://www.yahoo.com/'); // Execute curl_exec($ch); // Check if any error occurred if(!curl_errno($ch)) { $info = curl_getinfo($ch); echo 'Took ' . $info['total_time'] . ' seconds to send a request to ' . $info['url'] . '<br>download size ' . $info['size_download']; } // Close handle curl_close($ch); ?> </div> <?php include("../includes/layouts/footer.php");?> what is the currect way to display the download time and download size? Thanks.
  14. Below is PHP using CURL with JSON to POST data to the MemberClicks API. I keep getting a 413 error when trying to process the page. I've tried a few things, but none seem to work. So far I don't get and syntax errors just on the 413. <?php if (!empty($_POST['userid']) && !empty($_POST['username']) && $_POST['data']=="Update") { $dateATOM = date(DATE_ATOM); $groupid = "12345"; $org = "ABCco"; $userid = $_POST['userid']; $username = $_POST['username']; $contactname = $_POST['contactname']; $email = $_POST['email']; $firstname = $_POST['firstname']; $middlename = $_POST['middlename']; $lastname = $_POST['lastname']; $businessaddress1 = $_POST['businessaddress1']; $businessaddress2 = $_POST['businessaddress2']; $businesscity = $_POST['businesscity']; $businessstate = $_POST['businessstate']; $businessinternationalstate = $_POST['businessinternationalstate']; $businesscountry = $_POST['businesscountry']; $businesszip = $_POST['businesszip']; $companyname = $_POST['companyname']; $title = $_POST['title']; $businessphone = $_POST['businessphone']; $mobilephone = $_POST['mobilephone']; $extension = $_POST['extension']; $fax = $_POST['fax']; $url = $_POST['url']; // Establish $url = 'https://####.memberclicks.net/services/auth'; $data = 'apiKey=123&username=user&password=pass'; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_POST, true); curl_setopt($ch, CURLOPT_POSTFIELDS, $data); $httpHeaders[] = "Accept: application/json"; curl_setopt($ch, CURLOPT_HEADER, true); curl_setopt($ch, CURLOPT_HTTPHEADER, $httpHeaders); curl_close($ch); $token = $jsonResult->token; // end --> $raw_json ='{"userId": "'.$userid.'","groupId":"'.$groupid.'","orgId":"'.$org.'","contactName": "'.$contactname.'","userName": "'.$username.'","attribute":[{"userId": "'.$userid.'","attId":"528954","attTypeId":"10", "attData": "'.$username.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"528955","attTypeId":"16", "attData": "'.$contactname.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"528950","attTypeId":"1", "attData": "'.$email.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529209","attTypeId":"2", "attData": "'.$firstname.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529210","attTypeId":"7", "attData": "'.$middlename.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529211","attTypeId":"3", "attData": "'.$lastname.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529213","attTypeId":"28", "attData": "'.$businessaddress1.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529214","attTypeId":"29", "attData": "'.$businessaddress2.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529215","attTypeId":"30", "attData": "'.$businesscity.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529216","attTypeId":"31", "attData": "'.$businessstate.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529240","attTypeId":"7", "attDate": "'.$businessinternationalstate.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529217","attTypeId":"33", "attData": "'.$businesscountry.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529218","attTypeId":"32", "attData": "'.$businesszip.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529219","attTypeId":"7", "attData": "'.$companyname.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529220","attTypeId":"7", "attData": "'.$title.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529221","attTypeId":"5", "attData": "'.$businessphone.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529244", "attData": "'.$mobilephone.'","attTypeId":"5","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529222","attName":"Extension", "attData": "'.$extension.'","lastModify": "'.$dateATOM.'"},{"userId": "'.$userid.'","attId":"529223","attTypeId":"14", "attData": "'.$fax.'","lastModify": "'.$dateATOM.'"}, {"userId": "'.$userid.'","attId":"529228","attTypeId":"15", "attData": "'.$url.'","lastModify": "'.$dateATOM.'"}]}'; $json = json_decode($raw_json); $url2 = 'https://####.memberclicks.net/services/user/'.$userid.'/?includeAtts=true'; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url2); curl_setopt($ch, CURLOPT_VERBOSE, true); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false); curl_setopt($ch, CURLOPT_POSTFIELDS, $json); curl_setopt($ch, CURLOPT_POST, true); $httpHeaders[] = "Accept: application/json"; $httpHeaders[] = "Authorization: ".$token; curl_setopt($ch, CURLOPT_HEADER, true); curl_setopt($ch, CURLOPT_HTTPHEADER, $httpHeaders ); curl_setopt($ch, CURLOPT_FAILONERROR, true); $results = curl_exec($ch); if (curl_errno($ch)) { echo "<h2>Yes</h2>"; print "<strong>".curl_getinfo( $ch, CURLINFO_HTTP_CODE )."</strong><br><br>"; print "<strong>".curl_error($ch)."</strong><br><br>"; echo "<hr><p>".var_dump($json)."</p>"; } else { echo "<h2>No</h2>"; print curl_getinfo( $ch, CURLINFO_HTTP_CODE ); curl_close($ch); echo "<hr><p>".var_dump($json)."</p>"; //print curl_error($ch); //echo "<p>".$raw_json."</p>"; //header("location: getAll.php?error=Something Happened!"); } return $results; } else { header("location: getAll.php?error=Nothing Happened!"); } ?>
  15. The problem is that, Server doesn't not able to open this url: https://gw1.cardsaveonlinepayments.com:4430/ OR https://gw3.cardsaveonlinepayments.com:4430/?WSDL When you run this url from your browser then it would work and show you some kind of data like: "GetCardType CardDetailsTransaction CrossReferenceTransaction ThreeDSecureAuthentication" But when we try open the url using script then server show warning: "failed to open stream: Connection refused" Same error with cURL, SOAP or fopen. Please check here: http://thelightroomclothing.com/log/test.php Port 4430 is enable on server. I am not sure which settings to ajdust on server to make it work. Please share your experience if you could help.
  16. Hi, I'm trying to get some search results from google by using cURL and preg_match. <?php $curl = curl_init(); curl_setopt ($curl, CURLOPT_URL, "https://www.google.se/#q=horses"); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); $result = curl_exec ($curl); curl_close ($curl); if(preg_match_all('#<cite>(.*)</cite>#', $result, $cite)) { foreach($cite[0] as $cite) { echo $cite . '<br />'; } } ?> It doesnt work, I've used this code on other websites to get other things and it works there. What is the problem? Thank you
  17. I am creating yahoo japan auction bidding website using php. I am almost finish 70% of work. but now i am totally stuck in automate bidding. because yahoo japan does not provide yahoo bid url so i am unable to bid. The bid process in yahoo japan is login before only bid. so i am try to use curl and many more methods but its not working please guide me. Now i got yahoo auction script in java format. how can i convert into php. my java script url is given below. https://code.google.com/p/dlcsdk/source/browse/trunk/DLC+Models/src/com/mogan/model/netAgent/NetAgentYJV2.java?r=52 Please help me i am already post this question in many forums My stackoverflow questions are give below http://stackoverflow.com/questions/18077269/yahoo-japan-action-bidding-api-url http://stackoverflow.com/questions/18288382/yahoo-auction-placebid-api-url http://stackoverflow.com/questions/18954018/yahoo-co-jp-login-using-php-curl http://stackoverflow.com/questions/18981712/auto-login-yahoo-co-jp-using-curl-or-oauth http://stackoverflow.com/questions/18997322/oauth-error-code-400-on-yahoo-co-jp-get-token-request
  18. I am trying to make are autologin for http://www.secretcity.de that i can get information of me Profile in realtime,The Problem are that he sende the data like username and Password to the page but the login not work, he not loged me in, Php not give any error Back, that are the url wo you see what i meen, Demo Test Script I Hope you know what i try to do That are the curl code i use: <?php //Die Session initialisieren $post_array = array( '__VIEWSTATE'=>'dDyphjgAQgsmw340QQiytu4WpJR3SwNCVqFZnz3ZCzbHsHQ/3oejJI3febr/AqRuqEdJvDLN9BwfHYKmO/avLklA6K0I180MHFePUgQl73IW4MGFq537vaQP/FfHV8uA+bimMr00R4tC5IpCYlMYKC8+HAY6Hxz3T7zAvAs63V6s4bQ2RLbCnQgb+rzYrYnogiXoxy4nPQbYHnVJ/XwvY5gM+428ntHIAf6aIXU5HrLF+na6V1StvUTahQ/7Am7Oas0Brk1lGfWgjEwcJyHogAmAwDwhUoD7+x3e3jsGI9pXUoPcih8WJxAMMT+2N8Y71IuOUko5Hmdk1Knr3Yy7qRY8OqGCZH7c29EQ9lnu1jo9EvWTTW8UI7HkOYBgijIhB2ppLrNEz4aRab1AxjIoOzVG4ioMjyiTycyZRsfpgpo1S5cH8p/FBmuU5grW7148o1bjvWx1N293fV5ptVWY2BOqsOYxsPpcfpzosVmRx1NyxPFIF3sownxzU8mUgRhePKB2pOzaIbZ3lN8FjdE/IEWXefFq9vYS0/DOU5qrfbgEswkSoqTiZEuIkBj6EOk2bPP7dFGJIDDeQISFfYyPUs5ZfeEmRcP+kIpgpxgUoEF+eJOkyi9b08elhBZ3LGakZdq5TiVQo2wiQli4E4eDvq54IxMZsO1xhlvAnIdDHv6wJAjr7F2WMl0gf14zXuLF7mXmuvxLWzFwNtocszMtxoY0y5SDx7qwGJwEcjnJnjab6Vu', '__EVENTTARGET'=>'', '__EVENTARGUMENT'=>'', '__VIEWSTATEENCRYPTED'=>'', '__EVENTVALIDATION'=>'Veg3M2INmpZ/vT80g0p8R8Qftlk4zBGw2hQcCuOp8bc0/2AhwQ0s9J0BYFRoL2VQzj+CDamE9C8kPEq37xkvNoEXFoxGEy1B78LDLyV7gbFtL9Ft/G9NTCzNjXK7/R0fSS9EMg==', 'ctl00$oCPH1$txtEmail'=>'USERNAME', 'ctl00$oCPH1$txtPassword'=>'PASSWORD', 'ctl00$oCPH1$butLogin'=>'Go!', 'ctl00$cbMerker'=>'', 'image'=>'', ); $url="http://www.secretcity.de/login.aspx?error=1&ref=Net/RayTransHistory.aspx?aid=389463"; $header = array("POST /login.aspx?error=1&ref=Net/RayTransHistory.aspx?aid=389463 HTTP/1.1", "Host: www.secretcity.de", "Connection: close", "User-Agent: Mozilla/6.0 (Windows NT 6.2; WOW64; rv:16.0.1) Gecko/20121011 Firefox/16.0.1", "Accept-Encoding: gzip", "Accept-Charset: ISO-8859-1,UTF-8;q=0.7,*;q=0.7", "Cache-Control: no", "Accept-Language: de,en;q=0.7,en-us;q=0.3", "Referer: ".$url, "Content-type: application/x-www-form-urlencoded", "Content-length: 0" ); $cookie="cookie.txt"; $cookieFile="cookies.txt"; $ch = curl_init(); curl_setopt ($ch, CURLOPT_URL, $url); curl_setopt ($ch, CURLOPT_TIMEOUT, 60); curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt ($ch, CURLOPT_COOKIEFILE, $cookieFile); curl_setopt ($ch, CURLOPT_COOKIEJAR, $cookie); curl_setopt ($ch, CURLOPT_REFERER, $url); curl_setopt ($ch, CURLOPT_POST, TRUE); curl_setopt ($ch, CURLOPT_POSTFIELDS, $post_array); $result = curl_exec ($ch); $info = curl_getinfo($ch); $errorMessage = curl_errno($ch); $errorNumber = curl_error($ch); curl_close($ch); ?> <?=$result?> and that are the login code on ther page: <div align="center" style=""> <div class="SCLoginHeader" style="text-align: left;"><img src="App_Themes/SecretCity/Images/SCLogin-Title.png" alt="Member's Login" border="0" /></div> <div class="SCLoginBody" style="text-align: left;"> <div align="center"> <table border="0" cellspacing="0" cellpadding="0" class="SCLoginTable"> <tr> <td class="SCLoginDesc">Avatarname:</td> <td class="SCLoginData"><input name="ctl00$oCPH1$txtEmail" type="text" value="sct doc hardi" maxlength="128" size="25" id="ctl00_oCPH1_txtEmail" class="SCLoginField" /></td> </tr> <tr> <td class="SCLoginDesc">Passwort:</td> <td class="SCLoginData"><input name="ctl00$oCPH1$txtPassword" type="password" maxlength="32" size="20" id="ctl00_oCPH1_txtPassword" class="SCLoginField" /></td> </tr> <tr> <td class="SCLoginDesc"> </td> <td class="SCLoginData"><input type="image" name="ctl00$oCPH1$butLogin" id="ctl00_oCPH1_butLogin" src="App_Themes/SecretCity/Buttons/BTN-90-SignIn.png" alt="Go!" border="0" /></td> </tr> </table> </div> <div align="center" style="text-align: center;"> <ul class="SCLoginBullet" style="text-align: left;"> <li><a id="ctl00_oCPH1_lnkSignup" class="SCLoginLink" href="/signup.aspx">Neuen Benutzer anlegen</a></li> <li><a href="lost_password.aspx" class="SCLoginLink">Passwort vergessen?</a></li> <li><a href="resendvalticket.aspx" class="SCLoginLink">Bestätigungsmail nochmal senden</a></li> <li><a href="validate.aspx?noerror=true" class="SCLoginLink">Bestätigungs PIN eingeben</a></li> <li><a href="validated_email.aspx" class="SCLoginLink">Emailbestätigung überprüfen</a></li> </ul> </div> </div> </div><br /><br /> </div> <input name="ctl00$cbMerker" type="hidden" id="ctl00_cbMerker" onfocus="alert(this.id)" /> <div id="ctl00_divUtherverseTrackingIframe"></div> </form> Oki maby you can help me withe that issus doc hardi
  19. Hi, I'm having trouble accessing the Toggl Reporting API. I wondered whether anyone has experience accessing this or similar REST based services? I get the error message 'api token not valid', although I have tried several api tokens that are definitely valid, and also tried encoding the token with base 64 (as suggested to access via http basic auth). I wanted to check whether there are any obvious errors in the code? I'm using cURL as suggested in the documentation but don't have much experience with this. header('Content-type: application/json'); $token = "[myapitoken]";//my api token function get_data($url) { $ch = curl_init(); $timeout = 5; curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_HTTPGET, true); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); curl_setopt($ch, CURLOPT_USERPWD, $token.':api_token'); $data = curl_exec($ch); curl_close($ch); return $data; } $returned_content = get_data("https://toggl.com/reports/api/v2/weekly?&workspace_id=282507&since=2013-05-19&until=2013-08-20&user_agent=[user agent]");//user agent here var_dump($returned_content);
  20. Hello, I'm trying to post data to some remote sites and I think I have something in the wrong order. Any help would be appreciated. Thanks for your time. Here is how its suppose to work... gets array of siteurl from sites gets the id of the ads being posted (multiple ids) selects the ad from received checks to see if the url from the ad (received) is the same as one of the siteurl in the array if it is the same then $url2 should equal $info ['url'] if it is not then $url2 should equal 'http://'.$row['siteurl'].'/ads.php?id='.$info ['id'].''; Basically it checks to see which site the original ad was posted from and then when it post back to that site it post the original url, if it post to a different site then it post a different url. Right now it post these results Ad 1 Site 1 http://siteone.com/ads.php?pageid=clickadd&id=1 Site 2 http://sitetwo.com/ads.php?pageid=clickadd&id=1 Site 3 http://sitethree.com/ads.php?pageid=clickadd&id=1 Ad 2 Site 1 http://siteone.com/ads.php?pageid=clickadd&id=2 Site 2 http://sitetwo.com/ads.php?pageid=clickadd&id=2 Site 3 http://sitethree.com/ads.php?pageid=clickadd&id=2 and it should post something like Ad 1 Site 1 http://siteone.com/ads.php?pageid=clickadd&id=1 Site 2 http://someurl.com // because the ad #1 originally came from site #2 Site 3 http://sitethree.com/ads.php?pageid=clickadd&id=1 Ad 2 Site 1 http://siteone.com/ads.php?pageid=clickadd&id=2 Site 2 http://sitetwo.com/ads.php?pageid=clickadd&id=2 Site 3 http://someotherurl.com // because the ad #2 originally came from site #3 foreach($id as $each) { $sql = mysql_query ( "SELECT * FROM receive WHERE id=" . $each ); $info = mysql_fetch_array ( $sql ); // Work out partial data , excluding url for checking and updating ... $wannasay = array ( "id" => $info ['id'], "site" => $info ['site']; // Generate Query string $dataels = array (); foreach ( array_keys ( $wannasay ) as $thiskey ) { array_push ( $dataels, urlencode ( $thiskey ) . "=" . urlencode ( $wannasay [$thiskey] ) ); } $data = implode ( "&", $dataels ); $result = mysql_query("SELECT siteurl FROM sites"); $nodes = array(); $i=0; while ( $row = mysql_fetch_assoc($result) ) { $nodes[] = 'http://'.$row['siteurl'].'/admin/curl.php'; $nodes2[$i] = $row['siteurl']; $i++; // Here is where I have problems foreach( $nodes2 as $value ) { if($value == $info ['site']) { $url2 = $info ['url']; } else $url2 = 'http://'.$row['siteurl'].'/ads.php?id='.$info ['id'].''; } $postfields = $data.'&url=' . $url2; // Update Full query string .. $node_count = count($nodes); $curl_arr = array(); $master = curl_multi_init(); for($i = 0; $i < $node_count; $i++) { $url =$nodes[$i]; $curl_arr[$i] = curl_init($url); curl_setopt($curl_arr[$i], CURLOPT_POST, 1); curl_setopt($curl_arr[$i], CURLOPT_POSTFIELDS, $postfields); curl_setopt($curl_arr[$i], CURLOPT_RETURNTRANSFER, 1); curl_multi_add_handle($master, $curl_arr[$i]); } do { curl_multi_exec($master,$running); } while($running > 0); for($i = 0; $i < $node_count; $i++) { $results = curl_multi_getcontent ( $curl_arr[$i] ); echo( "\n" . $results . "\n"); } } }
  21. Hi guys, Im trying to make a webscrapper and im having a huge problem when retrieving huge amount of data. I have tried to increase the memory through PHP.ini but its still doesnt solve the problem. The webscrapper I want to make is to retrieve data from database journal and put it into an excel file. While it is working with small datasets, it will run out of memory when retrieving large datasets. Here is the code : x <?php function fetchRawData($url,$search,$currentpagenumber,$numpage,$numrecordtotalsofar) { if($currentpagenumber<$numpage) { //initialise curl $url = "http://ieeexplore.ieee.org/search/searchresult.jsp?queryText%3D".$search."&rowsPerPage=100&pageNumber=".$currentpagenumber."&resultAction=ROWS_PER_PAGE"; echo "<br>new url = ".$url."<br>"; $ch = curl_init(); curl_setopt($ch,CURLOPT_URL,$url); curl_setopt($ch,CURLOPT_RETURNTRANSFER,true); curl_setopt($ch,CURLOPT_FAILONERROR,true); curl_setopt($ch,CURLOPT_FOLLOWLOCATION,true); // curl_setopt($ch,CURLOPT_TIMEOUT,50000); curl_setopt($ch,156,500000000); curl_setopt($ch,155,500000000); curl_setopt($ch,CURLOPT_SSL_VERIFYPEER,false); curl_setopt($ch,CURLOPT_SSL_VERIFYHOST,false); $data=curl_exec($ch); if(!$data) { var_dump(curl_getinfo($ch)); die(); } //parsing data $parsedData = array(); phpQuery::newDocumentHTML($data); $arrtitle = array(); $posttitle=1; if($currentpagenumber<2) { $numrecordsofar=1; } else { $numrecordsofar=$numrecordtotalsofar; } //get the title, author and year of publication foreach(pq("a") as $link) { $title = pq($link)->text(); if($title) { //use regular expression to get the relevant information if (preg_match("*articleDetails.jsp*", pq($link)->attr('href'))&&$gettitle<1) { if(!(preg_match("*View full abstract*", $title))) { $dummyvar=$numrecordsofar+$posttitle; array_push($arrtitle,$title); $countrecord++; $gettitle=1; } } } } //get the number of data foreach(pq("span") as $link) { $title = pq($link)->text(); if($title) { if (preg_match("*display-status results-returned*", pq($link)->attr('class'))) { $countnumberonly = preg_replace("*Results returned*", "", $title); $totalpageint = intval($countnumberonly); //calculate how many pages needed and record the current page $totalpageint = intval($totalpageint / 100)+2; } } } //initialise write to excel $objPHPExcel = new PHPExcel(); $objPHPExcel->getProperties()->setCreator("Maarten Balliauw") ->setLastModifiedBy("Maarten Balliauw") ->setTitle("PHPExcel Test Document") ->setSubject("PHPExcel Test Document") ->setDescription("Test document for PHPExcel, generated using PHP classes.") ->setKeywords("office PHPExcel php") ->setCategory("Test result file"); // Set active sheet index to the first sheet, so Excel opens this as the first sheet $objPHPExcel = PHPExcel_IOFactory::load("IEEE_Scrap.xlsx"); $objPHPExcel->setActiveSheetIndex(0); $objPHPExcel->createSheet(); $row = $objPHPExcel->getActiveSheet()->getHighestRow()+1; //get data from arrays for($j=0;$j<count($arrtitle);$j++) { if(isset($arrtitle[$j])) { $dummyvar=$numrecordsofar+$j; $objPHPExcel->getActiveSheet()->SetCellValue('A'.$dummyvar,$arrtitle[$j]); } else { $dummyvar=$numrecordsofar+$j; $globalIEEE[$tempcount+$j][0]="No Data"; $objPHPExcel->getActiveSheet()->SetCellValue('A'.$dummyvar,"No Data"); } } $objWriter = new PHPExcel_Writer_Excel2007($objPHPExcel); $objWriter->save('IEEE_Scrap.xlsx'); //close curl and phpexcel curl_close($ch); unset($ch); unset($objPHPExcel); unset($objWriter); $currentpagenumber++; $numrecordtotalsofar=$numrecordtotalsofar+$countrecord; set_time_limit(0); sleep(5); $rawHTML = fetchRawData($url,$search,$currentpagenumber,$totalpageint,$numrecordtotalsofar); return $data; } } ?> The logic is first I retrieve the data on a page then putting it into an array after parsing it then initalise phpexcel to write the data from the array into excel then unset cURL and phpexcel and then move on to next page. Sorry the code is a bit messy as I have tried so many modifications but still cant get it work. Please help me !
  22. I have the need to use cURL to request content from my own website. I've been using the same function on several sites on my own server but recently needed to set up my code on a client's Windows machine (running apache and php w/ cURL enabled). Something about the new host's configuration causes all my cURL requests to timeout when calling URL's on the same site as the code initiating the request. if I set the url to yahoo.com or bing.com it works but if I set it to $_SERVER['HTTP_HOST'] or my domain name it times out. any thoughts?
  23. I'm trying to grab xml data from a url that I pass get variables through and I get this error when trying to echo out the results. org.xml.sax.SAXParseException: Premature end of file. I'd also like to parse the xml data once it is recognized but the first priority is to recognize or at least let me know that the code is working. Here is the code I have below and I'm trying to get a shipping cost for an item and the necessary information that needs to be passed through the url to the shipping company's custom shipping rate calculator uses the get variables to calculate the exact shipping cost. When I use an example link and just add in some get variables manually and then enter it into the browser's url bar, the xml data shows up fine and there is the total amount for shipping but I don't understand why I can't get curl to do it. Thanks for your time! $xml = 'weight_system="IMPERIAL" shipper_number="000222000" destination_postal_code="'.$data5['zip'].'" service_type="1" '; $xml2 = ' total_pieces="'.$value.'" total_weight="'.$weight.'" '; $token = 'KFSSAFD4DSE'; $base_url = 'https://www.shippingco.com/XML/RatingXML.jsp'; $request_url = $base_url . '?'.htmlentities('shipment=<shipment ' . $xml . '><total ' . $xml2 . '/></shipment>&token=' . $token); // Get cURL resource $curl = curl_init(); // Set some options - we are passing in a useragent too here curl_setopt_array($curl, array( CURLOPT_SSL_VERIFYPEER => false, CURLOPT_SSL_VERIFYHOST => 2, CURLOPT_CAINFO => '/home/website/public_html/cacert/cacert.pem', CURLOPT_RETURNTRANSFER => 1, CURLOPT_URL => $request_url, CURLOPT_USERAGENT, 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)', CURLOPT_FAILONERROR => false )); // Send the request & save response to $resp $result = curl_exec($curl); echo curl_error($curl); echo $result; // Close request to clear up some resources curl_close($curl);
  24. Sample json query : https://api.datamarket.azure.com/Bing/Search/Web?$format=json&Query=%27Xbox%27 . If you click on the link you are prompted for a username and password. both fields just require you to insert your Bing API. How can I send the password with the query attached? I'm using cURL to send the query. I have tried using the normal method outlined by bing to receive the search results but it doesn't seem to work for me.
  25. I am trying to login to the website in the below example. This script logs in using a username and password but then is directed to a page with a security question. This is where the problem is, I am unable to POST my answer on this page. The result of my output is just the page with the security question. $username = 'XXXX'; $password = 'XXXX'; $loginUrl = 'https://www.dandh.ca/v4/dh'; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $loginUrl); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, 'Login='.urlencode($username).'&PW='.urlencode($password).'&Request=Login&formName=Login&jsEnabled=0&queryString=&Platform=Full&btLogin='.urlencode('Log In')); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/cookies.txt"); curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/cookies.txt"); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_exec($ch); // process main login curl_setopt($ch, CURLOPT_POSTFIELDS, 'securityAnswer=XXXX&Request=postForm&formName=loginChallengeValidation&btContinue=Continue'); echo curl_exec($ch); // process security question curl_close($ch); Here is the source code of the security question HTML form: <form name="securityForm" id="secForm" method="post" action="/v4/dh"> <input type="hidden" name="Request" value="postForm"><input type="hidden" name="formName" value="loginChallengeValidation"> <table border="0" align="center" cellpadding="10" cellspacing="0"> <tr> <td><span style="font-weight:bold;">Security Response</span> <br> <p> Please answer the following security question. Once the answer is confirmed you can continue. </p> <p style="margin-left: 20px;"> Question: What is the name of your best friend from childhood?</p> <p style="margin-left: 20px;"> Answer: <input type="text" id="securityAnswer" name="securityAnswer" size="50" maxlength="50"> </p> <p style="margin-left: 20px;"> <input type="submit" name="btContinue" value="Continue"> </p> <p> <i>If you do not know the answer to the security question, please email <a href="mailto:passwords@dandh.com">passwords@dandh.com</a> and they will send you a temporary password. <a href="/v4/dh?Request=postForm&formName=LogOut">Click here</a> to return to login page.</i> </p> </td> </tr> </table> </form>
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.