Jump to content

bschultz

Members
  • Content Count

    468
  • Joined

  • Last visited

Community Reputation

0 Neutral

About bschultz

  • Rank
    Advanced Member
  • Birthday 07/20/1974

Profile Information

  • Gender
    Male
  • Location
    Minnesota, USA

Recent Profile Visitors

The recent visitors block is disabled and is not being shown to other users.

  1. Now I feel dumb...submit button! Thank you!
  2. <?php $username = 'xxx'; $password = 'xxx'; $postdata = '?username='.$username.'&password='.$password; $url = 'https://www.premiereaffidavits.com/scripts/cgiip.exe/WService=webprime/login.html'; $ch = curl_init(); curl_setopt ($ch, CURLOPT_URL, $url); curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE); curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"); curl_setopt ($ch, CURLOPT_TIMEOUT, 60); curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt ($ch,
  3. I'm trying to login to a site, navigate a couple of pages deep, and then scrape a table. I had all this working, but the site changed their login workflow...and I can't figure out what I need to do to make this work. The form used in a browser to login has itself as the action page. In a browser, you are automatically redirected to another page, that puts a random get variable at the end of the URL. Here's the main login page: https://www.premiereaffidavits.com/scripts/cgiip.exe/WService=webprime/login.html When I run a standard curl login code...I only get the login page returned
  4. This is always echoing FIRST...even for today (12/30)...any ideas why? <?php $year = date('Y'); $yearplus = ($year + 1); $first = date('m-d', strtotime("12-25")); $second = date('m-d', strtotime("01-01")); if (date('m-d') <= $first || date('m-d') >= $second) { $first_monday = date('Y-m-d', strtotime("January 2 $year last Monday")); echo "FIRST"; } else { $first_monday = date('Y-m-d', strtotime("January 2 $yearplus last Monday")); echo "SECOND"; } echo $first_monday; exit; ?>
  5. The broadcast industry (radio and TV) has a special calendar. The week starts on Monday. Week #1 is ALWAYS the week with January 1st. So, if January 1st is on a Sunday, week #1 will include December 26th through January 1st. This differs from the ISO standard used by PHP. I don't know where to start to write a function to give me the broadcast calendar week numbers for a given date. Google didn't have much...and Stack Overflow had a couple examples...but none of them worked. They all showed today being week #53, where today (December 30) is in week #1. Any help would be appreciat
  6. This is fixed. Turns out one of the pages that the 4th page was calling via GET does NOT require a password, and it has all the content I need. I exploded the string it returned and got all the info I needed. Thanks.
  7. Maxxd, when using a browser, there is a button to press to go from page 1 to page 2. This button is just a link, so the second curl request uses this link as the URL. Using Firefox Developer Tools - Network, page #4 makes two GET calls to two other pages (which includes the SQL Selects), then the content loads on page #4 Here are the two external page requests curl 'http://209.151.229.186/AffWeb_USRN/V2/ASP/GTD.asp?SQLCMD=spGetStationOptions%20%27WBJI-FM%27,%20%27Virtual%20News%20Network%20MF%27,%20%2710/19/2020%27,%20%2710/19/2020%27&DT=1603034713365' -H 'User-Agent: Mozill
  8. 4 pages deep means login page (page 1). Simulate a link click to page 2. Simulate a link click to page 3. Simulate a link click to page 4. Pages 2 and 3 have no javascript or Ajax coding. Page 4 does! Pages 2 and 3 have some coding that is tied to the login to display certain info. As far as I can tell, page 4 uses AJAX to set some database calls...thus without the AXAX info passed via CURL, I get AJAX:0 error. I'm assuming the AJAX:0 error is in the code of the page...but when I visit that page via a browser, it works...so no error. What code would you like me to post? Page 1,
  9. I'm trying to login and scrape a page 4 pages deep. I can get to the fourth page...but that page only returns AJAX ERROR:0. I know NOTHING about AJAX calls via Curl. Can someone please help me with what to look for in the source code of the 4th page (when using a browser) to what I'm supposed to pass along via CURL? If you need the source code or login credentials to see what's happening in the background, I can generate a temp password for you. Thanks!
  10. Upon further investigation, there is a cookie being set by a JQuery script...which in turn sets the rowid. So, how can I get CURL to interact with Java? Headless browser? I've never had much success with headless browsers before. Any tips? Thanks!
  11. But the rowid isn't set until you submit the form...so parse-url returns NULL. The flow of the login is: login page -> formprocessing page -> landing page. In the script, I'm sending a POST to the form processing page. Once that is done, the form processing page sets the rowid. I can see the rowid in my browser, but CURL doesn't know that value.
  12. My real job is as a radio announcer. We are required to play advertising commercials for various programs that we broadcast. I wrote a script a few years ago to automatically login to the providers website (PHP, Curl) and download the mp3's that we are supposed to play each day. Now, the provider has updated their website and changed the login process. It used to be (in a browser) that when you logged in, you were taken to a landing page with a unique "rowid" in the in URL. Once you knew that rowid, you could simply go to that page, and bypass the login process. Not very secure...but
  13. I have a given date on a webpage, that I'm scraping to insert into a DB. The date is in this format: Sun, Feb 9<br />3:00 PM ET I need to insert this into the DB in this format: 2020-02-09 15:00:00 It will always be this year...How can I change this data to be inserted correctly? I'm trying this...and it's inserting as 1969 $date = 'Sun, Feb 9<br />3:00 PM ET': $healthy = array("<br />", " ET", "Sun.", "Mon.", "Tue.", "Wed.", "Thu", "Fri.", "Sat.", "Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"); $yummy = array(" ", "2
  14. I got it working...left the dollar sign out of the function call getplayer('qb1'); and then an if / elseif statement inside the function: if ($position == 'qb1') { $new_position = $qb1; } elseif ($position == 'qb2') { $new_position = $qb2; } Thanks....
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.