Jump to content

Sequential URL Fetching


phantom552

Recommended Posts

Let's say I have a laundry list of *possible* XML files with a sequential commonality, IE:

 

http://somewebsite.com/archivedthing/1.xml

 

to

 

http://somewebsite.com/archivedthing/100000.xml

 

With no way of knowing beforehand how many of the possible XML files in that range are valid.

 

Is there a way to automatically run through all of these URLs and dump the valid numbers into a delimited file of some kind? I think I can figure out how to check each URL for the presence of the XML data I want, but I'm not sure how to automatically progress through each possible URL, and dump the data I need into a file I can use later.

 

Thanks!

Link to comment
Share on other sites

Made a little progress, but I know my syntax is fubar on the $xml line, trying to add $value between http://www.website.com/item= and &xml so I end up with http://www.website.com/item=____&xml where ____ is a value in the range. Any help is appreciated.

 

 

$possibleids = range(1,100);
foreach ($possibleids as $value){
$xml = 'http://www.website.com/item=$value&xml';
}

Link to comment
Share on other sites

This is certainly not something you want to be doing using your web browser. This is the sort of thing suited to a command line script. I'm not sure why you would not know the exact xml urls that need reading, please elaborate.

 

You have a simple error in your code. Variables are not translated inside single quotes, only inside double quotes. If you want to use single quotes then the following will work:

 

$possibleids = range(1,100);
foreach ($possibleids as $value)
{
$xml = 'http://www.website.com/item=' . $value . '&xml';
}

Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.