etrader Posted March 10, 2011 Share Posted March 10, 2011 Is it possible to extract all links from a website (not a single webpage) by php? I am asking about the general idea, as I wish to customize: e.g. from a specified directory and certain domains only. Thanks Quote Link to comment https://forums.phpfreaks.com/topic/230267-extracting-all-links-from-entire-website/ Share on other sites More sharing options...
btherl Posted March 10, 2011 Share Posted March 10, 2011 Yes it is possible. Quote Link to comment https://forums.phpfreaks.com/topic/230267-extracting-all-links-from-entire-website/#findComment-1185813 Share on other sites More sharing options...
Maq Posted March 10, 2011 Share Posted March 10, 2011 Is it possible to extract all links from a website (not a single webpage) by php? I am asking about the general idea, as I wish to customize: e.g. from a specified directory and certain domains only. Was there something specific? Quote Link to comment https://forums.phpfreaks.com/topic/230267-extracting-all-links-from-entire-website/#findComment-1185830 Share on other sites More sharing options...
silkfire Posted March 10, 2011 Share Posted March 10, 2011 Yes it's very possible and PHP is oftentimes used to perform this. Use cURL + preg_match_all. What's your site and what links to you want to extract and which links do you want to filter out? Quote Link to comment https://forums.phpfreaks.com/topic/230267-extracting-all-links-from-entire-website/#findComment-1185838 Share on other sites More sharing options...
etrader Posted March 10, 2011 Author Share Posted March 10, 2011 Thanks for your kind attention. For example, extracting pdf links from http://www.getfreeebooks.com Actually, I do not know how to crawl pages throughout a website to catch links Quote Link to comment https://forums.phpfreaks.com/topic/230267-extracting-all-links-from-entire-website/#findComment-1185851 Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.