Jump to content

[SOLVED] Building a webcrawler


MDanz

Recommended Posts

i hope this isn't difficult i just about finished my site, now i want to populate it.

 

I have an insert form with 4 fields(url, name, info, keywords).

 

i want a webcrawler to go around the web collecting and inserting into these 4 fields and submitting.

 

how would i go about this is it difficult?

 

whats the easiest language to do this in?

Link to comment
https://forums.phpfreaks.com/topic/174237-solved-building-a-webcrawler/
Share on other sites

You cannot have a field containing keywords. Any database field should only hold one piece of information. Keywords are many pieces of information. Your database should have two tables and look something like this:

 

Sites: Key, URL, Name, Info

Keywords: ForeignKey, Keyword

You cannot have a field containing keywords. Any database field should only hold one piece of information. Keywords are many pieces of information. Your database should have two tables and look something like this:

 

Sites: Key, URL, Name, Info

Keywords: ForeignKey, Keyword

 

That would be the better option. But you could store all the keywords in one field.

Then when pulling the keywords out of the database "explode" that field to get the individual keywords.

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.