[SOLVED] Building a webcrawler

[MDanz]

i hope this isn't difficult i just about finished my site, now i want to populate it.

 

I have an insert form with 4 fields(url, name, info, keywords).

 

i want a webcrawler to go around the web collecting and inserting into these 4 fields and submitting.

 

how would i go about this is it difficult?

 

whats the easiest language to do this in?

[paralyzah]

You cannot have a field containing keywords. Any database field should only hold one piece of information. Keywords are many pieces of information. Your database should have two tables and look something like this:

 

Sites: Key, URL, Name, Info

Keywords: ForeignKey, Keyword

[keeps21]

You cannot have a field containing keywords. Any database field should only hold one piece of information. Keywords are many pieces of information. Your database should have two tables and look something like this:

 

Sites: Key, URL, Name, Info

Keywords: ForeignKey, Keyword

 

That would be the better option. But you could store all the keywords in one field.

Then when pulling the keywords out of the database "explode" that field to get the individual keywords.