Jump to content

[SOLVED] Building a webcrawler


MDanz

Recommended Posts

i hope this isn't difficult i just about finished my site, now i want to populate it.

 

I have an insert form with 4 fields(url, name, info, keywords).

 

i want a webcrawler to go around the web collecting and inserting into these 4 fields and submitting.

 

how would i go about this is it difficult?

 

whats the easiest language to do this in?

Link to comment
Share on other sites

You cannot have a field containing keywords. Any database field should only hold one piece of information. Keywords are many pieces of information. Your database should have two tables and look something like this:

 

Sites: Key, URL, Name, Info

Keywords: ForeignKey, Keyword

 

That would be the better option. But you could store all the keywords in one field.

Then when pulling the keywords out of the database "explode" that field to get the individual keywords.

Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.