Jump to content


Looking for an experienced PHP developer for ongoing work

php curl bots

This topic has been archived. This means that you cannot reply to this topic.
No replies to this topic

#1 melbahtoast

  • Members
  • Pip
  • Newbie
  • 6 posts

Posted 18 April 2013 - 11:40 PM

I'll start off by saying this is not a small gig.  I am primarily an html/css girl so I will try to explain this the best I can.


I am looking to have 100+ bots scripted to pull and return specific info to the front end of my site based off of user inputted data, which would also be handled on the front end.  I have the exact steps that each bot must follow (in plain english) in order to return results.


Currently I have a working version of this using CURL, but it is very primitive.  Right now, I gather information from the client using a form on the front end of my site, which emails me the data in a specific way.


ex: Client enters

Name: Bob

Color: Red

Number: 82


Email kick back will look something like

"Bob 82"

"82 Red"

"82" Red

"Bob" Red


I then drop the footprints from my email into the CURLbot I have, which searches google to return results.  If results are found that match the user submitted info on any of the 100 websites I have specified in a CSV, CURLbot will put an "x" in the field next to the url.  If it is not found, it prints a link (which is different for every site in the csv) instead of an "x".


So essentially, I need this in a much more user friendly "automated" way.  Also, instead of utilizing google, I would need each bot to scrape (I really hate using this word because I feel like this isn't really a "scraper") the web site I specify.  The majority of these bots can "scrape" the web site itself, however a few would have to utilize google to find information on the specific web site.


Also, I would need help integrating this into the current web site.  This tool can only be utilized after the client has paid, so possibly set up of user accounts to restrict access, and so users can look back on search queries.


I believe how I would like it to work would be:

1. client inputs data

2. data stored in database/create queue 

3. bots collect information either from database, or utilizing the footprint script I already have, scrape

4. print/return results to client side


In addition, I would need set up of hosting and servers for the bots, as they cannot reside on the same server as our site.


This would be on going work as well, as I am regularly adding, changing, dropping the web sites on the list of sites I have.


I am ideally looking for someone who is positive, outgoing, has vested interest, and wants to be a part of this project, not just as a programmer but as part of our team.


All reasonable estimates will be considered.  I'm aware this isn't a quick gig, and the compensation is there, however, as with all projects, I do have a budget.


PM me or skype id: melissamcveigh


Thanks for your time!