Jump to content

Scraping A Website For Data


mattm1712

Recommended Posts

  • 10 months later...

ok, I've been struggling with this topic for about 6 months now, this is something I just can't seem to grasp;

 

I've been trying to scrap content from this site:

 

http://www.trulos.com/Trulos/Post-Truck-Loads/free-truckload-board.aspx?STATE=AL&Equipment=Reefer

 

to add to my own load board, 

 

I seem to be stuck, I've tried just about every task: but this site seem to not have class id's just simple <td> and tables, could someone help me out, and maybe provide an sample

 

I'm trying to scrap the loads that they have posted, 

 

7/24/2013 Montgomery  AL Robert LA Full 53 Reefer 0.00 21 0 Dispatch (800) 580-3101 Map

7/24/2013 Montgomery  AL Birmingham AL Full 48 Reefer 0.00 40 1 Dispatch (800) 580-3101 Map

 

Then I want to add it to my sql database that would show in my site: 

 

http://www.loadsready.com/

 

any help would be greatly appreciated. 

Edited by topcodedeveloper
Link to comment
Share on other sites

Wow, that's not quite the answer i was looking for, i would expect a more professional answer. Now I remember why I don't user these type site's oh and by the way you might want to change your profile pic,

you resemble one one of inmates were I happen to work. 

 

 

Thanks for the help!!!!

 

OH Just an FYI, don't matter to me if you decide to band my from this site, it would just show me how UN professional you are.

 

Virgin Media IP (82.8.81.100)
 
Bournemouth, United Kingdom   
Edited by topcodedeveloper
Link to comment
Share on other sites

Well I'm sorry, but there's no professional way to say you're a bad person for wanting to steal someone else's content. And even if there was, I wouldn't say it that way..because why should I act professional towards someone trying to rob someone else? Also, this is a free, public help site. If you are looking for someone "professional" then go hire someone. However, I would think if someone were professional, they wouldn't be trying to steal other peoples' content, nor agree to be a part in it on any level. So IMO you and I obviously have 2 different ideas of what "professional" means. I think you're looking for the "mob" type, not the "business" type.

Link to comment
Share on other sites

Also, the site you liked to has Term of Service which state, in part:

 

 

 

Except for public domain material, the content on this site is protected by intellectual property laws, including U.S. copyright laws. You are hereby granted a non-exclusive license to use the material contained within this site, including transmitting email of individual stories to friends directly from this site. You are also granted a limited license to print copies of any Content posted at the site, but only for your personal use. Except as expressly provided above, all rights are reserved. Among other things, except to the extent required for the limited purpose of reviewing material on our site, electronic reproduction, adaptation, distribution, performance or display is prohibited. Changes to or deletion of author attribution or copyright notices are prohibited. Commercial use of any of the Content is strictly prohibited. Use of any of our trademarks as metatags on other web sites also is strictly prohibited. You may not display this website in frames (or any of the Content via in-line links) without our express written permission, which may be requested by contacting us at Support@Trulos.com. You may, however, establish ordinary links to our site without our written permission.  dsa

 

Also

ok, I've been struggling with this topic for about 6 months now, this is something I just can't seem to grasp;

 

Six months?

Edited by Psycho
Link to comment
Share on other sites

Calling someone a thief and not being able to code as a response wasn't too nice, I surely wouldn't like it, neither would you.

 

They could have easily had special permission from the owners of http://www.trulos.com

 

I doubt was the case or would have stated that fact, but should think of this the next time someone is asking for help on the same subject.

Link to comment
Share on other sites

Right. I get what you're saying, but when literally 99.999% of all cases of scraping another site involve people NOT getting "special permission" from the owner, I think my money is on thief. If someone were determined to get permission for that sort of thing, they should work with them to create a better method in the first place, like make an API to request the info from the server.

Link to comment
Share on other sites

Nonetheless.. you don't just show up somewhere asking how to scrape someone else's site and expect it not to be taken as "I'm trying to steal someone else's content." It ranks up there with people posting "hack my site" threads. It is common knowledge that these sorts of questions almost certainly come from people trying to do things they shouldn't be doing. The legit people are the exception, not the norm. So it is on them to prove that they are legit, and I do not feel bad for assuming they are not legit unless proven otherwise. I mean come on.. it's like if some random dude came up to you and asked you for help breaking into a house and taking things. Not even saying he totally knows the owner and it's totally okay. I do not think it is unfair of me to assume he's a thief asking for help to steal something.

Link to comment
Share on other sites

I will admit I was a little butt hurt, with the remark made, so yes I may have been a little harsh my self in responding to the thread.

 

Considering You may not know me, nor I know you, how can you judge one person just based on fact, with out all the evidence in front of you.

I assure you that my attentions are good, in fact I've tried contacting the sites owner, finding out their online form has a SQL error, so I've tried contacting them via local email instead, and not heard from them yet.

 

I was simple trying to reach out for some assistance, but it seems that I may have came to the wrong place. 

 

And as for capturing your IP address and location I'm able to pull this information from my server.

 

So again, how quick one is to judge. And!! as for coding I'm sure you may not no every language there is in coding, When I stated I've been struggling with this issue for the past six months, I should of mentioned that I also have a full time job, and only do this on the side, in which doesn't give me much time. 

 

So with that said, I'll apologizes, for my action and with draw my account from the site and just move along.

 

Thanks any way for all your help..

Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.