gizmola

It is non-trivial. Without a LOT more specifics about what you want to do, and why, there is no way to answer your question without writing a novel on the subject.

It's not clear to me exactly what you are after. Also tigerdirects pages are pretty messy. In this example I just dump out the "ProductReview' portion of the DOM: error_reporting(E_ERROR); $urls[] = "http://www.tigerdirect.com/applications/SearchTools/item-details.asp?EdpNo=2819129&CatId=4938"; $urls[] = "http://www.tigerdirect.com/applications/SearchTools/item-details.asp?EdpNo=1808177&csid=_61"; function curlload($url) { $ch = curl_init(); curl_setopt($ch, CURLOPT_URL,$url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 1); $source = curl_exec($ch); return $source; } foreach ($urls as $url) { $source = curlLoad($url); $dom = DOMDocument::loadHTML($source); $prodReviewElement = $dom->getElementById('ProductReview'); $prodReview = $dom->saveXML($prodReviewElement); echo "***********************************************\n\n"; echo "$url\n"; echo "***********************************************\n\n"; echo $prodReview; } Have a look at the DOM manual, domdocument etc. The only tricky thing I saw was that they often don't use id's so if you plan to try and extract individual elements, it looks like many of them would be by class, where you'd have to do an XPath search, which is a bit more complicated, but still the best approach.

More information on what is being sent to the VB application would be helpful. Also, what is the Xara authored page? Is it static html? PHP is very well suited to a task like this because you can freely intermix HTML and php in a php script. So I would suggest you assume you'll use PHP, and start by moving your entire .html page into a PHP script exactly as is. You can then have something like this: The status of the System is: You can easily drop in and out of php blocks in this manner. PHP will start with the assumption it is parsing HTML until it sees a php start tag. You are then simply left with the task of updating the status. To limit complexity, I would suggest you ditch the vb program, and just write a simple php script that your agent can make a simple get or put to. Something like this: //updatestatus.php if (isset($_GET['status']) { $status = $_GET['status']; // Do something to persist status } If your client can make an http GET request, you are 90% of the way there. Without knowing more about the possibilities of the client process that provides the update information, it's hard to advise you. At that point you have a variety of options for how to persist your status. The script could write the status to a file, or to a simplexml db, or any of numerous other options. One really simple "2 birds with one stone" option would be to design the code to set a key in APC cache, and have your display script get the key from there. You would simply have to supply a default in case the key doesn't exist. If these updates will be relatively frequently APC could be a really good option, as it's an in-memory cache, that also provides php opcode caching of the scripts themselves. APC is very simple to install usually, as long as you have pecl, however, I have limited experience with windows servers, and getting apc going. You can read more about it here: http://php.net/manual/en/book.apc.php Regardless of what you ultimately decide, PHP should get you up and running in minimal time.

It looks to me like your problem doesn't involve curl at all. It's instead in trying to parse out the portions of the data you want from the tigerdirect markup. Trying to find variable data inside html markup using simple string matching or regular expressions is notoriously painful and error prone. A much better solution is to take the page and use the DOM functions to find and extract the portions you need.

Jim, Clearly you do not understand that you have not provided enough information for us to help you fully, and you are in denial about that. I'll just start with #2. To be fair you did provide an example: wp_s2member_custom_fields a:2:{s:6:"reason";s:2:"ue";s:6:"county";s:1:"4";} Now even if it wasn't already obvious that when an array or object gets serialized it's data gets encoded, and that you're interested in the 'reason' column attribute. Here's a program that illustrates this clearly (and one you should have written yourself if you don't understand these concepts. $t = array('reason' => 'ue', 'county' => 4); var_dump($t); $s = serialize($t); echo $s; $unserialized = unserialize($s); var_dump($unserialized); The output of this program: array(2) { ["reason"]=> string(2) "ue" ["county"]=> int(4) } a:2:{s:6:"reason";s:2:"ue";s:6:"county";i:4;}array(2) { ["reason"]=> string(2) "ue" ["county"]=> int(4) } So we can extrapolate from this that the original data in the "reason" element as "ue" and that county was apparently a string containing "4". It's a mystery to me how you would think that this is enough information for anyone to figure out in specific terms what your code should be. We don't even know what "ue" indicates, and we sure don't know what county the number "4" is, or how you would deduce whether that number tells you anything about whether this is "one of the 10 colleges in Indiana". I'm not going to continue to break this down further, as it should be painfully clear at this point, that you need to put more effort into describing your problem, and clearly describing what you are trying to do. Even without a clearly written question, the initial advice of people was -- you're going about this the wrong way. To make an analogy, this is a bit like someone going to a car forum, and stating that they want to put 5 monster truck tires on a VW bug so they can drive sideways down the freeway. After some back and forth, people accepted that the entire idea is ludicrous and wrong headed, and fraught with issues that you will only encounter were you to get even part way down the road, they nevertheless accept the challenge of your premise and endeavor to advise you, starting with the fact that they really need more information about the specific model of Bug, and exactly what roadways you plan to drive down. In essence your reply is -- but I already have the VW! Let's just be clear about what phpfreaks does. We help people who are putting in a good faith effort to learn how to program by advising, clarifying and from time to time giving a push in the right direction. We're not here to write code to your specifications. Already three times now in this thread you've seemingly ignored the opportunity for you to investigate a suggestion. I wrote: You either ignored this or didn't read it. Since your question involved serialized data, I thought you understood what serialized data is. Had you followed my advice, you might be well on your way to being done already. Psycho suggested: Your reply: "Can't I just get help on the question I asked? What you linked doesn't apply to my needs, and I'm not the only one asking about serialized data on this forum. " Let's straighten some things out -- First off, nobody here is working for you, or for phpfreaks for that matter. Everyone involved is volunteering their time. Secondly, you don't know what the hell you are talking about. Your question is not about serialized data, it's about having a mysql string that has a pattern of characters in it that you want to SELECT into a computed column. These strings happen to be the output from the serialize function but they could just as well have been base64 or json, or xml. All that matters is that you have varchars and you want to find patterns in those varchars. PFMaBiSmAd provided you this very helpful snippet: This shows you a great example of how you can use mysql string handling functions to locate and extract the reason data. In short -- you have been given a number of different options. Get busy analyzing them. Test out PFMaBiSmAd's code. This is a "teach a guy to fish" forum. Stop telling us about your query that does 1/2 of the job, and none of the part that you are floundering on. If this task is beyond your capabilities, then perhaps you should hire someone to do the work for you.

For perspective: I'm 28, and I worked professionally in college. Including that job as a programmer in college, I've had 5 jobs in 10 years. My salary has doubled in those 10 years, and I've always quit in favor of a more advanced company (except one time I was fired for "performance reasons" 2 weeks before my stock options would have vested and 3 weeks before the company was sold for $100 million. Convenient for them that I was slacking off right then). Ouch man, that sounds criminal.

What do you have so far?

DROP your hit_url_content key and hit_url_item key and replace them a concatenated key of (hit_url_content, hit_url_item) and try the explain again. Does it change?

We don't have any information on how you are invoking this script. I don't see anything that would explain a double insert, other than the script being run 2x. Based on the information we have so far, I'd try Firefox with firebug turned on, and look at the net tab.

I wasn't part of that discussion, although I do understand that sometimes people are backed into a corner. Serializing data and storing it in a string, and then trying to query on that data, won't scale at worst, and can become a bottleneck and performance suck at best. In your query above you do a LIKE '%something%'. I'm not here to preach, but simply to inform that any LIKE '%something%' query means the entire table must be scanned, every time you query. With that said, the best answer I can give, is that you need to unserialize the data back into a variable, and then loop through that data, looking for the key in question, and comparing it to the values you want.

If this is important data that you want to query on, why are you seralizing it in obtuse strings that can't be queried? That defeats the entire purpose of using a relational database and forces you to write volumes of code. For example, if you ever want to query and get a list of people who "are a fan of highschool basketball" + some other criteria, you guarantee with this structure that you will read every single row from the database every single time you want that list. Then in reading them you'll need to deserialze and THEN check the criteria. Talk about painful....

What client library do you currently have installed? It ought to work with that one already....

Have you inspected each file to see if there is any difference? Does the file validate? Could one or the other server be redirecting? Try looking at both of them with Chrome developer or Firebug and look at the NET connections, headers etc. Something must be different.

There is no reason for a normal user to tamper with the links, images, css you are serving, and no reason to be concerned about it, if they do.

Congrats to you Scootstah and Thorpe! Let me be the first to welcome you to the "Man, Fatherhood sure does change your life for real, I mean like a whole lot, and I'm not f-ing kidding, man" club. In this club mostly, what you do is look at all the naive single people who have no clue, shake your head, and chuckle to yourself about how clueless they are, and how you were just like them once. I'm telling you, it's something.

Here's where I question whether or not what I wrote previously was too cryptic. Yes you most certainly can change something. You can use PDO. See http://www.php.net/manual/en/ref.pdo-sqlsrv.php. Not unlike ODBC, PDO will give you a single interface that lets you talk to both the databases, and it has bind variables, the use of which eliminates the need to worry about sql injection, or the need to run escape routines on strings.

The structure of the class is really something you should decide upon, and that requires context. How often will this be built upon? Will you be adding similar features to the system that could reuse some of the class in whole or part? Have you applied DRY? Are there clear patterns of repetition that could be refactored into small methods? Would you like to build unit tests for your code, so that as things are added to or changed in the future, you could catch breakage in the future? It looks like you're implementing some design patterns already, so as long as the classes involved implement your intentions I wouldn't be concerned. With that said there some things that raise some reuse flags. For example, you have a hardwired holidays list: $holidays = array("02-01-2012", "06-04-2012", "09-04-2012", "07-05-2012", "04-06-2012", "05-06-2012", "28-06-2012", "27-08-2012", "25-12-2012", "26-12-2012"); Is this system going to expire at the end of the year? If not, I would think you would want to be able to update that list in some way, without having to change the code of this class. Later you do logging and mailing of information. I would again assume that logging to a file is something you may want to do in other places, and a log class would be advantageous. $csv = $_SERVER['DOCUMENT_ROOT']."/var/log/newcustomersadded.csv"; $handle = fopen($csv, 'a'); $data = "\n".date ('Y-m-d H:i:s').",".$orderId.",".$codeStr.",".$customer->getName(); fwrite($handle, $data); fclose($handle); mail('xxxx,'New Customer', 'New Customer Created: '.$codeStr); All the database code is very specific which makes me question the purpose of this class, considering it appears to be designed to process certain types of events. It's not clear how that would be put together in your final application. You also have a lot of DRY database code. A wrapper class or ORM would be helpful in that regard, but even more so, I can't recommend enough the use of PDO.

Yeah, thanks for reporting this. We know there is a problem but I have been short on time to look into it. It's on my list.

bugzy: You should call move_uploaded_file first. Then as Barand suggests, use that copy with its persistent location/name that you assigned, as the basis for making your thumbnail versions.

^^^^^^^^^^^^

Let's start with something simple: You write your update script in this manner: - SELECT all rows that are late AND haven't been alerted --- foreach through the results ----- send an email -------- update the status of the row to indicate they were alerted Now you put that in a cron job that runs every second What do you think might happen if 2 of those scripts were running at right about the same time? Do you suppose that people might get 1-2-3 or more emails? If you actually looked at the link to the flock manual I provided you would have seen this: ( flock -s 200 # ... commands executed under lock ... ) 200>/var/lock/mylockfile That example isn't what you would want -- you would want an exclusive lock, so a slight adjustment would need to be made: #!/bin/sh #runjob.sh cd /location/of/yourscript/ ( flock -x /usr/bin/php -f yourscript.php ) 200>/var/lock/mylockfile You then setup cron to run the script every few seconds, and you are off and running, knowing that even if the execution time of the script exceeds the cron interval, a second copy of the script will not be run simultaneously. As for queueing systems, I think I was pretty clear that a scalable solution to your problem will require a more complicated solution. I really have no idea what your skill level or aptitude is, and it's pretty unreasonable to think I would, however, beyond that, you have a mistaken impression of the purpose of our replies. If this was an email thread, I'd be in essence working for you for free. I have no interest in that, nor does anyone else who helps out here. Almost all of us make our living as professional programmers. We post here, because we are helping the OP, but also, are contributing to a conversation which gets archived and not infrequently indexed in google, and when other people face similar problems, they get pointed here. However more directly to your question, yes I'm helping you directly as well, because I've given you a heads up, that if you build some sort of effective business on top of this, it will become a bottleneck, and can't be effectively deployed on anything other than a single server. Since I do most of my work in the cloud, using clusters of servers, this is something I deal with every day. When I get into an area that is new to me, I have to spend time reading and doing research and even at times prototyping or doing POC applications. I simply pointed you in that direction, with no particular expectation that it would be of value or not. I can't teach you a thing, nor can anyone else. However, what we can do, is point you in a direction, and the rest is up to you. This place works best when people do their own research and come back with focused questions. I don't take offense easily, so the fact that you chose to respond in such a combative fashion, doesn't bother me personally. We could go back, and find probably hundreds of threads where I replied to someone, and they then marked their thread solved. And I could also provide anecdotal evidence, in rough accounting of people who thanked me in their threads, or sent me an email. I have no doubt that I've helped a lot of people over the years, so your assertion that I simply reply to people in order to try and make them feel inadequate or stupid, is humorous. One of the reasons I have been active on this forum for over a decade, is that there are lots of other developers who bring up libraries, frameworks and servers that they use, just as I brought up RabbitMQ and ActiveMQ in this thread. Rather than curse them for talking about something I wasn't familiar with, I simply put those things on a list to investigate when I have free time. It's a fairly common practice for most experienced deverlopers, in my experience.

More elaborate/scalable solutions to this problem usually involve a message broker, typically that implements publish/subscribe. Often used message brokers in the open source world are RabbitMQ and ActiveMQ. Short of that, as others have advised, the single server/KISS solution to this is indeed a command line script that only handles this particular job, as efficiently as possible, run in a cron job that executes every few seconds or at whatever frequency is acceptable to you. Cron will let you run a script every second, however, in practical use, a script like this that does not involve a queue of some sort, must query the database repeatedly with a date range, and there is of course overhead in this activity. You should insure that query is optimized and utilizes an index and returns in milliseconds. One thing you typically want to do however, is protect against multiple versions of the script running at the same time, which could lead to race conditions or database deadlocking, or any of a number of other issues. Fortunately, there is a real simple solution available on linux servers, of putting your call inside a bash script that uses the flock program to set a filesystem lock (see http://linux.die.net/man/1/flock)

Your problem has nothing to do with $_FILES, it has to do with move_uploaded_file. The purpose of that function is to move a file from what is essentially php's temporary holding location, and move it to a location of your choosing. Once you've moved it, it is no longer at the temporary holding location. Based on what your code is trying to do, you should do the move_uploaded_file call. Then use copy to make the copy of the moved file and place it in the same location.

Did you look at your markup? Is there data there? Are you sure that $fListData is an array, and has the data you expect it to have?

That is referring to a class variable $Form, which also happens to be an object. Consider this example: class Request { private $Form; public function setForm($form) { $this->Form = $form; } public function getFormInput() { return $this->Form->input; } } #assume we require() the definition of the Form class... $form = new Form(); $request = new Request(); $request->setForm($form); $input = $request->getFormInput();