Best way to test my PHP code for inefficiency?

[njdubois]

Hello community!  You may or may not have seen my post about Calling Godaddy out.  This post is expanding on an important point a admin brought up.  Am I 100% sure it isn't my code?  The answer is that I am not.

 

First off, some background.  We have this issue with our godaddy shared server where every 6 to 8 months there is a huge slow down, almost to the point where things are un-usable.  It's hard to believe that the code is causing this, wouldn't it happen much more often?  Why after more than half a year of smooth sailing does it show such a slow down?

 

My client and I had a long talk about this.  He brought up that it seems when ever we change something here, something over there stops working right.  I think that may be just not noticing something that wasn't working right, or just isn't working the way he expects it to.  Why would changing this, and not touching, nor changes made effecting this in anyway cause it to break?

 

About me, I started programming about 16 years ago.  I'm really good at teaching myself, mainly by actually getting into the code and doing it.  So I'm completely self taught.  I started with a GW-Basic chip that was on board my first computer.  Got windows and learned visual basic and c/c++.  Met this guy while I was working the drive through at mcdonalds, told him I'm a coder, we met and the rest is history.  Started with a vb.net/ms access platform, and I actually learned PHP/MYSQL/Javascript and AJAX over the past 5 years.  Maybe 3.5 to 4 years.

 

Again, completely self taught, and I don't know any other programmers.  I don't have any peer review and never have.  Is there a better way to do what I did?  I'll never know.  We arranged it so he is paying me according to the fact that I'm educating myself as I go.  So of course if I could find some way to confirm my code is great, that would make a good resume item!

 

So, going back to the reason of this post.  Is my code 100% solid?  I don't know.  

 

What are my steps for finding out?  Starting with the cheapest/free methods of auditing your code, up to the more expensive ways?  I'm sure I'll have offers from the community, please keep in mind that we don't have boat loads of cash.  So we have to try the cheap ways first, but please feel free to offer your services if your up to it.  There would have to be some kind of contract drafted, to protect our large database, passwords and of course my clients "idea," or "product?"

 

I also want to take this time to say thanks to the PHP freaks community.  If I can't find it on google, this has been the first and only place I ever go.  I've had zero bad experiences, everyone is extremely helpful, and if I knew other programmers I'd send them here in a heart beat! So thanks for that!

 

Thanks again!

Nick

[dalecosp]

If I recall from the other thread, part of the problem is that it's on shared hosting and you've got little control and almost no tools to work with.

Would it be possible to create a server (even a VM ... that might be perfect for the task), and run some of the standard tools on it (I mentioned some of them there).  I use Oracle's VirtualBox on my Win7 workstation @work ... works very well for development, and I can run whatever I want in it.

[mac_gyver]

Why after more than half a year of smooth sailing does it show such a slow down?

it seems when ever we change something here, something over there stops working right.

this is just a general throwing out ideas about those two symptoms.

 

if you have a database driven design that is on the edge of working/not working due to the amount of resources/processing/queries it uses, as the amount of data and/or the amount of visitors grow (or you are on a busy shared server where other accounts are also using the database resources), you will reach the point where the amount of processing time required exceeds the amount available.

 

a big inefficiency is running queries inside of loops (or running queries when you don't need to), because each query you run (or each prepared query you provide data for and run) requires that the query statement or the data to be sent from php to the database server and for most simple queries, the time taken just to communicate the information is much longer than the time it actually takes for the query to run on the database server.

 

insuring you are running only the queries you need and that those queries use appropriate indexes are the two biggest things you have within your control that affect how efficient your code uses database resources.

 

also, doing things like having frequent ajax requests that cause queries to be performed or perhaps having the same query executed frequently, but the database server doesn't have the query cache on or set to a large enough size or perhaps queries that return a huge result that fills the cache and other queries that could have used cached results must wait for the actual query to run and retrieve the data.

 

these and a bunch more things affect how much resources your code uses. specific help requires knowing exactly what your code/queries are doing and how frequently they are doing it.

Edited December 3, 2013 by mac_gyver

[njdubois]

Hello again!

 

I sure could setup a VM and test it.  You replied to my email problems post, I think your recommendations pertain to sending email.  SPF and IIRC.  I am planning on looking into SPF more, I think that is my solution to these email problems.  I think I have exhausted all other options.

 

What would you recommend I do after I get the VM setup.  This of course is going to take time, I have no clue how to setup a web host with mysql and php support and would have to invest time into that.  Once it is up, what tools/utilities should I look at?  Are these tools going to actually look at my code and say, you should do this part this way?  Or are they going to say this chuck of code took this long, you should look at it?

 

What are my other options?  Keeping in mind that this isn't a small "project?"  

For example, I was reading online and found this page:

http://php.dzone.com/articles/5-things-you-should-check-now

 

It covered some great ways to improve on your site, and I plan on using these pointers in the future.  The problem is I've got the whole thing up, thousands on thousands of lines of code with more tasks to do on the waiting list.  So I would have to start at the beginning and go through each page.  This isn't a problem, but if I am going to do that, I'd prefer to have a whole arsenal of possible problems/solutions to look at so I can do it all at once instead of constantly going through everything.

 

Is there a service or company that I could submit my code and for as low a price as possible they audit it.  Does such a company/service exist?

 

More importantly.  I've always pictured it as more black and white.  It works, or it does not work.  Right now, the code is working.  Most of the project works and has worked no problems other than these shared server slowdowns every 6 to 8 months.  Is there different grades of code?  IE, am I going to go through the code and find out I did things completely backwards and will shave minutes of time off operations?  Or regardless of my code, it "works" and am only going to shave a second here or there?

 

Thanks again for the reply!

 

Nick

[njdubois]

@mac_gyver :

 

Some of the stuff you mentioned was mentioned in the link I posted last.  Though I think you went into more detail.

 

Yes, There is all of the above.  One page in question is a calendar form.

 

I do the work to draw a month.  The last days of the previous month and the first days of the next month and all.  For each day, I pull data.  How many sales calls, total incoming calls, total appointments set and after hour calls are displayed on each "day" per each client.  I don't pull each day, I pull the range from the earliest day shown for the last month, to the last day displayed of the next month, if there is any.  I think its 5 "weeks" and the first and last week sometimes have days of the last and next month.  I then loop through the days and put the correct data where it needs to go.  This is one big ajax request, and AJAX returns the whole calendar HTML formatted and all.  You can click on a day and a listbox gets populated with the days calls in a little more detail.  Yet another AJAX/Mysql call.

 

I had thought that using ajax is all around a better way of doing things.  Instead of loading a whole page with every submit, I'm just loading a part. So I have applied it all over the place.  I learned javascript so I could use AJAX.  I may have been using javascript for a year now.  Maybe!  

 

I do have many many loops, both whiles and foreach.  All over the project.  Everywhere.  This is exactly why I would like to have someone a few levels higher than me on the programming ladder to critique my code.  I am sure there is a billion ways to improve it and I am all about self improvement.  These loops maybe 1 of many things I could do better.

 

I'm not afraid of doing the work, I just need to know what to do.  If someone said Nick, do this instead, I would go through my code and replace what needed to be replaced.

 

I wish I went to college.  I was leaving high school acing an AP Computer Science class.  I aced it without trying.  Unfortantatly I screwed my credit up....with student loans..... so it isn't an option right now.  On top of that, I need solutions, I don't have the option to put this project on hold while I take 4 years of college.  I think my best and only choice is to cough up what little cash I have to pay someone to audit my code.

 

Keep the suggestions coming!  As always I'm never pointed in a wrong direction by posting here!

 

Thanks

 

Nick

[kicken]

Is there different grades of code?  IE, am I going to go through the code and find out I did things completely backwards and will shave minutes of time off operations?  Or regardless of my code, it "works" and am only going to shave a second here or there?

Yes, there are various levels of working code. As mentioned above, one of the biggest issues new coders have is running queries in loops. Depending on how much data one is dealing with correcting this to use a proper JOIN setup could barely have any effect, or cut a process that takes minutes down to seconds.

 

Creating proper indexes on your tables and using the proper datatypes for your columns can also have a noticeable impact on your run time. If you were say querying a large table for rows falling between two dates, but have the date column un-indexed such a query could take minutes to complete. Adding an index onto that date column could result in the same query taking only seconds.

 

Without seeing any of your code we can't really give you any specific thing to go and do, only recommendations as to what to watch out for. One of the tools that would help you identify issue would be to setup your site with something like the XDebug Profiler and browse the site, then check the profiler output. This will help you identify which areas of your code are taking the longest to run, then you can look at those areas and try to figure out why they are taking a long time.

[njdubois]

Creating proper indexes is something I've seen mentioned a few times now.  What does this term mean?  I have an auto incrementing ID field that is unique per each record, is that what indexing is?

 

Most of my field data types are "text" and all ID fields are long int.  

 

So, per your last post, and all other posts so far.  I need to look at:

 

My loops, and how they handle pulling data.

 

Pulling data in general.

 

My field Data types.

 

indexing my databases.

 

and setting up a service like XDebug Profiler.

 

Keep the advice coming!  Thank you so much!

 

Nick

[kicken]

Creating proper indexes is something I've seen mentioned a few times now.  What does this term mean?  I have an auto incrementing ID field that is unique per each record, is that what indexing is?

An index is something you set on certain columns to help the database search those columns for information. Your primary keys are one index, but depending on your data usage you may want to add more. For example in a blog system you would want to add an index on the date that an entry is published. Having this index allows the database to quickly filter or sort records based on the date. For the most part, and fields that you use often in either a WHERE condition or in a join's ON condition should be indexed.

 

Keep the advice coming! Thank you so much!

If you have some pages that are noticeably slow consistently then you could use those pages to start learning out what to look for. One quick-n-dirty way to figure out what part of the code is being slow is to call microtime before and after a block of code, then subtract the two values to get the time taken to run. Once you figure out which part of the code is taking the longest to run, look over it and try to figure out why. You can post the code here for people to review if you want. If the code involves and DB queries, posting the relevant table structures would help as well.

 

This is a function I wrote a while back to help make timing code easier:

function markTime($lbl=null, $finish=false){
    static $points=array();
    static $collecting=false;

    if ($finish && $collecting && count($points) > 0){
		$collecting=false;

        if ($lbl == null){ $lbl = count($points); }
        $points[] = array($lbl, microtime(1));

		$firstPoint = reset($points);
		$lastPoint = end($points);

		ob_start();
		printf("<div class=\"timerBlock\">\r\n");
		printf("\t<p>Total Time: %0.3f seconds; Points: %d</p>", $lastPoint[1]-$firstPoint[1], count($points));
		printf("\t<p>Points:</p>\r\n<ol>\r\n");
		$previousPoint=null;
		foreach ($points as $p){
			printf("\t<li>%s\r\n\t<ul>\r\n", $p[0]);
			printf("\t\t<li title=\"Since Start\">%0.3fs</li>\r\n\t\t<li title=\"Since previous\">%0.3fs</li>\r\n",
				$p[1] - $firstPoint[1],
				($previousPoint)?$p[1] - $previousPoint[1]:'-'
			);
			printf("\t</ul>\r\n\t</li>\r\n");
			$previousPoint=$p;
        }
		printf("</ol></div>");
		$content = ob_get_contents();
		ob_end_flush();

    }
    else if (!$finish){
        if ($lbl == null){ $lbl = count($points); }
        $points[] = array($lbl, microtime(1));
		if (!$collecting){
			$collecting=true;
			register_shutdown_function(create_function('', 'MarkTime("Finish", true);'));
		}
    }
}

You can just call the function at various points in the script. At the end of the script it will output a list of each point and the time difference. For example:

MarkTime('Start of loop A');
for ($i=0; $i<1000; $i++){
   $x = new DateTime();
}
MarkTime('End of Loop A');
for ($i=0; $i<2000; $i++){
   $y = new stdClass();
}
MarkTime('End of loop B', true);

[mac_gyver]

here's a thought concerning random unexplained page wait/hanging that seem to come/go related to code changes. if you are using session variables and have multiple requests made to the server by the same visitor's browser session, doing things like ajax requests to pages that have session start statements or dynamically producing and outputting images/media files that also have session start statements in the code, you may be having a session data file locking problem, where the visitor's session data file is being accessed by one request and additional requests wait/hang until the first request finishes and releases the lock on the session data file.

[ignace]

If you can run your code locally, you can use these tools to analyze your code:

http://phpqatools.org/

 

Install xdebug and enable profiling, go through your application and it'll show you how long each call took and how often they are called. You need WebGrind or something similar to view the profiler output. Perhaps you have a function that is called many times, simply caching it's results will speed up your website.

 

function simple_cache_example($foo) {
  static $cache = array();
  
  if (isset($cache[$foo])) {
    return $cache[$foo]; // already calculated
  }

  // do calculation
  $cache[$foo] = $calculated;
}

Enable the query log and set it to table. Go over your application and view the executed queries in mysql.general_log, perhaps you'll see that one query is executed several times though it's result does not change:

 

SELECT .. FROM some_table WHERE something = 'this';
..
SELECT .. FROM some_table WHERE something = 'this';
..
SELECT .. FROM some_table WHERE something = 'this';
..
SELECT .. FROM some_table WHERE something = 'this';

Cache the result from the query as shown in the above function.

 

Same for simple variations:

 

SELECT .. FROM some_table WHERE something = 'this';
..
SELECT .. FROM some_table WHERE something = 'that';
..
SELECT .. FROM some_table WHERE something = 'there';
..
SELECT .. FROM some_table WHERE something = 'maybe';

Simply combine it and return the correct result on each consecutive call:

 

SELECT .. FROM some_table WHERE something IN('this', 'that', 'there', 'maybe');

..

if ($cache[$what]) {
  ..
}

You may not have these tools available due to the shared hosting.

 

APC:

Caches the PHP opcode, so your scripts no longer have to be 'compiled' between requests.

It can also be used to cache your own data, like large result sets.

apc_store

apc_fetch

 

Varnish:

Caches your website pages and thus reduces hits to your database. For pages with partial dynamic content you can use edge side includes which are fetched by varnish and placed ad-hoc.

https://www.varnish-cache.org/trac/wiki/ESIfeatures

 

This is simply scratching the surface of the things you can do to speed up your application/website.

Edited December 3, 2013 by ignace