Work website

[Andy-H]

My boss is off on paternity leave at the moment and is the only person with access to our works website host, amongst many other things.

 

 

My job title is "Junior web developer" but because I am a programmer I am expected to know "everything" about computers and the people I work with are turning to me to fix the problem; which my boss has known of for a long time, and hasn't found a solution.

 

 

Basically, were on our internal network and all requests are filtered through a single switch (I assume, as we all have the same IP address to "the outside world"), now, sometimes one person will complain that the website will not load, while others load it perfectly, other times, everyone complains that the website won't load or is running "like shit". I don't have access to the code but have access to the router, I have assumed for a long time that the lag is caused by poorly written code, but the symptoms suggest otherwise (unless there is a session-dependant bottleneck???). Is there anything I can do to diagnose this problem and get it sorted once and for all.

 

 

Any help greatly appreciated.

[Andy-H]

Just realised that I do have access to the scripts, it's awful, SELECT *'s all over on tables with around 40 columns when only 2-6 columns are used, then mysql_fetch_array used in conjunction with this, database is about as far from normalised as I've ever seen.

 

 

I have written a CRM when I first started here, normalised database, clean code, slick AJAX UI but my boss doesn't want to launch it until he has finished the tracking side of the system.

 

 

I still don't get why it's only affecting some users though? Surely it would affect everyone? Can previous script calls affect the current call (in relation to speed, bottlenecks)? Are there any legacy known memory leaks in PHP I could check for?

 

 

Surely if it was down to the bad programming, everyone would be complaining?

[scootstah]

Are there any errors in the Apache error log?

 

What does "not load" mean? Do they get a 404? A 500? Nothing? Can they ping the website? Does it never work for those people or only some of the time not work?

[Andy-H]

About a million and two warnings.

[PFMaBiSmAd]

The mysql query cache would allow repeated/exact same queries to immediately return data, where as a new and unique query would need to wait for the actual query to execute. This could give the appearance that it works OK for a group of visitors, but not others.

 

When multiple clients behind a router make requests to the same server, the requests are actually being made using a different remote port number (see the $_SERVER['REMOTE_PORT'] value) for each client. The server sends the response back to that same port number. About the only things that would be communication related would be any caching of content (or lack there of) and the amount of content that is being output on each page request (i.e. large images, huge amounts of html/content.)

 

Other than the time it takes to generate the page on the server, about the only thing that comes to mind that causes php produced pages to 'hang' are when you are using session variables on a page and content (images, ajax) on the page is also making http request for the same session and because of file locking on the server, the requests hang until the specific session data file can be accessed.

 

I would add a calculation of the page generation time and either display that or log it to see if the actual pages are taking a long time to generate or if the problem is somewhere else.

[PFMaBiSmAd]

About a million and two warnings.

 

Code should not normally produce any kind of php error, warning, or notice during its normal execution. ONLY for abnormal things, like an actual error occurring, a legitimate visitor doing something that your logic didn't take into account, or a hacker trying to break in. Those are the only time you should see messages in the log file, so that you can find and fix the condition and the logic to address those errors.

 

Note: HIDING the errors by altering php's error_reporting and/or display_errors/log_errors settings does NOT fix anything. In fact, the message from the reporting and display/logging of the error is just the step in the error response code. Php must still detect and handle each error as it occurs every time the script runs, even if the final message is not reported and displayed/logged.

 

It sounds like the code needs a complete audit.

[Andy-H]

Thanks, learned quite a lot from that, I will create a log for page load time depending on remote port tomorrow and check that out, see which pages are taking too long to load.

 

 

I didn't know that about the file locking but will check the sessions that are accessed simultaneously too.

 

 

I know about the errors but when my boss wrote this he was learning PHP out of books and programming the system as he went along - I think he was a little embarrassed for me to see the code lol

 

 

The only problem is, my boss is a company director, and I'm not sure if he want's me to mess with the code as he hasn't said anything about it thus far, and was quite reluctant to give me access, at the same time, I have 14 people shouting "the system's running slow, it won't load, FIX IT" in my direction lol

 

 

The only other issue that comes to mind is earlier today, people were using the system without a problem, but one guy was getting excessive hang time, when requesting the login script, he got a http error code, I think it was a 404 (not sure as he had a customer on the phone at the time so I needed to be quick), when I opened another browser tab and directed it to the same URL (which was index.php), it loaded instantaneously, which I found a little strange.

 

 

What would you do in my situation? I mean, he's my boss but he's also a mate, so I don't want to go round pointing his flaws out to everyone and cause unnecessary embarrassment, but the issue needs resolving, and has done for a long time.

 

 

[Philip]

I'd personally not say anything about flaws, but something more along the lines of "I'd like to help improve the website by doing some performance tweaks to the code base which would cut down on the number of help [tickets/emails/etc]"

[PFMaBiSmAd]

If this problem has existed for some time, it's long past the point where someone with sufficient php programming and troubleshooting skill should have looked at it. It won't fix itself and it will only get worse if it is due to things like a database design that is inefficient.

[Andy-H]

So do I wait for him to get back from paternity leave then ask him? Or just go to another director and say "I think I can make the system more efficient, should I?" or just do it and when he get's back tell him?

[Philip]

How long until he gets back? If it is like months then I'd go to someone else, but if its just a few days, I'd wait.

[Andy-H]

Maybe another week

[Philip]

In that case, I'd wait until he gets back to talk to him about it. Otherwise you might create extra tension for what he might see as going behind his back.

[Andy-H]

Ok thanks

[PFMaBiSmAd]

Some other suggestions.

 

Make sure that php's error_reporting is set to E_ALL, so that all the php detected errors will be written to the error log. You could be missing fatal runtime errors and notices that would help find the cause of the problem.

 

The code itself needs to have error checking (check if function calls work or not), error reporting/logging (output a user message and log all the information about any errors that are occurring), and error recovery (take an appropriate action for the seriousness of the error that occurred) logic in it.

 

For example, if the database server is down or you have exceed all the available connections to the database server, when the code attempts to make a database connection and it fails -

 

1) Test the return value from the statement making the database connection,

 

2) Output a user message - 'Sorry, an error occurred that prevents this page from working at this time!'

 

3) Write your own application level information to your own Log file (you want to use a file because if the server/php/page is working at all, the file system is working, but the database itself might not be) with all the available information about - who the user is that requested the page (assuming you have a log in system) and/or the IP address (in case it is a hacker triggering errors), what happened ("DB connection failed " . mysql_error()), when it happened (date and time), where it happened (the file name and line number), anything else that is relevant to the error that would help find what is causing it.

 

4) Don't continue executing the code on that page. For something like a database connection that is required for the page to do anything useful, you might as well use an exit/die statement to prevent the remainder of the code on the page from running or you might want to at least display a minimal page with some navigation link(s).

 

In step #3, for something like a query that failed with an error, you would also want to log the actual query. For something like a file operation that failed, you would want to log the filepath/filename that could not be accessed.

[Andy-H]

When PHP writes an error to the log does it have to load the error log file first like an fopen? because the error log is 4 gig and I think that it could be when it encounters an error that the hanging occurs?

[PFMaBiSmAd]

The error log is simply append to when additional messages are added. However, if its size it at the limit of what the operating system supports for one file, there are likely errors occurring when the operating system attempts to append to it.

 

At this point, past information in the error log is probably not doing anything to help you. You might save it as a different name and start a new empty log file.