allworknoplay Posted January 21, 2008 Author Share Posted January 21, 2008 Debugview won't really be of much use. You need TCPView EDIT - oops didn't see page 2!. Now see you are using Debugview on your API so ignore above I am guessing that Shell-exec on a heavily hammered webserver would lead to potential resource issues. Particularly if you get a stuck or delayed thread elsewhere at the O/S level due to other componentry involved with the exec'd program. Once you exec synching events must be a pain since you have minimal or no control over those child processes unless you're very careful how they're excec'd. Are you monitoring or producing any graphs of spawned procs active on the server at any given moment? Might be illuminating. I know ppl use it a lot and get no probems with very simple shell calls but my instinct would be to avoid it if poss. esp when called from a script language which may already have interaction issues and complex heirarchies of dependencies to track/trace/debug. Can you not get PHP to do the job? I have yet to turn on debugging on the apache logs which I think I should have... I think I have have found the problem and I think it was 2 things. 1) shell_exec's is a no no... I was doing so much work on my API program that I think resource wise, using shell_exec's just wasn't a good idea. 2) NFS mount. The data was being populated in another directory that was NFS mounted to another server. The NFS mount I don't think was able to handle the volume of the traffic because I found that the server with NFS wasn't being very responsive... So what I am going to do right now is have my API write the code locally to a drive which would remove the NFS issue. Then use SCP to copy the data over to our storage server once a minute. The truth is, I didn't need to constantly have an NFS mount and put the data remotely so much. Copying it once a minute over to the storage server would be just find.... But back to NFS, I think the "chatty" nature of constantly using the mount just couldn't handle it.... Again, I'm no expert in NFS, shell_exec's or anything like that but I did hear people swear by NFS so I thought it would work for me.... I should have SCP completed today so I can report back to you if NFS was an issue at all... Quote Link to comment https://forums.phpfreaks.com/topic/86636-access-log-help/page/2/#findComment-445587 Share on other sites More sharing options...
allworknoplay Posted January 22, 2008 Author Share Posted January 22, 2008 Ok I think it was a success!!! So it wasn't a shell_exec issue but NFS mount... I needed to find out what exactly the issue was so I put my script back to using shell_exec and got rid of my NFS mount. I made my API dump the data to a local directory and in the last 12 hours I got ZERO errors and not a single "200 -" Watching the access_log stream this morning brought tears to my eyes...everything looked perfect!!! I don't want to jump to the conclusion yet but I think the issue is mostly resolved. I don't know that much about NFS but apparently either the protocol itself or the server wasn't able to handle the workload... oh and here's something else that is interesting, on both my servers when I unmounted the NFS, the load dropped significantly!!! Before the load was about: 28, 30, 30 on both servers...when I did a TOP. Now they are both at zero.... 0.5, 0.20, 0.20 Amazing!!! Quote Link to comment https://forums.phpfreaks.com/topic/86636-access-log-help/page/2/#findComment-446119 Share on other sites More sharing options...
steviewdr Posted January 22, 2008 Share Posted January 22, 2008 NFS mustn't have been configed correctly. > Before the load was about: 28, 30, 30 on both servers...when I did a TOP. Strange you didnt spot that before now. Glad you got the problem traced. -steve Quote Link to comment https://forums.phpfreaks.com/topic/86636-access-log-help/page/2/#findComment-446324 Share on other sites More sharing options...
allworknoplay Posted January 22, 2008 Author Share Posted January 22, 2008 NFS mustn't have been configed correctly. > Before the load was about: 28, 30, 30 on both servers...when I did a TOP. Strange you didnt spot that before now. Glad you got the problem traced. -steve Yeah it would have been nice to be able to do benchmarks and then compare when an issue arises... Of course that is hindsight....everything looks pretty good so far...just amazing.... Quote Link to comment https://forums.phpfreaks.com/topic/86636-access-log-help/page/2/#findComment-446384 Share on other sites More sharing options...
madmax Posted January 24, 2008 Share Posted January 24, 2008 Glad you found your needle!! Quote Link to comment https://forums.phpfreaks.com/topic/86636-access-log-help/page/2/#findComment-447851 Share on other sites More sharing options...
allworknoplay Posted January 26, 2008 Author Share Posted January 26, 2008 Glad you found your needle!! Thank you madmax for taking the time to help troubleshoot this with me. Sometimes it just takes someone else to bounce back ideas on before you figure it out... Quote Link to comment https://forums.phpfreaks.com/topic/86636-access-log-help/page/2/#findComment-449984 Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.